This document has been archived. Title : ASSESSMENT OF THE NATIONAL SCIENCE FOUNDATION'S 1988-90 UNDERGRADUATE FACULTY ENHANCEMENT PROGRAM Type : Report NSF Org: EHR Date : September 1, 1993 File : nsf93122 ASSESSMENT OF THE NATIONAL SCIENCE FOUNDATION'S 1988-90 UNDERGRADUATE FACULTY ENHANCEMENT PROGRAM EXECUTIVE SUMMARY Prepared for the National Science Foundation by Westat, Inc., Rockville, Maryland September 1992 EXECUTIVE SUMMARY The Undergraduate Faculty Enhancement Program (UFEP) was developed by the National Science Foundation (NSF) to meet the needs of faculty members who teach undergraduate students. NSF recognizes that the faculty members most closely associated with undergraduate teaching often have limited opportunities to become familiar with new concepts and have limited access to the laboratory resources of the advanced research community. These faculty members need help in gaining access to the new developments and instrumentation, as well as knowledge of how these developments and instrumentation may be incorporated into undergraduate courses and laboratories. They also need opportunities to interact with colleagues and experts in the field. UFEP was designed by NSF to provide these opportunities for undergraduate teaching faculty. Under UFEP, NSF makes grants to project directors at colleges and universities, professional societies, industry, and other qualified organizations to conduct regional or national seminars, short courses, workshops, conferences, or similar activities for groups of faculty members in the sciences, mathematics, and engineering. These activities must be designed explicitly for their capacity to enhance participants' teaching activities for undergraduate students. The emphasis of the project must be on the active involvement of the participants in working with the topic of the project and in interaction with experts in the field and fellow participants. Through these workshops, NSF also seeks to encourage the increased participation of underrepresented groups in science, mathematics, and engineering. Assessment Procedures In September 1990, NSF awarded a contract to Westat, Inc., of Rockville, Maryland, to assess the effectiveness of UFEP during its first 3 years of operation. This included projects conducted during 1988, 1989, and 1990. This report presents the principal findings based on data collected through mail questionnaires completed by 91 out of the 92 eligible UFEP project directors, and telephone interviews conducted with a sample of participants in UFEP projects. A 97 percent response rate (469 interviews) was obtained during the telephone interviews. Data from the participant interviews are weighted to represent the universe of UFEP participants. Certain atypical projects were excluded from the main data collection. Excluded were four conferences to which UFEP contributed funds, one faculty-mentor project, and the very large projects conducted by the American Society for Engineering Education and the National Chautauqua Workshop Program. These two large projects were excluded because their size and variability would have allocated too much of the participant sample to these projects and reduced the precision of the estimates based on participant responses. Each of the excluded projects is discussed in Appendix A of the report. An Assessment Advisory Committee was appointed to provide assistance in the design of the assessment and the development of the questionnaires, and in the interpretation and presentation of the findings. The Advisory Committee also wrote the interpretive overview that appears with the report. Additional information for this interpretive overview was obtained during visits to professional society meetings that grew out of the Advisory Committee's concern that feedback about UFEP be obtained from appropriate members of the disciplines who are not directly connected with UFEP. The Advisory Committee was interested in evaluating the effect of UFEP within the larger context of the disciplines, seeking to answer the general question of whether UFEP was meeting the needs of the disciplines. During the professional society meetings, discussion sessions were also held with groups of faculty members who had been participants in UFEP projects. Discussions with participants focused on issues such as their successes and problems in using and implementing what they had learned at the project when they returned to their home institution; the strong and weak points of the projects (e.g., length, type of activities, followups after the workshop ended); and suggestions to NSF about how to improve UFEP in the future. Insights from these discussions with participants are included in the interpretive overview and conclusion sections of the report. Projects Supported by UFEP During the first 3 years of operation (FY 1988 - FY 1990), the Undergraduate Faculty Enhancement Program awarded approximately $6.7 million to 92 projects, each of which supported one or more workshops or short courses serving an estimated 2,890 participants. This report is based on responses provided by project directors to a survey questionnaire and by participants in the workshops and short courses to a telephone interview. UFEP also contributed $127,100 toward 4 conferences, which were attended by about 1,150 people, and $61,400 toward a Faculty-Mentor project for 9 faculty members run by the American Society for Microbiology. In addition, UFEP contributed approximately $1.1 million in operating funds to the American Society for Engineering Education Faculty Professional Development Program and the National Chautauqua Workshop Program for courses in 1988-90, which together served about 3,400 people. These projects are described in Appendix A of the report. Project Characteristics and Activities Workshops and short courses varied substantially in length, ranging from a few days to 4 weeks. Almost half of the projects (45 percent) had workshops that lasted 1 week or less (most of these were either 5 days or 1 week); about one-third of the projects had workshops that lasted between 1 and 2 weeks (a couple of projects were 10 days, and most were 2 weeks); and the remaining 22 percent of projects had workshops that lasted more than 2 weeks (ranging from 17 days to 4 weeks). Projects in these categories of duration served 65 percent, 22 percent, and 13 percent of the participants, respectively. Projects also served varying numbers of participants; a few projects had fewer than 10 participants, while a couple of larger multiyear projects served 200-300 participants each through multiple workshops and short courses. About one-third of the projects were in each of the categories of fewer than 20 participants, 20-29 participants, and 30 or more participants. Almost all of the 91 projects included lectures or seminars (90 projects) and laboratory or computer sessions (82 projects) during their workshop sessions. Other activities used frequently during workshop sessions were small discussion groups (67 projects), participant presentations (54 projects), and participant projects (49 projects). Project directors were also asked to select up to three activities that were their major workshop activities. Most frequently mentioned as "top three" activities were lectures or seminars (81 projects), laboratory or computer sessions (73 projects), participant projects (31 projects), and small discussion groups (29 projects). Almost all projects (89 out of 91) had followup activities or continuing contacts of some kind with project participants. The most frequent followup activities were sharing of materials among participants (58 projects), continuing technical assistance to participants (55 projects), sending written reports from participants to the project director (49 projects), and organizing informal group get-togethers (48 projects). Only 23 projects had followup activities that involved formal group sessions at scheduled times as a continuing part of the project. Project directors in UFEP are free to recruit and select participants in the ways they see as most appropriate, as long as they follow the general guidelines established by NSF. The most frequently used recruitment strategy, reported by the directors of 71 projects, was to send program announcements to department chairs or deans, and ask them to bring the program to the attention of faculty members who might want to participate. Other frequently used strategies were sending posters or brochures for posting in departments and placing announcements in newsletters or journals, both used by the directors of 51 projects. When asked whether they tried to increase the number of applications received from members of various target groups, directors of 65 of the projects indicated that they targeted one or more groups for recruitment. The directors of more than half of the projects (54 of 91) tried to increase the number of minority faculty who applied, and the directors of 44 projects tried to increase the number of women faculty who applied. Faculty members from 2-year colleges were targeted by directors of 22 projects; directors of 5 projects targeted faculty members with physical disabilities. The most common recruitment approach used by project directors who tried to increase the number of applications received from specific groups was to target mailings or recruitment to certain kinds of faculties or schools; directors of 43 projects used this approach. Similarly, directors of 21 projects seeking certain groups of faculty members for their projects used advertising or application materials that included a statement indicating that applications from these groups were encouraged. The only other approach mentioned frequently was direct contacts (e.g., calls or visits) to department chairs or others at a school. Most projects were staffed with other members of the project director's institution. Among the 77 projects in this group, 67 used other faculty members as project staff; other people in the institution, such as graduate students, were used by less than half of the 91 projects. More than half of the projects (54 out of 91) had project staff that came from outside the project director's institution. The most frequent source of these additional project staff was other colleges and universities, used by 45 projects; people from industry, used by 17 projects; people from professional organizations or societies, 15 projects; and people from government agencies, 12 projects. Participant Information and Support Participants learned about the faculty projects they attended in a variety of ways. The most frequent means was through a flyer, poster, or letter put up in the department or circulated by faculty mail (42 percent). Other important sources of information about the projects were direct mailings received by the participant (23 percent), and notices in a journal or newsletter, or at a professional meeting (19 percent). The issue of who paid the participant's travel costs to the faculty project is of interest since the UFEP program guidelines indicate that the home institution of the participant is expected to cover these expenses. This is one way that institutions can demonstrate their support for their faculty's participation in such programs. However, only 64 percent of participants indicated that their home institution or department paid for their travel, while 22 percent of participants paid for travel themselves. Department chairs or deans could demonstrate interest and support for their faculty's participation in other ways, as well. Approximately three-quarters of the participants indicated that their department chair or dean expressed a great or moderate degree of interest both before and after their attendance at the UFEP workshop. Over half (55 percent) of participants indicated that after they returned from the faculty project, either they or their department chair or dean had purchased, or applied to purchase, equipment or instructional materials related to the project. Participant Outcomes Participants were asked to indicate how valuable or worthwhile the project was, overall, using a 5-point scale, with 1 being "not at all valuable or worthwhile" and 5 being "very valuable or worthwhile." The perceived value of the projects was very high, with 45 percent of participants giving the project they attended a rating of 5, and an additional 40 percent of participants giving their project a rating of 4. Using the same 5-point scale, participants were asked to indicate how valuable or worthwhile certain activities and resources associated with faculty projects were to them. Participants could also indicate that certain activities or resources were not applicable to their project. Activities and resources associated with the faculty projects were generally perceived as being quite valuable or worthwhile. Most noteworthy for their frequency (i.e., most projects had them) and their very high ratings (ratings of 4 or 5 by threequarters or more of participants) were interactions with instructors, informalinteraction with other participants, content of the lectures or seminars, hands-on learning activities, and materials to be taken back to their school. Participants were asked to indicate whether they had engaged in a variety of activities since returning from the faculty project, regardless of the relationships of those activities to the faculty project attended. When asked to indicate the extent to which their experience at the faculty project influenced their work, more than 80 percent of participants indicated that they had modified their teaching methods (81 percent), acquired new equipment, materials, or computer software for undergraduate courses or laboratories (86 percent), introduced new content into existing undergraduate courses or laboratories (93 percent), and incorporated equipment, materials, or computer software into undergraduate courses or laboratories in ways in which they had not previously been used (81 percent). In addition, a majority of participants developed new undergraduate courses or laboratories (62 percent) and participated in formal programs designed to develop curriculum or improve instruction (63 percent). With the exception of participation in curriculum development programs, most participants who engaged in these courserelated activities indicated that their activities had been moderately or strongly influenced by their experience at the UFEP project. In addition, most participants (92 percent) indicated that they had shared with colleagues new materials or skills acquired through the UFEP project. Most of these participants said that they had been moderately to strongly influenced by their experiences at the faculty enhancement project they attended. Almost all participants (95 percent) had attended professional meetings, seminars, or workshops, and most (81 percent) had gained competence in a new area of their own or another discipline. Although about half (53 percent) of the participants subsequently delivered a paper at a professional meeting, and 43 percent submitted an article to a professional journal, the participants who engaged in these professional activities were not strongly influenced to do so by experiences at the faculty enhancement project. Faculty members can benefit in a number of ways from participating in these faculty development projects. Participants were asked to indicate the extent to which they derived certain benefits from their participation, using another 5-point scale, with 1 being "not at all" and 5 being "very much." Benefits that received the highest ratings were increased knowledge of the field, given a 4 or 5 rating by 76 percent of participants; personal growth or renewal and increased motivation or stimulation for teaching excellence, each given a 4 or 5 rating by 73 percent of participants; and increased contacts with colleagues from other institutions, given a 4 or 5 rating by 64 percent of participants. One kind of activity encouraged by UFEP is the introduction of new or innovative technologies relevant to undergraduate teaching responsibilities. Participants were asked whether the faculty project they attended introduced them to technologies that were new to them, or that they had never previously had a chance to try. More than three-quarters (78 percent) of participants indicated that their faculty project had involved such new technologies, and almost half (47 percent) of these participants indicated that the new technologies were very useful or applicable to their undergraduate teaching responsibilities, with an additional 43 percent of that group indicating that the technologies were somewhat useful for their teaching. Almost all participants (91 percent) had either followup activities or other project contacts after completing the UFEP workshop. Over half of participants reporting such activities indicated that they had correspondence or calls with the project director (52 percent), and that there had been meetings, conferences, get-togethers, or workshops with the project director or other participants (52 percent). Approximately a quarter of participants with followup activities or contacts reported receiving a newsletter (29 percent), corresponding with or calling other participants (28 percent), and maintaining informal contacts with other participants or the project director (24 percent). Conclusions The Undergraduate Faculty Enhancement Program appears to be fulfilling its mandate quite well; that is, it is assisting undergraduate faculty members to learn new ideas and techniques in their fields and to use the knowledge and experience to improve their undergraduate teaching abilities. The project offerings have been enthusiastically received by faculty members, who indicate that they have received substantial benefits from the projects for themselves personally and for their teaching. The projects have provided exposure to new ideas and technologies, which have strongly influenced the introduction of new content and equipment into undergraduate courses. The projects are also reaching groups that have traditionally been underrepresented in the sciences, mathematics, and engineering, although there is room for improvement in this regard. Program guidelines encourage projects in which participants develop instructional materials that include new ideas and techniques. While this was not measured directly, most participants indicated that, since returning from the faculty project, they had introduced new content into existing undergraduate courses or laboratories; acquired new equipment, materials, or computer software for undergraduate courses or laboratories; incorporated equipment, materials, or computer software into undergraduate courses or laboratories in new ways; modified their teaching methods; and developed new undergraduate courses or laboratories. In addition, most participants indicated that they had shared with colleagues new materials or skills that they had acquired. Participants indicated that these activities had been moderately to strongly influenced by their experiences at the faculty project. The UFEP program guidelines also specify that projects should permit participants to obtain personal experience working with new ideas and techniques, rather than just hearing about them. UFEP projects were successful in this regard: most of the projects included laboratory or computer sessions during the workshop, and half of the projects included participant projects of some kind as a workshop activity. Three-quarters of participants rated the hands-on learning activities in the projects they attended as highly valuable or worthwhile. UFEP program guidelines also encourage projects that enable participants to work with and evaluate innovative technologies relevant to their academic responsibilities. UFEP projects also accomplished this goal: three-quarters of participants indicated that the project they attended introduced them to technologies that were new to them, or which they had never previously had a chance to try. Of the faculty introduced to new technologies, 90 percent rated these technologies as very or somewhat useful or applicable to their undergraduate teaching responsibilities. Projects that allow participants to work with industrial scientists, mathematicians, and engineers are also encouraged by the program guidelines, and one in five UFEP projects used people from industry as part of their project staff. Participants perceived the UFEP projects to have been highly beneficial. The overall evaluation of the project they attended was very high, with 85 percent of the participants indicating that the project was highly valuable or worthwhile. The activities or resources rated as most valuable or worthwhile were interactions with instructors, informal interactions with participants, content of the lectures and seminars, hands-on learning activities, and materials to be taken back to their school. The value of interactions with instructors and other participants was especially apparent during the conversations with participants held at the professional society meetings. Repeatedly, participants said that the chance to interact with faculty colleagues was a very important part of the workshops, and that many of the good ideas about how to apply what they were learning in the workshops came from these interactions. The intensive, residential nature of the workshops was highly beneficial in this regard. The extent to which specific benefits were perceived to have been obtained through the project were particularly high for increased knowledge of the field, personal growth or renewal, increased motivation or stimulation for teaching excellence, and increased contacts with colleagues from other institutions. These benefits reflect those that NSF intended when it established UFEP: to have undergraduate teachers who are up to date in their knowledge, excited about their disciplines, and regard the teaching of undergraduates as important and rewarding. UFEP served its intended target audience of "faculty who teach primarily undergraduate students" (UFEP Program Announcement and Guidelines for projects beginning in 1989). Almost half (47 percent) of the faculty participants were from academic departments where the highest degree granted was a bachelor's degree; an additional 18 percent of participants were from departments where the highest degree granted was an associate's degree. Almost all (89 percent) of the participants had recently taught introductory courses, and 75 percent had recently taught upper division undergraduate courses. Thus, the faculty served were indeed those who teach primarily undergraduate students. The faculty served were also relatively removed from their own graduate training -- over a third (37 percent) of participants had received their highest degree more than 20 years ago, and an additional 33 percent had received their degree 11-20 years ago. These participants may be especially able to benefit from exposure to new theoretical developments and instrumentation. UFEP also targeted and served groups that have been underrepresented in science, mathematics, and engineering, although there is room for improvement in this regard. According to the National Center for Education Statistics (1991 Digest of Education Statistics), underrepresented minorities (i.e., all minority groups except Asians) constituted 2 percent of engineering and 3 percent of natural sciences full-time regular instructional faculty in 1987-88; women constituted 2 percent of the engineering faculty and 17 percent of the natural sciences faculty. In UFEP, however, 6 percent of the faculty participants were from underrepresented racial or ethnic minority groups, and 21 percent were women. Faculty from Historically Black Colleges and Universities (HBCUs) constituted 6 percent of the UFEP participants, although they were only about 2 percent of science and engineering faculty nationwide. Faculty members from HBCUs attended 48 percent (44 of 92) of the UFEP workshops Thus, underrepresented minority and women faculty participated in UFEP at a slightly greater rate than their rate of employment as faculty members in science and engineering, and they were widely dispersed across the UFEP projects. One point that emerged from discussions with participants at the professional society meetings was that travel costs may present a problem for some participants and potential participants. While NSF expects that the home institution will pay travel costs for the participant, only 64 percent of participants had their travel paid in this way. Lack of travel funds was also the most frequently given reason for not attending followup activities. Discussions with participants revealed that for many, travel was limited to workshops within driving distance of their home institution, either because this was the only travel for which their department had funds, or because they were paying for the travel themselves. Some relied on the stipend they received from the project to help defray their travel expenses. Since the participants with whom these discussions were held were those who had managed to secure funding (or provide it themselves) to both the workshop and a professional society meeting, it is likely that travel funds were an issue for others, especially for potential participants who did not attend a workshop. For some faculty, lack of travel funds may act to deter participation, or at least to limit the choice of workshops to those geographically close to the home institution. Another point that emerged from the participant discussion sessions was that workshop length may be an important consideration for many participants and potential participants. While most participants felt that the workshop they attended was the right length for the amount of material covered (i.e., that the project directors had done a good job of matching length and content), many participants expressed a preference for 1-week workshops. While longer workshops may have some benefits associated with them, such as being able to cover more material, these participants felt that longer workshops deter attendance, especially of teachers from community colleges (with multiple teaching assignments and year-round sessions) and those with family responsibilities. Short workshop sessions were the norm for these UFEP projects, with 45 percent of the projects having workshop sessions ranging from 5 days to 1 week. Thus, there seems to be a fairly good match in terms of workshop length between participant preference and workshop offerings. It is also apparent that the dissemination of information by NSF about all available project offerings is not as effective as it could be. While almost all UFEP workshops were fully subscribed, if the program is to grow to meet the needs of a larger proportion of undergraduate faculty, timely information concerning UFEP projects must be more effectively disseminated. One approach would be for NSF to heavily advertise that the recently-developed brochure listing all UFEP projects that will be offered during a coming year is available from NSF, and can be obtained in either paper copy or electronic mail versions. Such advertising could begin in late fall, specifying when the brochure is expected to be available. This would alert faculty to begin thinking about UFEP workshops as a possibility when making their summer plans. In conclusion, the Undergraduate Faculty Enhancement Program is meeting many of the needs it was designed to address. Faculty participants indicated that they received substantial benefits from the projects for themselves and for their teaching. Personal growth or renewal, increased knowledge of the field, and increased motivation or stimulation for teaching excellence are benefits most participants believe they received from project participation. The projects provided exposure to new ideas and technologies, which strongly influenced the introduction of new content and equipment into undergraduate courses. The projects successfully reached the intended audiences of "faculty who teach primarily undergraduate students" and groups that have been traditionally underrepresented in science, mathematics, and engineering, although there is room for improvement with regard to minority participation. However, information about all available UFEP projects needs to be more effectively disseminated by NSF if all potential audiences for UFEP are to be reached and served. In addition, travel costs may need to be subsidized in some way for some potential participants to be able to take advantage of the benefits available through UFEP projects.