This document has been archived.

The evaluation results reported in the previous section indicate that ILI awards directly stimulate innovative curricular changes that produce many kinds of beneficial impacts on under-graduate students at the grantee institutions. The results also show that ILI is effective in leveraging resources at grantee institution and stimulates broader investments in SMET education. However, an important additional measure of the achievements and impacts of ILI projects is their influence on individuals and organizations that are not direct beneficiaries of ILI awards. In fact, ILI proposals are evaluated by review panels based on the promise that they will benefit undergraduate SMET education nationally, not just at the grantee institution. To obtain a greater understanding of the far-reaching effects of ILI activities, the evaluation team examined the indirect impacts of the program on the larger culture beyond the grantee institutions.

Summary of Findings About
Far-Reaching Impacts

ILI-generated instructional innovations are widely disseminated using a variety of mechanisms. PIs report an average of 5 PI-initiated dissemination products or activities. This includes presentations of papers at regional or national conferences, publications in journals and other professional media, presentations or colloquia at other institutions, and workshops or short courses to acquaint others with project developed approaches and materials.

Disseminated products stimulate innovation at other institutions and organizations. Although the cumulative national and worldwide impacts on undergraduate SMET education of the PI-initiated innovation dissemination activities are difficult to quantify precisely, the results of the tracer studies conducted by the evaluation team show that significant instructional innovations have been adopted by workshop attendees and other identifiable second-stage users of project-generated products throughout the nation.

ILI grants stimulate research productivity. As a combined consequence of student and faculty research usage, ILI projects have generated significant numbers of research products, resulting in overall averages of 2 research publications and 6 presentations of research findings for 1990 and 1992 projects.

Unsuccessful ILI applicants persevere to achieve their goals. Altogether, 78 percent of the curricular improvement projects that were submitted as proposals to ILI in 1990 or 1992 ultimately became implemented to some degree, either with ILI support won in subsequent competitions or with support from other sources.

Fewer impacts are reported on K-12 students and teachers and on other community organizations. One-third of ILI PIs report that their project has impacted K-12 students in one way or another, and a similar percentage report project impacts on K-12 teachers. Seventeen percent of PIs report involvement in other forms of community outreach, local industry, public health agencies, or other community organizations.

Table of Contents

Dissemination of
ILI-Generated Products

If ILI projects are to exert a far-reaching influence beyond the grantee institutions, their activities and accomplishments first must become widely known among other members of the SMET education fields at the undergraduate level and beyond. In fact, this is a fundamental objective of the ILI program. Grantees are expected to make conscientious efforts to share the innovative instructional approaches and materials they develop in their projects.

Types and Extent of Dissemination Activities. The instructional innovations generated by the ILI program are widely shared with others outside the grantee institution using a variety of dissemination mechanisms (Table 9). Two-thirds of the grantees in 1990 and 1992 initiated some form of dissemination, and a similar percentage engaged in less formal sharing of project-related information in response to individual inquiries. Presentations at national and regional conferences are the most frequently reported means of PI-initiated dissemination, although publications in journals and other professional media, presentations or colloquia at other institutions, and workshops or short courses are also employed to acquaint others with project-developed approaches and materials. However, only 32 percent of PIs can confirm that undergraduate students from outside their own institutions have been impacted by the instructional innovations developed in their ILI projects.

Table 9. Percentage of PIs employing various dissemination mechanisms, and average number of dissemination activities per award: 1990 and 1992 grants

Dissemination mechanism	Percent of grantees reporting mechanism (based on 1,074 responses)	Average number  of activities per award	Total number of activities
1. Papers or presentations at regional or national association/ meetings	54	1.7	1,807
2. Workshops or short courses	26	1.1	1,158
3. Speeches or colloquia at college campuses	31	1.1	1,132
4. Publications in professional media (journals, newsletters, etc.)	31	0.8	909
5. Total (1 to 4)	67	4.6	4,940
6. Informal responses to individual inquiries about project	65	6.6	7,088

Note: Percents do not add to 100 because respondents may report more than one type of dissemination mechanism.

SOURCE: 1995 mail survey of ILI grantees.

Many PIs commented during the site visits that they use the Internet as a vehicle for sharing project-developed laboratory experiments and other curricular materials. For example, the Computer-Integrated Process Engineering Laboratory at University of Florida is linked to the SUCCEED²³ (Southeastern University and College Coalition for Engineering Education) consortium through the Internet. It is difficult, however, to quantify precisely the cumulative national and worldwide impacts of this rapidly growing means of dissemination. Two of the 15 tracer studies that are discussed in the next section used Internet-based dissemination efforts. One of these projects features an Internet web site in which users can access software for 16 computer-based exercises that provide realistic observing experiences for introductory astronomy labs based around simulations, digital images, and observations. The site is accessed approximately 30 times per day.

Because ILI equipment also stimulates student and faculty research, significant numbers of research products have been developed and disseminated that are directly attributable to ILI awards. In fact, a greater number of research publications and presentations result from ILI awards than publications and presentations about instructional innovations. Respondents to the 1990 grantee survey reported about 2 research publications and 6 presentations of research findings per project.
 

Cumulative Dissemination Impacts  An estimate24 of the cumulative impacts resulting from all grants funded during the first decade of the program shows the program generating a wealth of activities and products:  · 8,000 papers presented at national and regional conferences;  · 5,600 workshops and short courses;  · 4,700 presentations or colloquia at other colleges and universities; and  · 4,200 publications, mostly in the form of journal articles but occasionally also including major products such as commercially published textbooks, laboratory manuals, etc.

Most ILI PIs are not involved in outreach activities with groups outside the college/university community, although a minority of PIs do establish such linkages to other school, community, or industry groups. One-third of PIs report that their project has impacted K-12 students in one way or another and a similar percentage report project impacts on K-12 teachers. Most often, these impacts are the result of summer or weekend workshops for elementary or secondary school teachers or students that incorporate project equipment or materials to some extent. However, nearly one-third of the K-12 outreach activities reported by PIs are limited to brief lectures or demonstrations that take place as part of a tour, field trip, or campuswide open house event.

Other community and industry-based groups benefit to an even smaller extent from the ILI program; 17 percent of PIs report involvement in some form of community outreach. Outreach activities such as training workshops for local industry or law enforcement personnel and research partnerships with local health departments, industries, or other community organizations (e.g., environmental assessment, archeological excavations, forensic testing) are included in this class of outreach.

Comparisons of dissemination rates among various subgroups of ILI projects revealed notable differences along three dimensions:

• Project Level. Projects targeting upper division students are not as widely disseminated as those targeting lower division students. An average of 8 versus 14 instances of dissemination²⁵ were reported per project by the two subgroups, respectively. This finding is largely considered to be a reflection of the more limited audience for the specialized, advanced projects.
• Grant Amount. More instances of dissemination were reported by larger projects (with awards in excess of $50,000) than smaller projects. An average of 20 versus 9 instances of dissemination were reported per project by the two subgroups, respectively. This finding can be partially attributable to several factors that have already been shown to be associated with larger projects, such as a greater number of PIs using project equipment and increased leveraging. An additional explanation, which was not directly addressed by this study but appears consistent with data collected during the site visits, could be that in many cases the large-scale projects tend to involve large computer laboratories with multiple applications and the development of innovative instructional software packages that are readily disseminated.
• Institution Type. Grantees at two-year institutions were much more likely to report using their ILI equipment for activities benefiting local schools, industries, or communities (70 percent) than were grantees from four-year institutions (45-58 percent).

___________________________________________________________________
²³SUCCEED is sponsored by NSF's Directorate for Engineering.
²⁴The extrapolations presented are based on the average numbers of such products generated by 1990 grantees during the five-year period following their grant awards. Because dissemination activities are likely to continue beyond the fifth year for at least some projects, these estimations are considered to be conservative.
²⁵Instances of dissemination refers to an aggregation of all reported instances of (1) publications in professional media(journals, newsletter, etc.), (2) Workshops or short courses, (3) presentations at other campuses, (4) papers or presentations at regional or national conferences and meetings, and (5) responses to information requests.

Table of Contents

 
A Closer Look at Selected Dissemination Efforts

To obtain a deeper understanding of the impacts that can result from ILI-related dissemination activities, tracer studies²⁶ were conducted with 15 selected projects that were thought to have had significant and traceable²⁷ dissemination impacts beyond the initiating/host institution. The results, though not generalizable, showed that the ILI products generated from these awards were disseminated widely and that a large majority of recipients incorporated the information and material into their undergraduate curriculum.

According to the tracer study assessment, the form of dissemination activity with the greatest impact was short courses and workshops for undergraduate faculty. Eleven of the 15 tracer study PIs provided the evaluators with participant lists indicating that over 900 undergraduate faculty members had been directly impacted by such workshops. Other forms of dissemination assessed in the tracer studies and the corresponding numbers of people who were directly impacted are shown in Table 10.
 
 

Tracer Study Projects Subject areas, locations, highest degree offered,  and project year  Astronomy  Gettysburg College, Gettysburg, PA, baccalaureate, 1991   Biology  San Francisco State University, San Francisco, CA, master's, 1992   Chemistry  Hope College, Holland, MI, baccalaureate, 1990  James Madison University, Harrisonburg, VA, doctorate, 1986  Williams College, Williamstown, MA, master's, 1990   Computer Science  Rochester Institute of Technology, Rochester, NY, doctorate, 1990   Engineering-Civil  Kansas State University, Salina, KS, doctorate, 1990   Engineering-Robotics  University of Illinois, Champaign, IL, doctorate, 1992   Geology  Harvey Mudd College, Claremont, CA, master's, 1992   Mathematics  City University of New York, Borough of Manhattan Community College, New York, NY, associate, 1992  Grinnell College, Grinnell, IA, baccalaureate, 1987  Temple University, Philadelphia, PA, doctorate, 1990   Physics  Dickinson College, Carlisle, PA, baccalaureate, 1992. Spinoff project at Joliet Junior College, Joliet, IL, associate, 1991   Lawrence University, Appleton, WI, baccalaureate, 1987  Rose-Hulman Institute of Technology, Terre Haute, IN, master's, 1992

__________________________________________________________________
²⁶See Chapter 1, p.5, for a discussion of the methodology for tracer studies.
²⁷Many forms of dissemination, such as journal/textbook publications and professional presentations, could not be examined using the tracer study methodolgy because it was too expensive to track down the recipients and users of such dissemination products or activities.

Table of Contents

Table 10. Forms of dissemination assessed
in the tracer studies and
the number of people impacted (total 15 projects)

Dissemination activity	Number of projects	Number of people impacted
Workshops or short courses for undergraduate faculty	11	928
Publications in professional journals	1	206
Training courses for private industry/ community groups	1	195
Workshops for high school faculty	3	181
Papers or presentations at regional or national association meetings	2	147
Participation in shared-equipment consortia or centers	2	142
Internet	2	100
Newsletter	1	82
Distribution of plans and/or hardware for laboratory instrumentation	1	9

SOURCE: 1996 tracer study intdrviews.

Dissemination efforts were generally quite extensive in the tracer study projects. In most cases, dissemina-tion activities had reached at least 100 undergraduate professors per project, although the total number of dissemination recipients ranges widely from a minimum of 4 to a maximum of nearly 200, depen-ding on the length of time the project has been in operation and the appeal of the project materials and information (i.e., targeting introductory courses or specialized upper division courses). Other audiences besides undergraduate faculty were exposed to ILI-generated products as well. Three projects used their ILI equipment in summer enhancement programs that impacted 150 high school teachers altogether. Nearly 200 law enforcement officials were exposed to forensics equipment and instructional materials related to one PI's ILI project while attending a workshop at the 20th Annual Symposium on Crime Laboratory Development, held at the FBI Training Center in Quantico, Virginia. And one PI who developed sophisticated robotics software with his ILI grant has sold the software to 68 individuals working with robotics in private industry.

For two-thirds of the tracer study projects, the workshop participants and other recipients of ILI-related dissemination report that they readily incorporated the information and materials that they received into their undergraduate courses. Eighty percent or more of the respondents for these projects reported using ILI-related materials in their undergraduate instruction. Adoption rates in the remaining six projects were more moderate, in the 40- to 55-percent range. Depending on the number of courses in which the material was used and the enrollment for these courses, the cumulative numbers of undergraduate students exposed to the ILI projects examined in the tracer studies outside the grantee institution can become quite substantial. In one case, over 150,000 undergraduates from around the country were estimated to have been introduced to the instructional innovations developed from a series of four interrelated ILI grants that involve a workshop-based physics curriculum. Estimates of cumulative number of students impacted for most of the other tracer study projects are, however, more moderate, in the range of 2,000 to 50,000, and the estimated number of students impacted for three projects is less than 500. An overview of the documented recipients of ILI-related dissemination from 15 tracer study projects is shown in Figure 6.

Figure 6. Overview of dissemination recipients from 15 tracer study projects

SOURCE: 1996 tracer studies interviews.

Undergraduate faculty who are recipients of dissemination activities often discover how to improve their SMET courses but lack the requisite equipment to do so. Data concerning the number of proposals that were stimulated by exposure to ILI-related dissemination activities suggest that ILI can be a catalyst to proactive laboratory improvement efforts. Among the 300 undergraduate faculty respondents examined in the tracer studies, a total of 100 proposals for laboratory instrumentation were submitted. Fifty-four of these proposals were successfully funded, including 34 ILI grants.

It is interesting to note that most dissemination activities examined in the tracer studies were supplemented or enhanced by other, usually NSF, grants. Dissemination activities in six projects were supported by one or more Undergraduate Faculty Enhancement grants. Five project PIs reported that dissemination activities were supplemented by one or more Course and Curriculum Development grants, and dissemination activities in three projects were based on two or more interrelated ILI grants.

Three interesting examples of successful dissemination efforts found in the tracer study sites are described below.

Tracer Study 1: Impacting Large Numbers of Undergraduate Students. The dissemination activities associated with a 1992 project housed at San Francisco State University (SFSU) entitled "Molecular Biology Laboratory Instrumentation for Undergraduate Instruction" exemplify how a large number of students at the secondary and postsecondary levels from across the country have been impacted by a single ILI grant. For this project, the PIs purchased equipment for use in teaching laboratory exercises illustrating basic molecular biological concepts in both the lower and upper division components of the undergraduate curriculum. The project created a modern molecular biology core facility that will house the major equipment, namely, an ultracentrifuge and rotors for fractionating and purifying macromolecules, a DNA sequencing and computer analysis system, a high technology TV camera with analysis software, a scintillation counter to monitor radioisotopes, and a set of general-use equipment, i.e., microfuges and gel boxes. Most of the dissemination from this project resulted from two types of workshops, one for high school biology teachers and the other for college professors.²⁸ Most attendees gained a great deal of knowledge that they used to their benefit, as illustrated by the following quote:

The experience was so positive to me that I decided to go back to SFSU during sabbatical in 1993 and spent eight months there working with [a co-PI in the chemistry department]. One of the outcomes has been a collaboration with a colleague in our biology department who is a molecular biologist (I am a biochemist in the chemistry department). We designed a new major called Biochemistry and Molecular Biology, which has been well received and has attracted some of our best students. I would not have had the confidence or vision to do this without my work in [the PI's] course.

Of the 192 undergraduate faculty members who participated in the NSF Chautauqua short courses based on this ILI project from 1992-95, 26 were contacted through electronic mail and invited to serve as respondents in the tracer study. A total of 19 responded, providing a 73 percent response rate. Respondents reported that the workshop provided direct experience in the field of recombinant DNA technology which helped "fill in a lot of gaps" that existed in their personal theoretical understanding of the field. In most cases, the workshop provided an impetus for respondents to incorporate recombinant technology into their own microbiology curriculums.

Fifteen of the 19 respondents incorporated information and materials from the workshop into their undergraduate instruction, usually in two or three different courses per respondent. All four respondents who did not make use of the workshop material explained that the content was not relevant to the courses that they teach. Among those professors who were able to apply the workshop material in their courses, the numbers of students per respondent who were exposed to the material ranged from 100 to 700 in most cases, although three respondents reported substantially higher student impacts in the range of 2,000 to 3,000 students per respondent over the course of three years. Extrapolations from the sample to the entire population of 192 faculty members exposed to this ILI project through the NSF Chautauqua short courses suggest that approximately 123,000 students may have been impacted at undergraduate institutions other than the grantee institution during the four years that this project has been operational.

________________________________________________________________

²⁸The workshops for college faculty were supported by NSF's Undergraduate Faculty Enhancement program

In addition, the PIs facilitated a teacher education in biology program with funding from NSF's Teacher Preparation program. This program provides summer training to high school biology teachers; 68 teachers participated between 1993 and 1995. The PIs also make project equipment available to freshmen and sophomores from the local community college under the National Institutes of Health Bridge for Underrepresented Minorities program. Approximately 60 students from the community college had used the equipment for summer research projects from 1993 to 1995.

Tracer Study Example 2: Exposing High School Mathematics Students to Computer Technology. A 1992 project that represents dissemination efforts that more strongly impact high school teachers than undergraduate faculty was called "PC²: Personal Computers/ Pre-Calculus." The project was housed at the CUNY Borough of Manhattan Community College (BMCC), a large, urban, two-year college whose student body is two-thirds female and over 80 percent minority. In order to attract and retain more women in the calculus sequence, the school secured an NSF calculus reform grant in 1989-91 that enabled them to add computer-based collaborative workshops under the Course and Curriculum Development program to calculus I, II, and III. It became clear to the college that to recruit and retain women and minorities, the program must begin at an earlier level, so they pushed their successful efforts back to the precalculus level. The computers acquired as a result of the ILI grant allowed BMCC to attract and prepare more women and minorities for careers in science, mathematics, engineering and technology. The PI for this project held two types of workshops in which project materials and instructional approaches were introduced: one for mathematics faculty at two-year institutions, and a second for high school mathematics teachers.

A total of 120 undergraduate faculty, nearly all from other two-year institutions, have been the participants in numerous NSF-sponsored workshops conducted by the PI. Of these participants, 29 were contacted through electronic mail or by telephone and invited to serve as respondents in the tracer study. Among the 14 professors who responded to our query, 6 reported using the information related to the project as an integral part of the undergraduate courses they teach. Most of those respondents who did not make use of the material from the workshop reported that it was not appropriate for the courses they teach-remedial math in most cases. One nonuser was interested in implementing the information but was hampered by a lack of equipment; another felt that the material was "too cumbersome and complicated"; and one other "may use it sometime in the future." Among the six respondents who did use the workshop material, the number of undergraduate students exposed to the ideas and materials presented in the workshop ranged from 100 to 200 per respondent.

Of the 33 high school mathematics teachers who attended the PI's Advanced Placement Mathematics Graphic Calculator Training, 15 were contacted by telephone and invited to serve as respondents in the tracer study. All 11 teachers who responded to our inquiries reported using the information related to the ILI project as an integral part of the high school courses they teach. The courses affected include advanced placement calculus, calculus, precalculus, advanced placement math, algebra, and trigonometry. The respondents reported that an average of 117 students were impacted by the ILI project. They also indicated that the calculator training has profoundly influenced their instructional practices as evidenced by the following representative quote:

We used everything we learned at the workshop. She exposed us to many of the different technologies that were out there at the time. Before we attended the workshop, none of our courses had technology incorporated into them. Now technology is incorporated into our statistics, precalculus, and calculus classes.

Many respondents also expressed appreciation for the opportunity to share and interact with their colleagues and indicated that the information was very much in line with the reform movement that is taking place in high school mathematics.

Tracer Study Example 3: Using a Broad Range of Dissemination Activities to Stimulate Further ILI Grant Activity. The PI for a 1986 project entitled "Applications of Lasers in Chemistry" has initiated a broad range of dissemination activities including the development and distribution of 20 laboratory experiments suitable for undergraduate chemistry courses, distribution of low-cost laser apparatus kit plans, publication of a newsletter, and facilitation of an annual short course on laser experimentation. Most of the dissemination activities mentioned above were funded through four Undergraduate Faculty Enhancement grants that were awarded in 1988, 1990, 1992, and 1993. The project was designed to introduce upper level undergraduate chemistry students to the uses of lasers in various aspects of chemistry. Courses in instrumental analysis, physical chemistry, and biochemistry have been enhanced by experiments involving Raman spectroscopy; regular, polarized, and phase-sensitive emission spectroscopy; multiple photon effects; laser vaporization; and thermal lensing that were developed as part of the project. The impacts of this project are far reaching, as demonstrated by the following quote from a participant in the PI's short course:

After attending the laser course, I developed my own lab experiments for the physical chem. lab using the He-Ne laser to measure refractive index, and to measure the rates of reactions. I am currently working on experiments for the pulsed nitrogen laser system. I recently went on a lecture tour in Thailand and gave talks on these experiments at Mahidol University in Bankok, and Chieng Mai University in Chieng Mai.

The PI for this project provided the active mailing list for his newsletter, which contained the names of 82 people (primarily undergraduate faculty) who had contacted him to request copies of his experiments, advice, or a tour of his laser facility. Among the 82 are 14 individuals, 10 of whom attended the PI's short course on laser experimentation and 4 of the PI's former students, who were known to have submitted successful ILI proposals for laser-related materials. Thirty-one individuals were contacted as part of the tracer study, and 24 ultimately responded.

Every respondent who had participated in the laser short course, and most that were subscribers to the newsletter, have used the information in an undergraduate teaching and research setting quite successfully. In particular, those that teach junior/senior-level physical chemistry courses have incorporated a great deal of the laser material in their own courses. They report using many of the experiments developed by the PI as examples of how lasers can be used to study chemical processes on the molecular level. A total 1,081 of students (excluding the PI's own students) were reported by the respondents to be exposed to information and experiments relating to this ILI project. The number of students impacted through individual respondents ranged from none in two cases (because the topic was not directly relevant to the courses taught by one respondent and the other does not have access to the appropriate laser apparatus) to approximately 75 students each year for a professor at a state college who incorporated the material into his general chemistry courses and his two upper division analytical courses after taking the short course in 1990. An overview of other impacts associated with the various dissemination activities is presented in
Figure 7.

One of the more productive avenues for sharing information with colleagues that was uncovered in this tracer study was related to activities carried out at the regional meetings of the Association of Chemistry Teachers at Liberal Arts Colleges as the following quotes illustrate:

At last fall's [1995] meeting of the Midwest Association of Chemistry Teachers at Liberal Arts Colleges, we held a group discussion between about 20 members who are physical chemistry teachers. The use of lasers was the main focus of the discussion. Indeed, more than a few of the participants were graduates of [the PI's] workshop so we talked extensively about his experiments. The discussion went over so well that we have now set up an e-mail network of p-chem. teachers to discuss each other's projects with lasers and other equipment.

Two years ago [my school] hosted the annual meeting of the Mid-Atlantic Association of Liberal Arts Chemistry Teachers. I invited [the PI] to present a session on lasers at that meeting. I believe approximately 10 faculty from various colleges in the mid-Atlantic region attended. I do not know how many were able to incorporate [the PI's] suggestions in their teaching, but I do know the session was very well received by those who attended. They especially appreciated [the PI's] sensitivity to their resource limitations, and his thorough presentation.

Figure 7. Impacts associated with dissemination activities from Tracer Study 3: "Applications of Lasers in Chemistry"

SOURCE: 1996 tracer study interviews.

Even those respondents that did not teach a relevant upper division laboratory course nevertheless felt that the knowledge gained from the PI's dissemination activities have greatly enhanced their understanding of the applications of lasers in chemistry, their ability to read and understand the research journal articles pertaining to laser experimentation, and their ability to present current information during their introductory lecture courses. In short, the PI has done an excellent job sharing his knowledge and experiences from the ILI grant.

Yet, the most impressive evidence of successful dissemination relating to this ILI project is the direct assistance the PI provided to 14 other successful ILI grantees. When preparing their ILI proposals, these successful grantees felt that the material in the PI's laser short course was very useful and that the PI himself was an invaluable consultant.

Table of Contents

 
Impacts of ILI on
Unsuccessful Applicants

In order to more tangibly determine how ILI impacts undergraduate education and career development among unsuccessful applicants, we investigated the broader outcomes of the application experience for unsuccessful PIs who submitted proposals to the 1990 and 1992 programs.

Outcomes of Unsuccessful ILI Applications. Nearly two-thirds of the unsuccessful applicants (62 percent) credited the ILI proposal development process with having "stimulated the formulation and refinement of a worthwhile project," the great majority of which ultimately were implemented. Of the approximately 3,000 unsuccessful proposals submitted in the 1990 and 1992 program years, 70 percent of the proposed ideas for improving undergraduate curricula had been implemented to some degree, by late 1995. (See Figure 8 for a breakdown of proposal outcomes.) Nearly one-fifth of the unsuccessful projects received funding support in subsequent ILI grant competitions.²⁹ A far greater proportion of the unsuccessful projects received subsequent funding support from other sources-usually the college or university itself, but occasionally from business and industry, or other federal sources.

Figure 8. Outcomes of unsuccessful proposals/ ideas submitted to ILI in 1990 and 1992 (N=3,064)

SOURCE: 1995 mail survey of unsuccessful applicants.

The outcomes reported above suggest that the stimulative effects of ILI extend well beyond the direct and indirect impacts of the projects that are actually funded in any given program cycle. The outcome data also provide an indication of the extent to which the ILI program achieves its objective of stimulating the generation and development of serious, feasible ideas for improving undergraduate education in the sciences.

In spite of these encouraging findings, nearly half (47 percent) of the unsuccessful applicants did not feel that the ILI reviewers' comments on their proposals were useful. Although many unsuccessful applicants reported that the reviewers' comments "constructively identified weak areas and suggested improvements," a nearly equal proportion reported that the comments were "confusing," "condescending," "discouraging," "excessively harsh," "vague," and "inconsistent" from year to year.
 

__________________________________________________________
²⁹Forty-six percent of unsuccessful applicants have reapplied for ILI funding for the same or a closely related project.Forty-two percent of the ILI reapplications were successful.

Comparisons of Successful and Unsuccessful Applicants. Comparisons were made between successful ILI applicants and unsuccessful applicants for the program years 1990 and 1992 to investigate how responsive the program is to the needs of various types of applicants. Although the successful and unsuccessful applicants were found to be very similar in terms of demographic characteristics such as academic rank, tenure status, and teaching load, notable differences were found among the two groups on the following characteristics:

• Undergraduate enrollment in department. Successful applicants report much larger undergraduate enrollments in their departments than do unsuccessful applicants (670 vs. 300 mean total undergraduate enrollment).
• Highest degree awarded in department. Successful applicants are less likely than their unsuccessful counterparts to work in departments where the doctorate is the highest degree awarded to students (22 percent vs. 33 percent) and more likely to work in departments where the baccalaureate is the highest degree awarded (48 percent vs. 28 percent).
• Highest degree sought by undergraduates. Successful applicants report a higher percentage of the undergraduate majors in their department seeking a graduate degree in sciences, mathematics, engineering, or technology than do unsuccessful applicants (39 percent vs. 30 percent).

Several differences also emerged between the two classes of applicants in terms of their prior funding experience and other variables associated with the proposal process. Successful applicants, as compared with unsuccessful applicants, were more likely to have

• heard about ILI from a colleague (45 percent vs. 35 percent, respectively) or from an ILI project or product (17 percent vs. 8 percent) than from their college administration or department (46 percent vs. 53 percent);
• reviewed successful ILI proposals from other institutions (32 percent vs. 20 percent);
• obtained proposal suggestions or advice from other ILI PIs (40 percent vs. 34 percent) or from NSF (38 percent vs. 32 percent); and
• obtained grant funding from NSF for a previous project (44 percent vs. 39 percent).