Summary | Table
Part A: EPSCoR Program and Its Evaluation | Part B: Evaluation Findings | Part C: Policy and Program Implications
|1. What Is the EPSCoR Program?||
Since its creation in 1950, the National Science Foundation (NSF) has sought to advance research and education in science, mathematics, engineering, and technology throughout the United States. Explicit in its original mandate was a cautionary warning to avoid concentrating federal funding of academic research and development (R&D)1 on a geographic basis. Thus, NSF’s authorizing legislation, as amended (42 U.S.C. 1862, Sec. 3e), stated:
In 1978, congressional concern about the geographical concentration of federal funds for academic research led Congress to authorize the NSF to conduct the Experimental Program to Stimulate Competitive Research (EPSCoR) (Greenberg, 1967; U.S. Congress, Office of Technology Assessment, 1991; and Martino, 1992). Eligibility for EPSCoR participation was restricted to states that historically had received relatively low levels of federal R&D funding. Additionally, such states were required to demonstrate a commitment to improving the quality of university-based research in science, mathematics, engineering, and technology.
EPSCoR, as the program’s name reflects, was designed to stimulate competitive research--and not to be a special set-aside program (EPSCoR Program Solicitation, 1989b). Through EPSCoR, NSF would become a partner with participant state governments and universities. The role of state government was seen as especially important in achieving the EPSCoR program’s objectives because of the overwhelming importance of public universities within the higher education systems of EPSCoR states. In 1995, for example, 53 of the 56 universities participating in the EPSCoR program were publicly supported universities.
Over time, NSF’s goals for the EPSCoR program broadened to include enhancement of educational and human resource opportunities for underrepresented faculty and student populations in science, mathematics, engineering, and technology, and the transfer of academic research to the private sector.
EPSCoR’s Program Strategy
EPSCoR’s program strategy is designed to increase the competitiveness of merit-reviewed proposals from investigators in states that have historically received low percentages of federal R&D support. The assumption is that increased competitiveness will subsequently lead to increased R&D funding in these states. Further, on the premise that university science and engineering departments can positively influence a state’s economic and human resource development, EPSCoR pursues long-term partnerships with state leaders in government, business, and higher education (Malecki, 1991; and Luger and Goldstein, 1991). In pursuit of these goals, EPSCoR applies the following strategies:
Although the strategies appear to focus on individual or groups of research projects, EPSCoR’s vision from the onset was systemic. The program always has sought to strengthen the entire science and technology (S&T) infrastructure in an eligible state--thereby increasing the ability of EPSCoR researchers to compete for federal and private sector R&D funding and accelerating the movement of EPSCoR researchers and institutions into the mainstream of federal and private sector R&D support.
Non-Federal Cost Sharing
The EPSCoR partnership between NSF and eligible states requires investment by both partners. A one-to-one dollar match was required for the awards made from 1980 to 1994.
In preparing to submit a proposal, an EPSCoR steering committee within each eligible state was expected to have undertaken a comprehensive analysis of the strengths and opportunities for developing its research institutions in support of the state’s overall R&D objectives. Examples of infrastructure activities EPSCoR has funded include the following:
The NSF initiated the EPSCoR program in Fiscal Year (FY) 1978 with seven planning grants. From that time through FY1996, the program awarded $182.2 million in grants and cooperative agreements. Participating in the program by the end of the period were 18 states and the Commonwealth of Puerto Rico. The states were Alabama, Arkansas, Idaho, Kansas, Kentucky, Louisiana, Maine, Mississippi, Montana, Nebraska, Nevada, North Dakota, Oklahoma, South Carolina, South Dakota, Vermont, West Virginia, and Wyoming.
By 1994, EPSCoR had been operational for over 15 years and the NSF decided that, as part of its plan to evaluate the programs of the Directorate for Education and Human Resources, the time had come to look at whether the EPSCoR program was achieving its intended outcomes. This evaluation of the EPSCoR program covers the period 1980-1994, with 1994 as the last year for which R&D expenditures were available at the time of the evaluation. The evaluation thus does not cover changes or developments in the EPSCoR program after 1995.
|2. What Is the EPSCoR Evaluation?||
The EPSCoR evaluation--conducted by COSMOS 5 Corporation--was designed with two objectives in mind:
The Evaluation Team
The design and conduct of the evaluation were carried out by a COSMOS research team in consultation with a variety of experts. First, the team convened an EPSCoR Workgroup, consisting of EPSCoR leaders, to provide advice to the evaluation. Next, the team met (twice) with an expert group of advisors, who discussed the design and preliminary findings. Finally, the team commissioned two special papers by two additional experts (Gumport,1996; and Hackett, 1996) to serve as background to the entire evaluation design. 2
The Evaluation Design
EPSCoR had deliberately selected the states with the lowest R&D activity in the United States. Consequently, a control group design could not be implemented by the evaluation. An alternative design was to test a causal model of EPSCoR using a theory-based model. The theory-based approach could not yield the same degree of certainty as a control group design. However, the extent to which the model could be supported by empirical evidence and alternative explanations ruled out, the more confidence the NSF, Congress, and other stakeholders would have in the model’s specifications regarding claims for EPSCoR’s effects on R&D funding outcomes.
The EPSCoR model investigated by COSMOS was specified as a series of hypothesized causal links depicted in Exhibit 1, which presupposes the following critical steps:
The evaluation sought the answers to four main questions, reflecting the presumed conditions in Exhibit 1:
To address these questions, the evaluation team collected data by using a variety of procedures. First, evidence about R&D funding, derived from NSF’s ongoing science resource studies, was analyzed. Second, information about EPSCoR’s funding practices was based on an analysis of NSF’s records of its individual research awards and interviews with former and current NSF program officers. Third, the evaluation team made multiple site visits, over a nine-month period, to five representative EPSCoR states (and 14 campuses in those states), to collect data about EPSCoR funded research projects, state S&T developments, and university research policies and practices. Many individuals were interviewed and numerous documents collected during these site visits. Finally, information about EPSCoR-related scientific publications and subsequent funding was derived from COSMOS’s monitoring work for the EPSCoR program conducted from 1994 to 1996, in which all 19 states were the subjects of site visits and in which principal investigators were asked to submit copies of their publications for the team’s review.
The evaluation’s findings and conclusions responding to the four evaluation questions are presented in Section B. The results are organized around three major themes:
In Section C, the evaluation presents the implications of the findings for EPSCoR policy and program operations. First, however, the main evaluation conclusions (based on the data in Section B) are previewed below.
|3. What Were the Evaluation’s Main Conclusions? (A Preview)|| The EPSCoR program’s
objective was to reduce the geographic concentration of federal R&D
funds in the United States and its territories. The most direct measure
of this desired impact was a comparison of changes in the share of federal
R&D expenditures (and NSF obligations) across states between 1980
and 1994. For EPSCoR to demonstrate a positive impact, 1) the EPSCoR states’
share of R&D funding would have had to increase relative to the non-EPSCoR
states’ share, and 2) EPSCoR’s program strategies would have had to show
plausible influence in producing the observed increase in R&D funding
EPSCoR States Increased Their Share of Federal R&D Funding
From 1980 to 1994, EPSCoR states’ share of federal R&D funding increased from .25 percent to .40 percent per state, or from $10.1 million to $50.5 million, per state. (The “per state” assessment is given because the program continually added newly eligible states during this period of time; overall, the EPSCoR states’ share of federal R&D funding represented 7.65 percent or $960 million by 1994). EPSCoR states’ shares of NSF funding showed a similar pattern of increase. Although the increases were small in absolute terms, the increases represent a successful outcome for the EPSCoR program (during the same period, the non-EPSCoR states’ shares decreased).3
Some of EPSCoR’s Program Strategies Appear to Have Been Responsible for Increasing Federal R&D Funding in EPSCoR States
Within each EPSCoR state, the program required the formation of a statewide steering committee, to represent the R&D interests of key organizations and sectors of the state (EPSCoR Program Solicitation, 1989a). The committee assumed a pre-review function over the component research projects that formed a state’s overall proposal to NSF. Thus, the steering committee engaged in a peer review process that involved out-of-state experts. In many states, such innovation in itself created a new S&T environment and a foundation for increased R&D competitiveness.
EPSCoR Influenced the States’ S&T Environments through the Steering Committees and the State Funding Match Requirement
The EPSCoR steering committees and the state funding match requirement promoted dialogues, planning, and new S&T initiatives. In particular, the steering committees opened communications between state university research officials and their counterparts in business, industry, and state government. This led to the identification of state research priorities that factored in the combined resources of a state’s university system with the unique opportunities offered by a state’s natural or institutional environment. EPSCoR’s one-to-one dollar match requirement helped to focus the states’ R&D priorities and to foster collaboration among universities and between universities and state government and industry. State governments’ willingness or ability to provide the required matching funds, however, has been a recurring problem in some states.
The EPSCoR steering committees usually served as the initial opportunity for research collaboration among a state’s universities and between the universities and state government agencies and industry. These collaborations were institutionalized in many instances through the development of formal consortium relationships, state S&T plans, new S&T agencies, and state science advisory councils.
EPSCoR Enhanced Some Aspects of Universities’ Orientation to Research
Compared to its effects on the states’ S&T environments, EPSCoR’s influence on university policies or resources devoted to research was not significant. However, the EPSCoR awards did stimulate inter-university collaboration (in some cases overcoming strong, traditional rivalries), heighten university-industry research collaboration, install more rigorous standards of peer review, and create new interdisciplinary research facilities. EPSCoR also enabled state universities to offer larger and competitive startup packages to attract talented new faculty, which tended to increase the research orientation of state university faculties in the sciences.
EPSCoR-supported research also showed evidence of scientific productivity and, hence, competitiveness. This was evident from the number of subsequent academic publications and external funding awards associated with the EPSCoR-supported research projects.