previous section  |  Table of Contents  |  next section

Part II • Investment Process Goals and Results for FY 1999 NSF’s investment process goals focus on the means and strategies the Foundation uses to make investment decisions and shape its portfolio of awards in order to achieve its mission and desired outcome goals. Investment process goals address various aspects of NSF’s awards process, such as the use of merit review and the need to keep the awards system open to new people and new ideas. The Investment Process Goals also establish customer service standards for the agency, e.g., the time it takes to process a proposal. In addition, the facilities oversight goals for all federal science, space, and technology agencies are also included in NSF’s set of investment process goals. In FY 1999, seven investment process goals were fully achieved and three were not achieved.

Summary of Results: FY 1999 Investment Process Goals	Goals Achieved: Goals Not Achieved:	7 3

 

Investment Process Goal 1 – Use of Merit Review

At least 90% of NSF funds will be allocated to projects reviewed by appropriate peers external to NSF and selected through a merit-based competitive process.

Results: NSF exceeded this goal, with 95% of project funds allocated to projects selected through merit review. Rigorous merit review is a critical component of NSF’s decision making process for funding research and education projects. The Foundation strongly believes that award selections based on a competitive merit review process with peer evaluation ensure that ideas from the strongest researchers and educators will be identified for funding. This goal will be maintained in FY 2000.

Investment Process Goal 2 – Implementation of Merit Review Criteria

NSF performance in implementation of the new merit review criteria is: Successful when reviewers address the elements of both generic review criteria appropriate to the proposal at hand and when program officers take the information provided into account in their decisions on awards; Minimally effective when reviews consistently use only a few of the suggested elements of the generic review criteria although others might be applicable

Results: Largely successful, needs improvement. In FY 1999, two new NSF merit review criteria replaced four existing criteria. For FY 1999, COVs and ACs were asked to judge how well NSF is implementing the new criteria using the alternative format. In FY 1999, a total of 38 COV reports and 5 AC reports rated NSF programs on their use of the new merit review criteria. NSF was rated successful in achieving this goal by 33 COV reports and 3 AC reports. One AC report gives NSF a qualified successful rating, and 2 AC reports rate NSF minimally effective in implementing this goal. In most cases where NSF was not fully successful, it was found that reviewers and applicants were not fully addressing both review criteria. NSF has established guidelines requiring applicants and reviewers to address these criteria in proposals and reviews. NSF has recently reissued guidance to the applicants and reviewers, stressing the importance of using both criteria in the preparation and evaluation of proposals submitted to NSF. NSF is considering taking additional steps to ensure that applicants address these criteria when reporting project results. This goal will be maintained in FY 2000.

Investment Process Goal 3 – Time to Prepare Proposals

95% of program announcements and solicitations will be available at least three months prior to proposal deadlines or target dates.

Results: This goal was not achieved. This customer service standard was established in response to a survey where NSF applicants revealed that having a minimum of three months between program announcements and proposal deadlines was valued highly. In FY 1999, 75% of program announcements and solicitations were made available at least three months prior to their deadline/target date. Another 4% came within a few days of the three-month period. NSF intends to conduct an assessment of the FY 1999 data to determine why more announcements were not made available three months prior to deadline/target dates. NSF intends to focus additional efforts in this area to correct identified bottlenecks in the announcement posting process. In FY 2000, a web-based system for creating program announcements will be put in place. This system is expected to decrease the time required for an announcement to be posted on the NSF web site, which should aid the agency in achieving this goal.

Investment Process Goal 4 – Time to Decision

Process 70% of proposals within six months of receipt.

Results: This goal was not achieved. This customer service standard was established in response to a survey of NSF applicants who indicated that processing proposals within six months of receipt was valued highly. In FY 1999, 58% of proposals were processed within six months of receipt, somewhat better than the 52% average rate in the last five years, but nevertheless short of the 70% goal. NSF recognizes the validity of the community’s interest in this customer service standard and is striving to expedite the time between proposal submission and agency decision without jeopardizing the quality and integrity of the review process. This goal is a challenge for NSF; therefore annual performance targets are being scaled to reach the customer service standard by FY 2002. In FY 2000, NSF staff will work towards shortening the award process time by making more effective use of electronic mechanisms in conducting the review, working cooperatively to eliminate overloads and bottlenecks and carefully tracking the stage of processing and age of all proposals.

Investment Process Goal 5 – Award Duration

Increase average duration of awards for competitive research grants from a FY 1998 baseline of 2.7 years to at least 2.8 years.

Results: This goal was achieved. Providing an adequate duration for awards is necessary for obtaining high quality proposals and allowing the proposed work to be accomplished as planned. The entire science enterprise benefits when researchers are able to devote more time to research and less time to writing proposals.

Investment Process Goal 6 – Maintaining Openness in the System

NSF will increase the percentage of competitive research grants going to new investigators to at least 30%.

Results: This goal was not achieved. NSF believes that it is important that the proposal and award process be open to new people and new ideas, to help ensure that NSF continues to support research at the frontier of science and engineering. NSF is committed to maintaining openness in the system and will strive to increase the percentage of awards to new investigators. This goal will be maintained in FY 2000. NSF staff will examine trends over time.

Investment Process Goal 7 – Identifying Emerging Opportunities

All directorates within NSF will establish websites for the science and engineering community to provide suggestions for and comment upon emerging opportunities.

Results: This goal was achieved. During FY 1999, each of NSF’s seven directorates and the Office of Polar Programs had a website that included a mechanism to solicit suggestions and comments about emerging opportunities for significant advances in disciplinary and interdisciplinary fields of exploration; innovative research facilities; and new approaches to education and training. NSF believes that identifying and acting upon emerging opportunities in science and engineering is a basic strategy that supports all of NSF’s outcome goals for research and education. NSF is committed to engaging the community in an ongoing dialogue in order to assure that priorities for future funding are established properly.

Investment Process Goal 8 – Encouraging Integration of Research and Education

NSF will ensure that all of its new announcements of opportunities and proposal solicitations will contain an explicit statement encouraging proposers to integrate research activities with improving education or public understanding of science.

Results: This goal was achieved. During FY 1999, all new NSF proposal announcements and solicitations included a statement that requested proposers to address how their work would integrate research activities with education activities. NSF is interested in having awardees pay deliberate attention to their effectiveness as both researchers and educators. In the long-run, the interaction of research and education will provide for the development of a science and technology workforce as well as draw academic scientists and engineers into the challenge of improving K–12 education. In FY 2000, NSF will develop explicit instructions to request Principal Investigators (PIs) to address the integration of research and education in their proposals, and develop and implement a system to verify that PIs have done so.

Investment Process Goal 9 – Encouraging Attention to Diversity in All Aspects of NSF Programming

NSF will ensure that all of its new announcements of opportunities and proposal solicitations will include a statement encouraging proposers to address improving the participation of underrepresented groups in science and engineering in the course of their research and education activities.

Results: This goal was achieved. During FY 1999, all new NSF proposal announcements and solicitations included a statement that asked proposers to address how their proposed research and education activities encourage the participation of underrepresented groups in science and engineering. NSF emphasis on diversity responds to the legislative requirement that NSF address issues of equal opportunity in science and engineering. In FY 2000, NSF will identify mechanisms to increase the number of underrepresented minorities in the proposal applicant pool, and will identify mechanisms to retain that pool.

Investment Process Goal 10 – Facilities Oversight

The following goals are for federal science, space, and technology agencies that support construction projects and have responsibility for managing facilities (NSF, NASA, DOE). Construction and Upgrade: Keep construction and upgrades within annual expenditure plan, not to exceed 110% of estimates. Keep construction and upgrades within annual schedule, total time required for major components of the project not to exceed 110% of estimates. For all construction and upgrade projects initiated after 1996, keep total cost within 110% of estimates made at the initiation of construction.

Results: Data indicate that the majority of construction and upgrade projects were within annual expenditure plans and on schedule. During FY 1999, no construction and upgrade projects were completed. Therefore, the goal of keeping total project costs within 110% of initial estimates was not applicable. In FY 1999, NSF implemented a new facilities reporting system. In FY 2000, the facilities reporting system will be reviewed and updated to increase efficiency and to improve the reliability of data. These goals will be maintained in FY 2000.

Operations: Keep operating time lost due to unscheduled downtime to less than 10% of the total scheduled possible operating time.

Results: In FY 1999, NSF implemented a new facilities reporting system. The data indicate that a substantial majority of NSF-supported facilities realized the goal of keeping operating time lost due to unscheduled downtime to less than 10% of the total scheduled possible operating time. In FY 2000, the facilities reporting system will be reviewed and updated to increase efficiency and to improve the reliability of the data. This goal will be maintained in FY 2000.

previous section  |  Table of Contents  |  next section