Survey Quality Measures
1. Overview (2011 survey cycle)
The National Science Foundation (NSF) R&D Expenditures at Federally Funded R&D Centers Survey is the primary source of information on separately budgeted R&D expenditures at FFRDCs in the United States. Conducted annually for university-administered FFRDCs since FY 1953 and all FFRDCs since FY 2001, the survey collects information on R&D expenditures by source of funds. In FY 2010 the survey was expanded to collect additional detail on R&D funded by the American Recovery and Reinvestment Act (ARRA), R&D expenditures by type of costs (salaries, equipment, etc.), and the FFRDC's total operating budget.
The survey is an establishment survey completed by FFRDC staff. The majority of respondents work in the accounting and budget offices of the FFRDCs.
c. Key variables
The survey requests data from FFRDCs in the following categories:
- R&D expenditures by source of funds (federal, state and local, business, nonprofit, or other)
- R&D expenditures by character of work (basic research, applied research, or development)
- R&D expenditures funded by ARRA
- R&D expenditures by type of costs (salaries, software, equipment, subcontracts, other direct costs, and indirect costs)
- Total operating budget
2. Survey Design
a. Target population and sample frame
The target population includes all 40 FFRDCs. Of the 40 FFRDCs, 14 are administered by academic institutions, 20 are administered by nonprofit organizations, and 6 are administered by industrial organizations. The current master list of FFRDCs is available at http://www.nsf.gov/statistics/ffrdclist/.
Legally, FFRDCs are engaged in basic research, applied research, development, or management of R&D activities, either upon direct request of the government or under a broad charter from the government, but in either case under the broad monitorship of the government. FFRDCs are operated, managed, and administered as separate organizational units within a parent organization or as separately incorporated organizations. They receive their major financial support (70% or more) from the federal government, usually from one agency, and are expected to have a long-term relationship with their sponsoring agency.
b. Sample design
The survey has been an annual census of the full population of eligible FFRDCs since FY 2001. Prior to FY 2001 only FFRDCs administered by academic institutions were surveyed.
c. Data collection techniques
The FY 2011 survey information was sent by e-mail in December 2011. Respondents could choose to submit an Adobe Portable Document Format questionnaire downloaded from the Web or use a Web-based data collection system to respond to the survey. Every effort was made to maintain close contact with respondents in order to preserve both the consistency and continuity of the resulting data. Questionnaires were carefully examined for completeness upon receipt. Computerized facsimiles of the survey data were then prepared for each FFRDC; these compared the current and 2 prior years of data and noted any substantive disparities. Respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data. These e-mail messages included a link to the survey Web-based data collection system, allowing respondents to view and correct their data online.
Respondents were asked to explain significant differences between current-year reporting and established patterns of reporting verified for prior years. They were encouraged to correct prior-year data if needed. When respondents updated or amended figures from past years, NSF made corresponding changes to trend data in this report and to the underlying microdata database. Forms were received from all of the 40 FFRDCs on the current NSF master list as of April 2012.
d. Estimation techniques
This survey is a census with 100% response. No estimation was required.
3. Survey Quality Measures
a. Sampling variability
The FY 2011 survey was a universe survey distributed to all FFRDCs; thus, there was no sampling error.
Because of the limited size of the FFRDC universe, there is little to no coverage error for this survey. FFRDCs are identified through the NSF master list of FFRDCs. NSF is responsible for maintaining the master list and queries all federal agencies annually to determine changes, additions, or deletions to the list.
The response rate for this survey is 100%; thus, there is no nonresponse bias in this survey.
One source of measurement error is the lack of institutional records for some of the categories requested on the survey. For example, some FFRDCs do not track their R&D expenditures by character of work (basic research, applied research, or development) and therefore must submit estimates of the amount spent within each category.
NSF discovered during the FY 2011 survey cycle that seven FFRDCs were including capital project expenditures in their R&D totals reported on the survey. Corrections were made for the FY 2011 survey cycle, which lowered the total expenditures by $468 million. However, previous years do include an unknown amount of capital expenditures in the total. The amount is estimated to be less than $500 million per year.
4. Trend Data
Annual data for all FFRDCs are available for FY 2001–11. Aggregate annual data for university-administered FFRDCs are available for FY 1953–2011. When the review for consistency between each year's data and submissions in prior years reveals discrepancies, it is sometimes necessary to modify prior years' data. Therefore, analysts studying trends are encouraged to use the most recent data from the National Center for Science and Engineering Statistics.
5. Data Availability
Data for FY 2008–11 of the FFRDC R&D Survey are available at http://www.nsf.gov/statistics/ffrdc/. Data from FY 2008 and prior years of the survey are included in the Academic R&D Expenditures series of detailed statistical tables, available by fiscal year at http://www.nsf.gov/statistics/rdexpenditures/.
Information from the survey is also included in the series Science and Engineering Indicators and National Patterns of R&D Resources, both available at http://www.nsf.gov/statistics/.
For more information about FFRDCs see NSF's Master Government List of FFRDCs. Data on federal obligations to FFRDCs are available in NSF's Survey of Federal Funds for Research and Development. NSF's Survey of Graduate Students and Postdoctorates in Science and Engineering collects information about postdoctorates working in FFRDCs.
6. Contact for more information
Additional information about this survey can be obtained by contacting:
Research and Development Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Phone: (703) 292-7765