Survey Overview (FY 2021 survey cycle)

Purpose.

The Federally Funded Research and Development Center (FFRDC) R&D Survey is conducted by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF). It is the primary source of information on separately accounted for R&D expenditures at FFRDCs in the United States.

Data collection authority.

The information is solicited under the authority of the National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010. The Office of Management and Budget control number is 3145-0100, with an expiration date of 31 August 2022.

Major changes to recent survey cycle.

None.

Key Survey Information

Frequency.

Annual.

Initial survey year.

2001.

Reference period.

FY 2021.

Response unit.

Establishment.

Sample or census.

Census.

Population size.

43.

Sample size.

Not applicable. The survey is a census of all known eligible FFRDCs.

Key variables.

Key variables of interest are listed below.

• R&D expenditures by source of funds (federal, state and local, business, nonprofit, or other)

• R&D expenditures by federal agency source

• R&D expenditures by type of R&D (basic research, applied research, or experimental development)

• R&D expenditures by type of costs (salaries, software, equipment, subcontracts, other direct costs, and indirect costs)

• Total operating budget

Survey Design

Target population.

All FFRDCs.

Sampling frame.

The total survey universe is identified through the National Science Foundation (NSF) Master Government List of FFRDCs. NSF is responsible for maintaining this list and queries all federal agencies annually to determine any changes to, additions to, or deletions from the list.

Sample design.

Not applicable. The FFRDC R&D Survey is a census of all eligible organizations.

Data Collection and Processing

Data collection.

The FY 2021 survey announcements were sent by e-mail to all FFRDCs in November 2021. The data collection period concluded in March 2022. Respondents could choose to submit an Adobe PDF questionnaire downloaded from the Web or use a Web-based data collection system to respond to the survey. Every effort was made to maintain close contact with respondents to preserve the consistency and continuity of the resulting data. Survey data reports were available on the survey website for each institution; these reports showed comparisons between the current year and the 2 prior years of data and noted any substantive disparities.

Data processing.

Respondents were asked to explain significant differences between current and past year reporting upon the submission of their surveys. Questionnaires were then carefully examined by survey staff. Reviews focused on unexplained missing data and explanations provided for changes in reporting patterns. If additional explanations or data revisions were needed, respondents were sent personalized e-mails asking them to provide any necessary revisions before the final processing and tabulation of data. Respondents were encouraged to correct prior year data, if necessary. When respondents updated or amended figures from past years, NCSES made corresponding changes to trend data in the 2021 data tables and to the underlying microdata. For accurate historical data, use only the most recently released data tables.

Estimation techniques.

No data were imputed for FY 2021.

Survey Quality Measures

Sampling error.

Because the FY 2021 survey was a survey distributed to all organizations in the universe, there was no sampling error.

Coverage error.

Because of the availability of a comprehensive FFRDC list, there is no known coverage error for this survey. FFRDCs are identified through the NSF Master Government List of FFRDCs. NSF is responsible for maintaining the master list and queries all federal agencies annually to determine changes to, additions to, or deletions from the list.

Nonresponse error.

Most FFRDCs have incorporated the data needed to complete most of the survey questions into their record-keeping systems. Twelve FFRDCs choose not to complete Question 5 of the survey, which asks for expenditures by type of cost. Eleven of those FFRDCs are managed by private companies for whom salary information is considered proprietary. One FFRDC, which is managed by a university, does not capture direct and indirect costs in their financial system in a way that is reportable on Question 5. Other FFRDCs did not answer all sections of Question 5: two FFRDCs could not provide information on software expenditures, and four could not provide data on equipment expenditures. One FFRDC did not report its operating budget (Question 6). One FFRDC could not provide expenditures funded by nonprofit organizations separately from those funded by businesses (Question 1); the combined amount was reported as business funded.

Measurement error.

NCSES discovered during the FY 2011 survey cycle that seven FFRDCs included capital project expenditures in the R&D totals reported on the survey. Corrections made for the FY 2011 survey cycle lowered total expenditures by $468 million. However, previous years still include an unknown amount of capital expenditures in the total. The amount is estimated to be less than $500 million per year.

Prior to the FY 2011 survey, the five FFRDCs administered by the MITRE Corporation had reported only internally funded R&D expenditures. After discussions with NCSES, these five FFRDCs agreed to report all FY 2011 operating expenditures for R&D and to revise their data for FYs 2008–10.

NCSES discovered during the FY 2013 survey cycle that Los Alamos National Laboratory (LANL) was reporting some expenditures that were not for R&D as defined by this survey. Corrections made for the FY 2013 survey cycle lowered the laboratory's total expenditures by $349 million. LANL also incorrectly was reporting that all expenditures were for basic research. In corrections made for FY 2013, LANL reported that $1,554 million (91%) of its total research expenditures was for applied research. LANL data from previous years still include an unknown amount of expenditures that were not for R&D and categorize all expenditures as basic research.

Prior to FY 2014, the Aerospace FFRDC reported only expenditures on internal R&D projects. After discussions with NCSES, the Aerospace Corporation agreed to report all R&D expenditures for FY 2014 and provide revised data to include all R&D expenditures for FYs 2010–13. R&D expenditures increased by more than $800 million each year.

During the FY 2014 survey, NCSES discovered that the National Optical Astronomy Observatory had been including data for the National Solar Observatory since FY 2010. The Association of Universities for Research in Astronomy, the administrator of both FFRDCs, provided revised data for both FFRDCs for FYs 2010–13.

During the FY 2016 survey, NCSES discovered that the Judiciary Engineering and Modernization Center was incorrectly classified as an industry-administered FFRDC in FYs 2011–15. This FFRDC is administered by the MITRE Corporation, a nonprofit organization and should have been classified as a nonprofit-administered FFRDC. The classification was corrected for FY 2016 and the Judiciary Engineering and Modernization Center's FYs 2011–15 data were reclassified as coming from a nonprofit-administered FFRDC.

Prior to the FY 2021 collection, NCSES discovered that the Green Bank Observatory had split from the National Radio Astronomy Observatory on 1 October 2016, both retained FFRDC status. For FYs 2017–20, R&D expenditures reported for the National Radio Astronomy Observatory, include the expenditures for the Green Bank Observatory. The Green Bank Observatory began reporting separately on the FY 2021 survey.

Data Availability and Comparability

Data availability.

Annual data are available for all FFRDCs for FYs 2001–21.

Data comparability.

When the review for consistency between each year's data and submissions in prior years reveals discrepancies, it is sometimes necessary to modify prior years' data. For accurate historical data, use only the most recently released data tables. Individuals wishing to analyze trends other than those in the most recent NCSES publication are encouraged to contact the Survey Manager for more information about the comparability of data over time.

Data Products

Publications.

The data from this survey are published annually in data tables in the series R&D Expenditures at Federally Funded R&D Centers (https://www.nsf.gov/statistics/ffrdc/). Information from this survey is also included in Science and Engineering Indicators (https://ncses.nsf.gov/indicators/).

Electronic access.

Public use files are available at https://www.nsf.gov/statistics/ffrdc/pub_data.cfm.

Contact Information

For additional information about this survey, or the methodology report, please contact the Survey Manager.

Michael T. Gibbons
Survey Manager
Research and Development Statistics Program, NCSES
Phone: (703) 292-4590
E-mail: mgibbons@nsf.gov

National Center for Science and Engineering Statistics
Directorate for Social, Behavioral and Economic Sciences
National Science Foundation
2415 Eisenhower Avenue, Suite W14200
Alexandria, VA 22314
Tel: (703) 292-8780
FIRS: (800) 877-8339
TDD: (800) 281-8749
E-mail: ncsesweb@nsf.gov