Overview 
Survey Design
Survey Quality Measures
Trend Data
Availability of Data
1. Overview 
This section provides a brief discussion of the methodology used in the conduct of the Survey of Research and Development Funding and Performance by Nonprofit Organizations. For a more in-depth discussion of the methodology used in conducting this survey, contact the survey project officer.
a. Purpose
The Survey of Research and Development Funding and Performance by Nonprofit Organizations: 1996 and 1997 collected information on the science and engineering (S&E) research and development (R&D) activities of nonprofit organizations (NPOs). It collected data both from NPOs that fund S&E R&D and from those that perform R&D themselves. The study was done to meet needs for information about the role of the nonprofit sector in funding and conducting S&E R&D in the United States.
b. Respondents
The survey respondents are U.S. nonprofit organizations that fund or perform R&D.
c. Key variables
- Nonprofit organization
- Nonprofit performer of R&D
- Expenditures for intramural R&D
- Character of work (basic research, applied research, and development)
- Fields of science and engineering
- Sources of funds (Federal, state, local, industry, universities and colleges, other nonprofit organizations, other sources including own funds and all foreign sources)
- Type of nonprofit R&D performer (research institutes, hospitals, professional or technical societies, private foundations, science exhibitors, trade associations, industrial consortia, academic consortia)
- Extramural funding of R&D at other institutions
- Nonprofit funder of R&D
- Types of nonprofit funders of R&D (independent, corporate, family, community and operating foundations; public charities; professional or technical societies and academies of science or engineering; trade associations)
- Funds provided for medical and non-medical R&D, and R&D capital
- R&D funds provided to types of recipients (colleges and universities, hospitals, research institutes, professional or technical societies and academies of sciences, private foundations, science exhibitors, industrial consortia, academic consortia, agricultural cooperatives, industry, Federally Funded Research and Development Centers, other U.S. institutions, all types outside the U.S.)
2. Survey Design
a. Target population and sample frame
The overall target population for the survey was nonprofit organizations that funded or perform S&E R&D of $250,000 or more during fiscal year 1996.
b. Sample design
Two frames were needed for the study, one consisting of possible research funders (independent nonprofit organizations funding S&E R&D) and the other consisting of possible research performers (independent nonprofit organizations conducting science and engineering research and/or development). Both frames were to exclude organizations already eligible for related NSF surveys, e.g., colleges and universities, industry, and government agencies. The sample of research funders was restricted to NPOs listed in the Foundation Directory of the Foundation Center (NY) with assets of at least $2 million or $50,000 in annual giving. The sample of research performers was based on three lists of highly likely R&D performers plus two different samples from an Internal Revenue Service (IRS) public-use file of approximately 600,000 NPOs. These two IRS file samples were 1) a probability- proportional-to-size sample of likely R&D performers, i.e., those that had a National Taxonomy of Exempt Entities code that indicated an interest in science, engineering or technology, and 2) a probability-proportional-to-size sample of all other NPOs in the IRS file. The National Center for Charitable Statistics (NCCS) trimmed the IRS file to fewer than 185,000 NPOs after removing private foundations, NPOs that had gross receipts of less than $25,000, and religious organizations.
c. Data collection techniques
Survey questionnaires were mailed in 1998 and 1999 to 1,131 organizations that had indicated on a National Science Foundation (NSF) screening form that the organizations funded or performed R&D worth at least $250,000 in 1996. Through August 1999 NSF mailed follow-up questionnaires to nonrespondent organizations, and from August through October 1999 NSF attempted to contact all nonrespondent organizations by telephone. During the course of the data-collection phase of the survey, 126 organizations that neither funded nor performed R&D were deleted from the survey universe. Thus, as of the closeout date of December 15, 1999, the survey universe comprised 1,005 organizations.
Of these 1,005 organizations, 352 or 35 percent, returned usable replies. During post-processing cleaning of the data, the 352 respondents were reduced to 343 respondents.
The survey was conducted by the Gallup under contract to the Division of Science Resources Studies.
d. Estimation techniques
The survey data were weighted to national estimates. The weights were calculated to reflect: (1) different selection probabilities for the different sample NPOs; (2) adjustments to compensate for different rates of nonresponse to the screening effort; and (3) adjustments for non-response to the main survey. Item nonresponse was addressed through logical and hot deck imputation.
3. Survey Quality Measures
a. Sampling variability
The sample was designed to have a representative sample of U.S. R&D-performing and R&D-funding nonprofit organizations. The 41 percent response rate was lower than anticipated and sampling error in individual data cells, especially the smaller cells, are quite high. The national estimates for total nonprofit R&D and major items, such as basic research, applied research and development, are presented with sufficient reliability for use.
b. Coverage
The sample was carefully designed with probability proportional to size and used the largest NPO database in existence, the 1996 Internal Revenue Service (IRS) public-use file of approximately 600,000 NPOs. Therefore, coverage at the institutional level is believed to be at or close to 100 percent. There may be minor coverage problems for nonprofit organizations that were established just prior to the 1996 survey year.
c. Nonresponse
(1) Unit nonresponse - Of the NPOs surveyed for 1996 and 1997, 59 percent did not respond. Weighting was carried out in three stages to correct for unit nonresponse. The first stage compensated for the different selection probabilities for the different sample NPOs. The initial weight (W1) for a given NPO was simply the inverse of its probability of selection into the sample. In the second stage, this base weight was adjusted to compensate for different rates of nonresponse to the screening effort. This nonresponse adjustment was calculated as the inverse of the screener response rate within eight weighting cells; the adjusted weight (W2) was the product of the initial weight and the inverse of the response rate for that cell. The final weight incorporated an adjustment for nonresponse to the main survey.
(2) Item nonresponse - In an attempt to complete all items, staff telephoned NPOs that left some data cells blank on their questionnaires. Then Gallup dealt with item nonresponse through logical and hot deck imputation. Gallup identified all missing data that could be resolved logically and used hot deck imputation for the remaining cells.
d. Measurement
Variations in respondent interpretations of the definitions of R&D activities are of particular concern. Specifically, some NPOs thought "extramural R&D funding" referred to funds they received from the Federal government and other sources rather than funds they gave to other organizations. Gallup and NSF staff telephoned NPOs for clarification when the data looked questionable.
For future surveys, NSF will minimize measurement error by more questionnaire pre-testing, improvement of questionnaire wording and format, and additional consultations with respondents.
4. Trend Data
The last survey of nonprofit organizations' R&D was for performers only. It collected Fiscal Year 1973 data. Selected data from FY 1973 are reproduced with the 1996 and 1997 data. Data users should read all technical notes and footnotes in the tables to understand how the 1973 coverage differs from that of the 1996 and 1997 survey.
5. Availability of Data
a. Publications
The data from this survey will be published in Early Release Tables and in Detailed Statistical Tables for the Survey of R&D Funding & Performance by Nonprofit Organizations: 1996 and 1997. Both sets of Tables and a Data Brief will be available on the SRS Web site. Data for major data elements for 1973 will be reproduced in the 1996 and 1997 sets of tables. Information from the 1996 and 1997 survey is also included in National Patterns of R&D Resources: 2000.
b. Electronic access
Data from this survey will be available on the SRS Web site.
c. Contact for more information
Additional information about this survey can be obtained by contacting:
John E. Jankowski
Program Director
Research and Development Statistics Program
Division of Science Resources Statistics
National Science Foundation
2415 Eisenhower Avenue, Suite W14200
Alexandria, VA 22314
Phone: (703) 292-7781
E-mail: jjankows@nsf.gov
