Appendix | Methodology
Glossary
Accuracy: Accuracy is the difference between the estimate and the true parameter value.
Administrative records: Microdata records collected for the purpose of carrying out various programs (e.g., tax collection). Unlike survey data, administrative data were not originally collected for statistical purposes.
Bias: Systematic deviation of the survey estimated value from the true population value. Bias refers to systematic errors that can occur with any survey under a specific design.
Census: A data collection that seeks to obtain data directly from all eligible units in the entire target population. It can be considered a sample with a 100% sampling rate. A census may use data from administrative records for some units rather than direct data collection.
Coverage: Extent to which all elements on a frame list are members of the population and to which every element in a population appears on the frame list once and only once.
Coverage error: Discrepancy between statistics calculated on the frame population and the same statistics calculated on the target population. Undercoverage errors occur when target population units are missed during frame construction, and overcoverage errors occur when units are duplicated, enumerated incorrectly, or are not part of the target population.
Cross-sectional sample survey: Based on a representative sample of respondents drawn from a population at a particular point in time.
Estimate: A numerical value for a population parameter derived from information collected from a survey or other sources.
Estimation error: Difference between a survey estimate and the true value of the parameter in the target population.
Frame: A mapping of the universe elements (i.e., sampling units) onto a finite list (e.g., the population of schools on the day of the survey).
Item nonresponse: Occurs when a respondent fails to respond to one or more relevant items on a survey.
Longitudinal sample survey: Follows the experiences and outcomes over time of a representative sample of respondents (i.e., a cohort).
Measurement error: Difference between observed values of a variable recorded under similar conditions and some fixed true value (e.g., errors in reporting, reading, calculating, or recording a numerical value).
Nonresponse bias: Occurs when the observed value deviates from the population parameter due to systematic differences between respondents and nonrespondents. Nonresponse bias may occur as a result of not obtaining 100% response from the selected units.
Nonresponse error: Overall error observed in estimates caused by differences between respondents and nonrespondents. It consists of a variance component and nonresponse bias.
Nonsampling error: Includes specification errors and measurement errors due to interviewers, respondents, instruments, and mode; nonresponse error; coverage error; and processing error.
Parameter: An unknown, quantitative measure (e.g., total revenue, mean revenue, total yield, or number of unemployed people) for the entire population or for a specified domain of interest.
Population: The set of persons or organizations to be studied, which may not be of finite size.
Precision of survey results: How closely results from a sample can reproduce the results that would be obtained from a complete count (i.e., census) conducted using the same techniques. The difference between a sample result and the result from a complete census taken under the same conditions is an indication of the precision of the sample result.
Probabilistic methods: Any of a variety of methods for survey sampling that gives a known, nonzero probability of selection to each member of a target population. The advantage of probabilistic sampling methods is that sampling error can be calculated. Such methods include random sampling, systematic sampling, and stratified sampling. They do not include convenience sampling, judgment sampling, quota sampling, and snowball sampling.
Reliability: Degree to which a measurement technique would yield the same result each time it is applied. A measurement can be both reliable and inaccurate.
Response bias: Deviation of the survey estimate from the true population value due to measurement error from the data collection. Potential sources of response bias include the respondent, the instrument, the mode of data collection, and the interviewer.
Response rates: These measure the proportion of the sample frame represented by the responding units in each study.
Sample design: Refers to the combined target population, frame, sample size, and sample selection methods.
Sample survey: A data collection that obtains data from a sample of the frame population.
Sampling error: Error that occurs because all members of the frame population are not measured. It is associated with the variation in samples drawn from the same frame population. The sampling error equals the square root of the variance.
Standard error: Standard deviation of the sampling distribution of a statistic. Although the standard error is used to estimate sampling error, it includes some nonsampling error.
Statistical significance: Attained when a statistical procedure applied to a set of observations yields a p-value that exceeds the level of probability at which it is agreed that the null hypothesis will be rejected.
Target population: Any group of potential sample units or individuals, businesses, or other entities of interest.
Unit nonresponse: Occurs when a respondent fails to respond to all required response items (i.e., fails to complete or return a data collection instrument).
Universe survey: Involves the collection of data covering all known units in a population (i.e., a census).
Validity: Degree to which an estimate is likely to be true and free of bias (systematic errors).