Work addressed two specific research questions related to multi-mode surveys: 1) Are the measurement error differences between modes ignorable? and 2) What are the properties of statistical inference methods that incorporate nonignorable measurement error differences under a mixed-mode survey design? Work was completed on three alternative mixed-mode survey mean estimators.  The general steps in the estimation method were: 1) Impute counterfactual data for the alternative mode(s) under specified mode choice and response models, 2) Use both observed and actual data for the complete set of respondents to compute mode specific survey means, 3) Adjust for nonresponse, 4) Compare mode specific mean estimates, and 5) Combine mode specific means under three alternative combining rules. Step 1) created data that were counterfactual in the sense that imputations for cases that responded using one mode were made as if they had responded via another mode. For the purpose of this research, nonresponse adjustments were omitted, although future research should address it.

Using 1973 Current Population Survey (CPS)-Internal Revenue Service (IRS) Match Data and 2012 CPS data, comparisons were made between the naïve and three alternative methods for Income and Health Insurance Coverage. Three alternative methods were: 1) Simple average estimator, in which mode specific means are combined by using a simple average estimator, 2) Combined estimator in which estimated means are weighted inversely according to their variances, and 3) Combined estimator in which estimated means are weighted inversely according to their mean square errors (MSEs).   A fourth choice was the naïve mean estimator ignoring any mode effects, i.e., 4) Combine data together in one sample and estimate the mean ignoring any differences in modes.  An empirical study and a simulation study on 1973 CPS Match Data were conducted to study the bias properties of naïve estimator and the three alternative estimators. A sensitivity analysis was conducted using the 2012 CPS data.

The empirical study was conducted on a subset of 1973 CPS-IRS Match Data using wage and salary income as reported in the CPS. CPS-reported wage and salary income for 1972 were compared against the 1972 person-level IRS wage and salary income data. The average relative differences by mode were not significantly different between the CPS in-person and the telephone modes. On the other hand, the direct comparisons of individual in-person respondents’ earnings to telephone respondent earnings suggest a possible nonignorable mode choice mechanism, that is, the mode choice mechanism depends on the variable itself. On average, in-person respondents reported their earnings $1,369 less per year than telephone respondents. After controlling for personal characteristics, education, work experience, race (white vs. other), occupation type (professional, sales, craft, laborer), and industry (construction, manufacturing, transportation, trade, service) and residential (household) characteristics, central city, suburb, region, the difference remained significant. The empirical study considers this subset as the population and draws 50 simulation samples under 400/800 sample sizes of persons. In addition to sample size variation, the simulations also varied two imputation methods: 1) ignorable mode choice, and 2) nonignorable mode choice and 1) item missing included, and 2) item missing excluded conditions. The results were in the expected direction. That is, the standard procedure of ignoring potential mode differences led to biased estimators.  The estimator that combined means inversely according to MSEs was generally least biased.  For this estimator to be feasible in practice, one of the modes must be selected as producing the most nearly unbiased estimators.  The differences between item mi...

Please report errors in award information by writing to: awardsearch@nsf.gov.