1. bookVolume 37 (2021): Issue 2 (June 2021)
    Special Issue on New Techniques and Technologies for Statistics
Journal Details
License
Format
Journal
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Assessing and Adjusting Bias Due to Mixed-Mode in Aspect of Daily Life Survey

Published Online: 22 Jun 2021
Page range: 461 - 480
Received: 01 Jun 2019
Accepted: 01 Oct 2020
Journal Details
License
Format
Journal
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

The mixed-mode (MM) designs are adopted by NSIs both to contrast declining response and coverage rates and to reduce the cost of the surveys. However, MM introduces several issues that must be addressed both at the design phase, by defining the best collection instruments to contain the measurement error, and at the estimation phase, by assessing and adjusting the mode effect. In the MM surveys, the mode effect refers to the introduction of bias effects on the estimate of the parameters of interest due to the difference in the selection and measurement errors specific to each mode. The switching of a survey from single to mixed-mode is a delicate operation: the accuracy of the estimates must be ensured in order to preserve their consistency and comparability over time. This work focuses on the methods chosen for the evaluation of the mode effect in the Italian National Institute of Statistics (ISTAT) mixed-mode survey “Aspects of Daily Life – 2017”, in the experimental context for which an independent control single-mode (SM) PAPI sample was planned to assess the introduction of the sequential web/PAPI survey. The presented methods aim to analyze the causes that can determine significant differences in the estimates obtained with the SM and MM surveys.

Keywords

Austin, P.C., and D.S. Small. 2014. “The use of bootstrapping when using propensity-score matching without replacement: a simulation study.” Statistics in Medicine 33(24) : 4306–4319. Wiley Online Library. DOI: https://doi.org/10.1002/sim.6276.Search in Google Scholar

Ballabio, S., A. Carra, S. Casacci, D. Ferrazza, F. Verrecchia, A. Vitalini, and L.C. Viviano. 2018. “Local decisions and new guidelines of the Official Statistics.” In Proceedings of Q2018 European Conference on Quality in Official Statistics, June 2018, Cracovia, Poland.Search in Google Scholar

Bethlehem, J., F. Cobben, and B. Schouten. 2011. Handbook of nonresponse in household surveys. Wiley Series in Survey Methodology. Hoboken, New Jersey: John Wiley & Sons, Inc. DOI: https://doi.org/10.1002/9780470891056.Search in Google Scholar

Buelens, B., and J.A. van den Brakel. 2014. “Measurement error calibration in mixed-mode Sample Surveys.” Sociological methods and Research 44(3) : 391–426. DOI: https://doi.org/10.1177/0049124114532444.Search in Google Scholar

Buelens, B., J.A. van den Brakel, and B. Schouten. 2018. “Current methodologies to deal with mode effects and mode bias in multi-mode designs.” MIMOD Deliverable 1 – WP2. Available at: https://www.istat.it/en/research-activity/international-research-activity/essnet-and-grants (accessed May 2021).Search in Google Scholar

De Leeuw, E.D. 2005. “To mix or not to mix data collection modes in surveys.” Journal of Official Statistics 21(2) : 233–255. Available at: https://www.scb.se/contentassets/-ca21efb41fee47d293bbee5bf7be7fb3/to-mix-or-not-to-mix-data-collection-modes-in-surveys.pdf (accessed May 2021).Search in Google Scholar

Efron, B. 1979. “Bootstrap methods: Another look at the jackknife.” The Annals of Statistics 7(1) : 1–26. DOI: https://doi.org/10.1214/aos/1176344552.Search in Google Scholar

Efron, B., and R. Tibshirani. 1986. “Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy.” Statistical Science 1(1) : 54–75. DOI: https://doi.org/10.1214/ss/1177013815.Search in Google Scholar

Eurostat. 2014. ESS Guidelines for the Implementation of the ESS Quality and Performance Indicators (QPI). Luxembourg ESTAT / D4/LA D (2014). Available at: https://ec.europa.eu/eurostat/documents/64157/4373903/02-ESS-Quality-and-performance-Indicators-2014.pdf/5c996003-b770-4a7c-9c2f-bf733e6b1f31 (accessed May 2021).Search in Google Scholar

Garofalo, G. 2014. “Il Progetto ARCHIMEDE obiettivi e risultati sperimentali.” Istat Working Paper. Available at: https://www.istat.it/it/files/2014/11/IWP-n.-9-2014.pdf (accessed February 2015).Search in Google Scholar

Hox, J., E.D. de Leeuw, and T. Klausch. 2015. “Mixed Mode Research: Issues in Design and Analysis.” In Total Survey Error in Practice, edited by P.P. Biemer et al.: 511–530. New York: John Wiley & Sons. DOI: http://dx.doi.org/10.1002/9781119041702.ch23.Search in Google Scholar

ISTAT (2019). “Aspetti metodologici dellindagine. Aspetti della vita quotidiana Periodo di riferimento: anno 2017”. Rome: ISTAT. Available at: https://www.istat.it/microdata/download.php?id=/import/fs/pub/wwwarmida/60/2017/01/Nota.pdf (accessed May 2021).Search in Google Scholar

Martin, P., and P. Lynn. 2011. “The effects of mixed mode survey designs on simple and complex analyses.” ISER Working Paper Series 2011–28. Available at: https://ideas.repec.org/p/ese/iserwp/2011-28.html (accessed May 2021).Search in Google Scholar

Rosenbaum, P.R., and D.B. Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70: 41–55. DOI: https://doi.org/10.1093/biomet/70.1.41.Search in Google Scholar

Rust, K.F., and J.N.K. Rao. 1996. “Variance estimation for complex surveys using replication techniques.” In Statistical methods in medical research 5(3) : 283–310. DOI: https://doi.org/10.1177%2F096228029600500305.Search in Google Scholar

Schouten, B.F., F. Cobben, and J. Bethlehem. 2009. “Indicators for the representativeness of survey response.” In Survey Methodology 35(1) : 101–113. Available at: Indicators for the representativeness of survey response (statcan.gc.ca).Search in Google Scholar

Schouten, B., N. Shlomo, and C. Skinner. 2011. “Indicators for Monitoring and Improving Representativity of Response.” Journal of Official Statistics 27: 231–253. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/indicators-for-monitoring-and-improving-representativeness-of-response.pdf (accessed May 2021).Search in Google Scholar

Tourangeau, R., and T. Yan. 2007. “Sensitive questions in surveys.” Psychological Bulletin 133(5) : 859–883. DOI: https://doi.org/10.1037/0033-2909.133.5.859.Search in Google Scholar

Vandenplas, C., G. Loosveldt, and J.T.A. Vannieuwenhuyze. 2016. “Assessing the use of mode preference as a covariate for the estimation of measurement effects between modes. A sequential mixed mode experiment.” Methods, data, analyses (mda) 10(2) : 119–142. DOI: https://doi.org/10.12758/mda.2016.011.Search in Google Scholar

Vannieuwenhuyze, J.T A., G. Loosveldt, and G. Molenberghs. 2010. “A Method for Evaluating Mode Effects in Mixed-mode Surveys.” Public Opinion Quarterly 74(5) : 1027–1045. DOI: https://doi.org/10.1093/poq/nfq059.Search in Google Scholar

Wolter, K.M. 2007. Introduction to variance estimation. Statistical for Social and Behavioral Science. New York: Springer Science & Business Media, LLC.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo