1. bookVolume 36 (2020): Issue 1 (March 2020)
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Can Interviewer Evaluations Predict Short-Term and Long-Term Participation in Telephone Panels?

Published Online: 17 Mar 2020
Volume & Issue: Volume 36 (2020) - Issue 1 (March 2020)
Page range: 117 - 136
Received: 01 Jan 2019
Accepted: 01 Sep 2019
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

Interviewers often assess after the interview the respondent’s ability and reluctance to participate. Prior research has shown that this evaluation is associated with next-wave response behavior in face-to-face surveys. Our study adds to this research by looking at this association in telephone surveys, where an interviewer typically has less information on which to base an assessment. We looked at next-wave participation, non-contact and refusal, as well as longer-term participation patterns. We found that interviewers were better able to anticipate refusal than non-contact relative to participation, especially in the next wave, but also in the longer term. Our findings confirm that interviewer evaluations – in particular of the respondent’s reluctance to participate – can help predict response at later waves, also after controlling for commonly used predictors of survey nonresponse. In addition to helping to predict nonresponse in the short term, interviewer evaluations provide useful information for a long-term perspective as well, which may be used to improve nonresponse adjustment and in responsive designs in longitudinal surveys.

Keywords

Barrett, K., M. Sloan, and D. Wright. 2006. “Interviewer Perceptions of Interview Quality”. In Proceedings of the American Statistical Association, Survey Research Methodology Section, 4026–4033. Alexandria, VA: American Statistical Association. Available at: http://www.asasrms.org/Proceedings/y2006/Files/JSM2006-000644.pdf (accessed February 2020).Search in Google Scholar

Block, E.S. and L. Erskine. 2012 “Interviewing by Telephone: Specific Considerations, Opportunities, and Challenges.” International Journal of Qualitative Methods 11(4): 428–445. DOI: https://doi.org/10.1177/160940691201100409.10.1177/160940691201100409Search in Google Scholar

Casas-Cordero, C., F. Kreuter, Y. Wang, and S. Babey. 2013. “Assessing the Measurement Error Properties of Interviewer Observations of Neighbourhood Characteristics”, Journal of the Royal Statistical Society: Series A 176: 229–249. DOI: https://doi.org/10.1111/j.1467-985X.2012.01065.x.10.1111/j.1467-985X.2012.01065.x380438224159255Search in Google Scholar

Chun, A., B. Schouten and J. Wagner. 2017. JOS Special Issue on Responsive and Adaptive Survey Design: “Looking Back to See Forward-Editorial”. Journal of Official Statistics 33(3): 571–577. DOI: http://dx.doi.org/10.1515/JOS-2017-0027.10.1515/jos-2017-0027Search in Google Scholar

Couper, M.P. and M.B. Ofstedal. 2009 “Keeping in Contact with mobile Sample Members.” In Methodology of Longitudinal Surveys, edited by P. Lynn, 183–203. New York: Wiley.10.1002/9780470743874.ch11Search in Google Scholar

Eckman, S., J. Sinibaldi, and A. Möntmann-Hertz. 2013. “Can Interviewers effectively rate the Likelihood of Cases to cooperate?” Public Opinion Quarterly 77(2): 561–573. DOI: https://doi.org/10.1093/poq/nft012.10.1093/poq/nft012Search in Google Scholar

Feldman, J.J., H. Hyman, and C.W. Hart. 1951. “A Field Study of Interviewer Effects on the Quality of Survey Data.” Public Opinion Quarterly 15(4): 734–761. DOI: https://doi.org/10.1086/266357.10.1086/266357Search in Google Scholar

Groves, R.M., F.J. Fowler, M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology. 2nd Ed., Wiley Series in Survey Methods. Hoboken, NJ: John Wiley & Sons.Search in Google Scholar

Groves, R.M. and S.G. Heeringa. 2006. “Responsive Design for Household Surveys: Tools for actively controlling Survey Errors and Costs.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 169(3): 439–457. DOI: https://doi.org/10.1111/j.1467-985X.2006.00423.x.10.1111/j.1467-985X.2006.00423.xSearch in Google Scholar

Haynes, M., M. Western, and M. Spallek. 2005. “Methods for categorical longitudinal Survey Data: Understanding Employment Status of Australian Women.” Paper prepared for the HILDA Survey Research Conference, University of Melbourne, Australia, 29–30 September, 2005. Available at: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=2ahUKEwjfrIGR1sbnAhUkShUIHf1PAQgQFjABegQIBRAB&url=https%3A%2F%2Fwww.researchgate.net%2Fpublication%2F43458841_Methods_for_Categorical_Longitudinal_Survey_Data_Understanding_Employment_Status_of_Australian_Women&usg=AOvVaw26JHyUI5MCUOF5FqgSI1Nc (accessed February 2020).Search in Google Scholar

Kennedy, C.K. 2010. Nonresponse and Measurement Error in Mobile Phone Surveys. Doctoral thesis, University of Michigan, Ann Arbor. Available at: https://deepblue.lib.umich.edu/handle/2027.42/75977 (accessed February 2020).Search in Google Scholar

Kalton, G., J. Lepkowski, G.E. Montanari, and D. Maligalig. 1990. Characteristics of second Wave Nonrespondents in a Panel Survey. In Proceedings of the American Statistical Association, Survey Research Methodology Section, 462–467. Alexandria, VA: American Statistical Association.Search in Google Scholar

Kaminska, O., A.L. McCutcheon, and J. Billiet. 2010. “Satisficing among reluctant Respondents in a Cross-National Context.” Public Opinion Quarterly 74(5): 956–984. DOI: https://doi.org/10.1093/poq/nfq062.10.1093/poq/nfq062Search in Google Scholar

Kirchner, A., K. Olson, and J.D. Smyth. 2017. “Do Interviewer Postsurvey Evaluations of Respondents’ Engagement measure who Respondents are or what they do?” A Behavior coding Study. Public Opinion Quarterly: DOI: https://doi.org/10.1093/poq/nfx026.10.1093/poq/nfx026Search in Google Scholar

Laurie, H., R. Smith, and L. Scott. 1999. “Strategies for reducing Nonresponse in a longitudinal Panel Survey.” Journal of Official Statistics 15(2): 169–282. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/strategies-for-reducing-nonresponse-in-a-longitudinal-panel-survey.pdf (accessed February 2020).Search in Google Scholar

Lemay, M. 2009. Understanding the Mechanism of Panel Attrition. Unpublished Doctoral thesis, University of Maryland, College Park, MD, U.S.A.Search in Google Scholar

Lepkowski, J.M. and M.P. Couper. 2002. “Nonresponse in the second Wave of longitudinal Household Surveys.” In Survey nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J. Little, 259–272. New York: Wiley.Search in Google Scholar

Lipps, O. 2012a. Using Information from Telephone Panel Surveys to predict Reasons for Refusal. Methods, data, analyses 6(1): 3–20. Available at: https://nbn-resolving.org/urn:nbn:de:0168-ssoar-314544 (accessed August 2019).Search in Google Scholar

Lipps, O. 2012b. “A Note on improving Contact Times in Panel Surveys.” Field Methods 24(1): 95–111. DOI: https://doi.org/10.1177/1525822X11417966.10.1177/1525822X11417966Search in Google Scholar

Little, R.J. and S. Vartivarian. 2005. “Does weighting for Nonresponse increase the Variance of Survey Means?” Survey Methodology 31(2): 161–168. Available at: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=2ahUKEwiTkPGq2MbnAhXHUBUIHWLjCq4QFjACegQIBRAB&url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.692.507%26rep%3Drep1%26type%3Dpdf&usg=AOvVaw1fkEbnvfviCWWInhD3jfs6 (accessed February 2020).Search in Google Scholar

Lugtig, P. 2014. “Panel Attrition: Separating Stayers, fast Attriters, gradual Attriters, and Lurkers.” Sociological Methods & Research 43(4): 699–723. DOI: https://doi.org/10.1177/0049124113520305.10.1177/0049124113520305Search in Google Scholar

Perez, F.P. and B. Baffour. 2018. “Respondent mental Health, mental Disorders and Survey Interview Outcomes.” Survey Research Methods 12(2): 161–176. DOI: https://doi.org/10.18148/srm/2018.v12i2.7225.Search in Google Scholar

Peytchev, A. and K. Olson. 2007. “Using Interviewer Observations to improve Nonresponse Adjustments: NES 2004.” In Proceedings of the American Statistical Association, Survey Research Methodology Section, 3364–3371. Alexandria, VA: American Statistical Association. Available at: https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1149&context=sociologyfacpub (accessed February 2020).Search in Google Scholar

Plewis, I., L. Calderwood, and T Mostafa. 2017. “Can Interviewer Observations of the Interview Predict future Response?” Methods, data, analyses 11(1): 29–44. DOI: https://doi.org/10.12758/mda.2016.010.Search in Google Scholar

Schonlau, M., N. Watson, and M. Kroh. 2010 “Household Survey Panels: How much do following Rules affect Sample Size?” Survey Research Methods 5(2): 53–61. DOI: https://doi.org/10.18148/srm/2011.v5i2.4665.Search in Google Scholar

Schouten, B., A. Peytchev, and J. Wagner. 2017. Adaptive Survey Design. Boca Raton: CRC Press.10.1201/9781315153964Search in Google Scholar

Sinibaldi, J., G. Durrant, and F. Kreuter. 2013. “Evaluating the Measurement Error of Interviewer observed Paradata.” Public Opinion Quarterly 77: 173–193. DOI: https://doi.org/10.1093/poq/nfs062.10.1093/poq/nfs062Search in Google Scholar

Stoop, I. 2005. The Hunt for the Last Respondent, Nonresponse in Sample Surveys. The Hague: Social and Cultural Planning Office of the Netherlands. Available at: https://dspace.library.uu.nl/bitstream/handle/1874/2900/full.pdf (accessed February 2020).Search in Google Scholar

Tillmann, R., M. Voorpostel, U. Kuhn, F. Lebert, V-A. Ryser, O. Lipps, B. Wernli, and E. Antal. 2016. “The Swiss Household Panel Study: Observing social Change since 1999.” Longitudinal and Life Course Studies 7(1): 64–78. DOI: http://dx.doi.org/10.14301/llcs.v7i1.360.10.14301/llcs.v7i1.360Search in Google Scholar

Uhrig, N.S.C. 2008. The Nature and Causes of Attrition in the British Household Panel Survey. ISER Working Paper 2008-05. Colchester: ISER, University of Essex. Available at: https://www.researchgate.net/profile/Sc_Uhrig/publication/242116089_The_Nature_and_Causes_of_Attrition_in_the_British_Household_Panel_Survey/links/0a85e537c6cd161804000000/The-Nature-and-Causes-of-Attrition-in-the-British-Household-Panel-Survey.pdf (accessed February 2020).Search in Google Scholar

Voorpostel, M. 2010. Attrition Patterns in the Swiss Household Panel by Demographic Characteristics and Social Involvement. Swiss Journal of Sociology 36(2): 359–377. Available at: https://serval.unil.ch/resource/serval:BIB_78D83DD99A6F.P001/REF (accessed February 2020).Search in Google Scholar

Voorpostel, M. and O. Lipps. 2011. Attrition in the Swiss Household Panel: Is change associated with drop-out? Journal of Official Statistics 27(2): 301–318. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/attrition-in-the-swiss-household-panel-is-change-associated-with-drop-out.pdf (accessed February 2020).Search in Google Scholar

Wagner, James. 2008. Adaptive Survey Design to Reduce Nonresponse Bias, Dissertation, University of Michigan.Search in Google Scholar

Watson, Nicole, and Mark Wooden. 2004. Sample attrition in the HILDA survey. Australian Journal of Labour Economics 7(2): 293–308. Available at: https://businesslaw.curtin.edu.au/wp-content/uploads/sites/5/2016/05/AJLE-v7n2-watson.pdf (accessed February 2020).Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo