Spanish exit polls. Sampling error or nonresponse bias?

  1. Jose M. Pavía 1
  2. Elena Badal 1
  3. Belén García-Cárceles 1
  1. 1 Universidad de Valencia, España
Journal:
Revista internacional de sociología

ISSN: 0034-9712

Year of publication: 2016

Volume: 74

Issue: 3

Type: Article

DOI: 10.3989/RIS.2016.74.3.043 DIALNET GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Revista internacional de sociología

Sustainable development goals

Abstract

Countless examples of misleading forecasts on behalf of both pre-election and exit polls can be found all over the world. Non-representative samples due to differential nonresponse have been claimed as being the main reason for inaccurate exit-poll projections. In real inference problems, it is seldom possible to compare estimates and true values. Electoral forecasts are an exception. Comparisons between estimates and final outcomes can be carried out once votes have been tallied. In this paper, we examine the raw data collected in seven exit polls conducted in Spain and test the likelihood that the data collected in each sampled voting location can be considered as a random sample of actual results. Knowing the answer to this is relevant for both electoral analysts and forecasters as, if the hypothesis is rejected, the shortcomings of the collected data would need amending. Analysts could improve the quality of their computations by implementing local correction strategies. We find strong evidence of nonsampling error in Spanish exit polls and evidence that the political context matters. Nonresponse bias is larger in polarized elections and in a climate of fear.

€ View funding

Funding information

Support from the Spanish Ministry of Economics and Competitiveness through project CSO2013-43054-R is also acknowledged.

Funders

Bibliographic References

  • Bautista, R., Callegaro, M., Vera, J.A. and Abundis, F. 2007. Studying nonresponse in Mexican exit polls. International Journal of Public Opinion Research, 19, 492– 503. http://dx.doi.org/10.1093/ijpor/edm013
  • Beltrán, U. and Valdivia, M. 1999. Accuracy and error in electoral forecasts: The case of Mexico. International Journal of Public Opinion Research, 11, 115–134. http://dx.doi.org/10.1093/ijpor/11.2.115
  • Bernardo, J.M. 1984. Monitoring the 1982 Spanish socialist victory: A Bayesian analysis, Journal of the American Statistical Association, 79, 510–515. http://dx.doi.org/10.1080/01621459.1984.10478077
  • Bernardo, J.M. 1997. Probing public opinion: The state of Valencia experience. In C. Gatsonis, J.S. Hodges, R.E. Kass, R. McCulloch, P. Rossi, and N.D. Singpurwalla, (eds.), Bayesian Case Studies 3 (pp. 3–21). New York: Springer-Verlag. http://dx.doi.org/10.1007/978-1-4612-2290-3_1
  • Bernardo, J.M. and Girón, F.J. 1992. Robust sequential prediction from non-random samples: The election night forecasting case. In J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith (eds.), Bayesian Statistics 4 (pp. 61–77). Oxford: Oxford University Press.
  • Best, S.M. and Krueger, B.S. 2012. Exit Polling: Surveying the American Electorate, 1972–2010. Los Angeles: CQ Press, Sage. http://dx.doi.org/10.4135/9781452234410
  • Biemer, P.P. 2010. Total Survey Error. Design, implementation and evaluation. Public Opinion Quarterly, 74, 817–848. http://dx.doi.org/10.1093/poq/nfq058
  • Blom, A.G., de Leeuw, E.D. and Hox, J.J. 2011. Interviewer effects on nonresponse in the European Social Survey. Journal of Official Statistics, 27, 359–377.
  • Cresie, N. and Read, T.R.C. 1984. Multinomial goodness-of-fit tests. Journal of the Royal Statistical Society B, 46, 440–464.
  • Childs, A. and Balakrishnan, N. 2000. Some approximations to the multivariate hypergeometric distribution with applications to hypothesis testing. Computational Statistics and Data Analysis, 35, 137–154. http://dx.doi.org/10.1016/S0167-9473(00)00007-4
  • Cleophas, T.J. and Zwinderman, A.H. 2011. Log likelihood ratio tests. In Statistical Analysis of Clinical Data on a Pocket Calculator (pp. 37–38). Springer. http://dx.doi.org/10.1007/978-94-007-1211-9_13
  • Díaz de Rada, V. 2013. La no respuesta en encuestas presenciales realizadas en Espa-a. Revista Internacional de Sociología, 71, 357–381. http://dx.doi.org/10.3989/ris.2012.02.07
  • Díaz de Rada, V. 2014. Analysis of incidents in face-to-face surveys: Improvements in fieldwork. Revista Espa-ola de Investigaciones Sociológicas, 145, 43–72.
  • Dopp, K. 2006. Vote Miscounts or Exit Poll Error? New Mathematical Function for Analyzing Exit Poll Discrepancy. http:// http://www.electionmathematics.org/em-exitpolls/Exit-Poll-Analysis.pdf. Last access 07/02/2014.
  • Edison Media Research and Mitofsky International 2005. Evaluation of Edison/Mitofsky election system 2004. http:// abcnews.go.com/images/Politics/EvaluationofEdisonMitofskyElectionSystem.pdf. Last access 07/02/2014.
  • El País. 2012. ¿Por qué fallaron las encuestas?, El País, 26 March. http://politica.elpais.com/politica/2012/03/26/ actualidad/1332790007_767368.html. Last access 06/03/2014.
  • Frankovic, K., Panagopoulos, C. and Shapiro, R. 2009. Opinion and election polls. In D. Pfeffermann and C.R. Rao, (eds.), Handbook of Statistics—Sample Surveys: Design Methods and Applications, Volume 29A (pp. 567– 596) Netherlands: Elsevier. http://dx.doi.org/10.1016/S0169-7161(08)00022-9
  • Groves, R.M., Dillman, D.A., Eltinge, J.L. and Little, R.J.A. 2002. Survey Nonresponse. New York: Wiley.
  • Haan, M., Ongena, P.Y. and Aarts, K. 2014. Reaching Hard-to- Survey Populations: Mode Choice and Mode Preference. Journal of Official Statistics, 30, 355–379. http://dx.doi.org/10.2478/jos-2014-0021
  • Haunberger, S. 2010. The effects of interviewer, respondent and area characteristics on cooperation in panel surveys: a multilevel approach. Quality & Quantity, 44, 957–969. http://dx.doi.org/10.1007/s11135-009-9248-5
  • Keeter, S. 2011. Public opinion polling and its problems. In K. Goidel (ed.), Political Polling in the Digital Age: The Challenge of Measuring and Understanding Public Opinion (pp. 28–53). Baton Rouge: Louisiana State University Press.
  • Klofstad, C.A. and Bishin, B.G. 2012. Exit and entrance polling: A comparison of election survey methods. Field Methods, 24, 429–437. http://dx.doi.org/10.1177/1525822X12449711
  • Krumpal, I. 2013. Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity, 47, 2025–2047. http://dx.doi.org/10.1007/s11135-011-9640-9
  • Martín Arroyuelos, A.M., Férnandez Aguirre, K. and Modro-o Herrán, J.A. 2014. Early forecasting of Parliamentary seats distribution: the representative polling stations method. In J. Bozeman, V. Girardin and C.H. Skiadas (eds.), New Perspectives on Stochastic Modeling and Data Analysis (pp. 123–135).
  • Menold, N. and Kemper, C.J. 2014. How do real and falsified data differ? Psychology of survey response as a source of falsification indicators in face-to-face surveys. International Journal of Public Opinion Research, 26, 41– 65. http://dx.doi.org/10.1093/ijpor/edt017
  • Merkle, D.M. and Edelman, M. 2002. Nonresponse in exit polls: a comprehensive analysis. In R.M. Groves, D.A. Dillman, J.L. Eltinge and R.J.A. Little (eds.), Survey Nonresponse (pp. 243–257). New York: Wiley. PMid:12186630 PMCid:PMC1222982
  • Mitofsky, W. 1991. A short history of exit polls. In P.J Lavrakas and J.K. Holley (eds.), Polling and Presidential Election Coverage (pp. 83–99). Newbury Park, CA: Sage.
  • Mokrzycki, M., Keeter, S. and Kennedy, C. 2009. Cell-phone-only voters in the 2008 exit poll and implications for future noncoverage bias. Public Opinion Quarterly, 73, 845–865. http://dx.doi.org/10.1093/poq/nfp081
  • Noelle-Neumann, E. 1984. The Spiral of Silence: Public Opinion – Our Social Skin. Chicago: The University of Chicago Press.
  • Orriols, L. and Martínez, A. 2014. The role of the political context in voting indecision, Electoral Studies, 35, 12–23. http://dx.doi.org/10.1016/j.electstud.2014.03.004
  • Panagopoulus, C. 2013. Who participates in exit polls? Journal of Elections, Public Opinion and Parties, 23, 444–455. http://dx.doi.org/10.1080/17457289.2013.811585
  • Pavía, J.M. 2010. Improving predictive accuracy of exit polls. International Journal of Forecasting, 26, 68–81. http://dx.doi.org/10.1016/j.ijforecast.2009.05.001
  • Pavía, J.M. and García-Cárceles, B. 2012. Una aproximación empírica al error de dise-o muestral en las encuestas electorales del CIS. Metodología de Encuestas, 14, 45–62.
  • Pavía, J.M. and Larraz, B. 2012. Nonresponse bias and superpopulation models in electoral polls. Revista Espa-ola de Investigaciones Sociológicas, 137, 237–264. http://dx.doi.org/10.5477/cis/reis.137.237
  • Pavía, J.M., Larraz, B. and Montero, J.M. 2008. Election forecasts using spatiotemporal models. Journal of the American Statistical Association, 103, 1050–1059. http://dx.doi.org/10.1198/016214507000001427
  • Pavía, J.M., Mu-iz, P. and Álvarez, J.A. 2001. Proyecciones en la noche electoral. Estadística Espa-ola, 43, 225–239.
  • Pavía, J.M. and López-Quilez, A. 2013. Spatial vote redistribution in redrawn polling units. Journal of the Royal Statistical Society A, 176, 655–678. http://dx.doi.org/10.1111/j.1467-985X.2012.01055.x
  • Pavía-Miralles, J.M. 2005. Forecasts from non-random samples: The election night case, Journal of the American Statistical Association, 100, 1113–1122. http://dx.doi.org/10.1198/016214504000001835
  • Pavía-Miralles, J.M. and Larraz-Iribas, B. 2008. Quick-counts from non-selected polling stations. Journal of Applied Statistics, 35, 383–405. http://dx.doi.org/10.1080/02664760701834881
  • Radcliff, B. 2005. Exit polls. In S.J. Best and B. Radcliff (eds.), Polling America: An Encyclopedia of Public Opinion, Volume I (A-O). Westport, CT: Greenwood Press.
  • Rechner, A.C. 2002. Methods of Multivariate Analysis. New York: John Wiley & Sons. PMCid:PMC1721411
  • Selb, P., Herrmann, M., Munzert, S. Schübel, T. and Shikano, S. 2013. Forecasting runoff elections using candidate evaluations from first round exit polls. International Journal of Forecasting, 29, 541-547. http://dx.doi.org/10.1016/j.ijforecast.2013.02.001
  • Slater, M. and Christensen, H. 2008. Applying AAPOR's final disposition codes and outcome rates to the 2000 Utah colleges' exit poll. In F.J. Scheuren and W. Alvey (eds.), Elections and Exit Polling (pp. 37–50). Hoboken, NJ: Wiley.
  • Sobrino, V.M. 2012. Los avances de resultados en una jornada electoral. Más Poder Local, 13, 42–44.
  • Tourangeau, R., Rips, L.J. and Rasinki, K. 2000. The Psychology of Survey Response. Cambridge, UK: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511819322
  • Turner, C.F., Ku, L., Rogers, S.M., Lindberg, P.D., Pleck, J.H. and Sonenstein, F.L. 1998. Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873. http://dx.doi.org/10.1126/science.280.5365.867 PMid:9572724
Data source: Dialnet