¿Influye el diseño de las preguntas en las respuestas de los entrevistados?
Resumen
En este trabajo se analiza hasta qué punto el orden de administración de las opciones de respuesta afecta a la falta de respuesta y a la elección de respuestas “no definidas”. Un segundo objetivo es considerar si la estrategia de “descomponer” una pregunta en varias disminuye el número preguntas sin respuesta, así como de respuestas “no definidas”. Un experimento con encuesta telefónica muestra que presentar las preguntas del cuestionario comenzando por las opciones desfavorables en preguntas descompuestas aumenta las respuestas ‘no sabe’, además de ampliar el tiempo de respuesta. Por el contrario, utilizar preguntas con opciones de respuesta que comienzan por las desfavorables reduce el rechazo a responder la pregunta. Las respuestas indefinidas se reducen cuando se emplea una pregunta que comienza por preguntas favorables.
Citas
Aldrich, J. H., Niemi, R. G., Rabinowitz, G., & Rohde, D. W. (1982). The Measurement of Public Opinion about Public Policy: A Report on Some New Issue Question Formats. American Journal of Political Science, 26(2), 391-414. https://doi.org/10.2307/2111047
Alwin, D. F. (1992). Information Transmission in the Survey Interview: Number of Response Categories and the Reliability of Attitude Measurement. Sociological Methodology, 22, 83-118. https://doi.org/10.2307/270993
Armstrong, J. S., Denniston, W. B., & Gordon, M. M. (1975). The use of the decomposition principle in making judgments. Organizational Behavior & Human Performance, 14(2), 257–263. https://doi.org/10.1016/0030-5073(75)90028-8
Azofra, M. J. (1999). Cuestionarios. Madrid: Centro de Investigaciones Sociológicas.
Baka, A., Figgou, L., & Triga, V. (2012). ‘Neither Agree, nor Disagree’: A Critical Analysis of the Middle Answer Category in Voting Advice Applications. International Journal of Electronic Governance, 5(3/4), 244. https://doi.org/10.1504/ijeg.2012.051306
Barthel, M., et al. (2021). Measuring news consumption in a digital area. Pew Research Center. Recuperado el 11 de abril de 2021 de https://www.journalism.org/2020/12/08/news-consumption-methodology/
Berchtold, A. (2019). Treatment and reporting of item-level missing data in social science research. International Journal of Social Research Methodology, 22(5). 431-439. https://doi.org/10.1080/13645579.2018.1563978
Bishop, G. F. (1987). Experiments with the middle response alternative in survey questions. Public Opinion Quarterly, 51(2), 220-232. https://doi.org/10.1086/269030
Bishop, G. F. (1990). Issue involvement and response effects in public opinion surveys. Public Opinion Quarterly, 54(2), 209-218. https://doi.org/10.1086/269198
Bishop, G., & Smith, A. (2001). Response-Order Effects and the Early Gallup Split-Ballots. Public Opinion Quarterly, 65(4), 479-505. https://doi.org/10.1086/323575
Brace, I. (2018). Questionnaire design. How to plan, structure, and write survey material for effective market research. Londres: Kogan Page.
Bradburn, N. M. (1983). Response Effects. En P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.) Handbook of Survey Research (pp. 289-328). Nueva York: Academic Press.
Bradburn, N. (1992). What have we learned?. En N Schwarz & S. Sudman (Eds), Context Effects in Social and Psychological Research (pp. 315-325). Nueva York: Springer.
Cabrera-Álvarez, P., y Escobar, M. (2019). El efecto de la ponderación y la imputación en el sesgo de los estudios electorales en España. Revista Española de Investigaciones Sociológicas, (165), 45-64. https://dialnet.unirioja.es/descarga/articulo/6752512.pdf
Cantril, H. (1944). Gauging Public Opinion. Princeton: Princeton University Press.
Carlson, J. E., Mason, R., Saitiel, J., et al. (1995). Assimilation and contrast effects in general/specific questions. Rural sociology, 60(4), 666-673. https://doi.org/10.1111/j.1549-0831.1995.tb00599.x
Chang, L., & Krosnick, J. A. (2010). Comparing oral interviewing with self-administered computerized questionnaires: an experiment. Public Opinion Quarterly, 74(1), 154-167. https://doi.org/10.1093/poq/nfp090
Clifford, S., Kim, Y., & Sullivan, B. W. (2019). An improved question format for measuring conspiracy beliefs. Public Opinion Quarterly, 83(4), 690-722. https://doi.org/10.1093/poq/nfz049
Couper, M. P. (2008). Designing effective web surveys. Nueva York: Cambridge University Press. https://doi.org/10.1017/CBO9780511499371
De Leeuw, E., Hox, J. J., Scherpenzeel, A. C. (2010). Mode Effect or Question Wording? Measurement Error in Mixed Mode Surveys. Presented at the Annual Meeting of the American Association for Public Opinion Research, Chicago, IL, USA.
Debell, M., Wilson, C., Jackman, S., & Figueroa, L. (2021). Optimal Response Formats for Online Surveys: Branch, Grid, or Single Item?. Journal of Survey Statistics and Methodology, 9(1), 1-24. https://doi.org/10.1093/jssam/smz039
Díaz de Rada, V. (2000). Utilización de nuevas tecnologías para el proceso de “recogida de datos” en la investigación social mediante encuesta. Reis: Revista Española de Investigaciones Sociológicas, (91), 137-166. https://doi.org/10.2307/40184278
Díaz de Rada, V., y Ayerdi, P. (2007). Algunos problemas de la encuesta telefónica para la proyección electoral. Revista Española de Investigaciones Sociológicas, (118), 153-204.
Díaz de Rada, V., y Portilla, I. (2015). Encuestas telefónicas: estrategias para mejorar la colaboración. Perspectiva Empresarial, 2(1), 97-115. https://doi.org/10.16967/rpe.47
Díaz de Rada, V., Domínguez, J. A., y Pasadas, S. (2019). Internet como modo de administración de encuestas. Madrid: CIS.
Dillman, D. A. (1978). Mail and telephone surveys. Nueva York: Wiley.
Dillman, D. A. (2020). “Three Decades of Advancing Survey Methodology”. En T. W. Smith (Eds.), A Meeting Place and More…: A History of the American Association for Public Opinion Research (pp. 95-117). Washington, DC: AAPOR. https://www.aapor.org/About-Us/History/A-Meeting-Place-and-More.aspx
Dillman, D. A., et al (1995). Effects of Category Order on Answers in Mail and Telephone Surveys. Rural sociology, 60(4), 674-687. https://doi.org/10.1111/j.1549-0831.1995.tb00600.x
Dillman, D. A., Smyth, J., & Christian, L. M. (2014). Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method (Cuarta ed.). Nueva York: Willey.
Escobar, M, y Jaime, A. M. (2013). Métodos de Imputación Múltiple para Predecir Resultados Electorales. En A. Mendoza Velazquez (Ed.), Aplicaciones en Economía y Ciencias Sociales con Stata. Texas: Stata Press.
Frey, J. (1989). Survey Research by Telephone. Londres: Sage.
Fuchs, M. (2005). Children and Adolescents as Respondents. Experiments on Question Order, Response Order, Scale Effects and the Effect of Numeric Values Associated with Response Options. Journal of Official Statistics, 21(4), 701-725.
Garbarski, D., Schaeffer, N. C., & Dykema, J. (2016). The Effect of Response Option Order on Self-Rated Health: A Replication Study. Quality of Life Research, 25(8), 2117-2121. https://doi.org/10.1007/s11136-016-1249-y
Gilbert, E. E. (2015). A comparison of branched versus unbranched rating scales for the measurement of attitudes in surveys. Public Opinion Quarterly, 79(2), 443-470. https://doi.org/10.1093/poq/nfu090
Groves, R. M., & Kahn, R. L. (1979). Surveys by telephone: a national comparison with personal interviews. Orlando, FL: Academic Press.
Gutiérrez, C., Sotomayor, R., y Garrido, F. (2004). Manual de encuestadores. Córdoba: Unidad de Estudios Aplicados del IESA–CSIC.
Hamilton, L. (2006). Statistics with STATA. Toronto: Thomson.
Häder, S., Häder, M., & Kuhne, M. (2012). Introduction: telephone surveys in Europe. En S. Häder, M. Häder & M. Kuhne (Eds.), Telephone Surveys in Europe: Research and Practice (pp. 5-30) Manheim: Springer Verlag.
Holbrook, A. L., Krosnick, J. A., Carson, R. T., et al. (2000). Violating Conversational Conventions Disrupts Cognitive Processing of Attitude Questions. Journal of Experimental Social Psychology, 36(5), 465-494. https://doi.org/10.1006/jesp.1999.1411
Holbrook, A. L., Krosnick, J. A., Moore, D., & Tourangeau, R. (2007). Response order effects in dichotomous categorical questions presented orally: the impact of question and respondent attributes. Public Opinion Quarterly, 71, 325-348. https://doi.org/10.1093/poq/nfm024
Insights Analytics España (29 de junio de 2021). Jornada Insights + Analythics: La prospectiva electoral: metodología y fiabilidad. Madrid. Recuperado el 26 de agosto de 2021 de https://ia-espana.org/evento-jornada-insights-analytics/
Kalton, G., Roberts, J., & Holt, D. (1980). The Effects of Offering a Middle Response Option with Opinion Questions. Journal of the Royal Statistical Society. Series D (The Statistician), 29(1), 65-78. https://doi.org/10.2307/2987495
Krosnick, J. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213-236. https://doi.org/10.1002/acp.2350050305
Krosnick, J. A. (1992). The Impact of Cognitive Sophistication and Attitude Importance on Response Order and Question Order Effects. En N. Schwarz & S. Sudman (Eds.), Context Effects in Social and Psychological Research (pp. 203-218). Nueva York: Springer.
Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51(2), 201-219. https://doi.org/10.1086/269029
Krosnick, J. A., & Berent, K. M. (1993). Comparison of party identification and policy preferences: the impact of survey question format. American Journal of Political Science, 37(3), 941-964. https://doi.org/10.2307/2111580
Krosnick, J. A., & Fabrigar, L. R. (1997). Designing Rating Scales for Effective Measurement in Surveys. En. L. Lyberg, P. Biemer, M. Collins, E. De Leeuw, C. Dippo, N. Schwarz, & D. Trewin (Eds.), Survey Measurement and Process Quality (pp. 141–64). Hoboken, Nueva York: John Wiley & Sons, Inc.
Krosnick, J. A., & Presser, S. (2010). Question and Questionnaire Design. En P. V. Marsden & J. D. Wright, Handbook of Survey Research (pp. 263-314). Emerald Group Publishing Limited.
Körmendi, E., & Noordhoek, H. (1989). Data quality and telephone interviews. A comparative study of face-to-face and telephone data collecting methods. Copenhagen: Danmarck Statistick.
Malhotra, N., Krosnick, J. A., & Thomas, R. K. (2009). Optimal design of branching questions to measure bipolar construct. Public Opinion Quarterly, 73(4), 304-324. https://doi.org/10.1093/poq/nfp023
Martin, E., et al. (2007). Guidelines for Designing Questionnaires for Administration in Different Modes. Washington: U.S.: Census Bureau, DC 20233.
Menold, N., & Bogner, K. (2016). Design of Rating Scales in Questionnaires. Mannheim, Germany: GESIS – Leibniz Institute for the Social Sciences. https://doi.org/10.15465/GESIS-SG_EN_015
Miller, P. V. (1984). Alternate Question Forms for Attitude Scale Questions in Telephone Interviews. Public Opinion Quarterly, 48(4), 766-778. https://doi.org/10.1086/268882
Moncus, J.J. (2021). What different survey modes and question types can tell us about Americans’ views of China. Pew Research Center, https://medium.com/pew-researchcenter-decoded/what-different-survey-modes-and-question-types-can-tell-usabout-americans-views-of-china-4523a47b5d99, acceso 14 de abril de 2021
Montagni, I., Cariou, T., Tzourio, C., & González-Caballero, J. (2019). ‘I don’t know’, ‘I’m not sure’, ‘I don’t want to answer?’: a latent class analysis explaining the informative value of nonresponse options in an online survey on youth health. International Journal of Social Research Methodology, 22(6), 651-667. https://doi.org/10.1080/13645579.2019.1632026
Nicolaas, G., Thomson, K., & Lynn, P. (2000). The feasibility of conducing Electoral Surveys in the UK by telephone. Londres: National Centre for Social Research.
Olson, K., & Smyth, J. D. (2015). The Effect of CATI Questions, Respondents, and Interviewers on Response Time. Journal of Survey Statistics and Methodology, 3(3), 361-396. https://doi.org/10.1093/jssam/smv006
Olson, K., Smyth, J. D., & Kirchner, A. (2020). The Effect of Question Characteristics on Question Reading Behaviors in Telephone Surveys. Journal of Survey Statistics and Methodology, 8(4), 636-666. https://doi.org/10.1093/jssam/smz031
O’Muircheartaigh, C., Krosnick, J. A, Helic, A. (mayo de 1999). Middle alternatives, acquiescence, and the quality of questionnaire data. Paper presented at the American Association for Public Opinion Research Annual Meeting, St. Petersburg, FL.
Payne, S. L (1951). The art of asking questions. Princeton: Princeton University Press.
Peytchev, A., Couper, M. P., McCabe, S. E., & Crawford, S. D. (2006). Web survey design: Paging versus scrolling. International Journal of Public Opinion Quarterly, 70(4), 596-607. https://doi.org/10.1093/poq/nfl028
Raghunatham, T., Berglund, P. A., & Solenberg, P. W. (2018). Multiple imputation in practice. Londres: Taylor.
Roberts, C., Gilbert, E., Allum, N., & Eisner, L. (2019). Research Synthesis: Satisficing in Surveys: A Systematic Review of the Literature. Public Opinion Quarterly, 83(3), 598-626. https://doi.org/10.1093/poq/nfz035
Schaeffer, N. C., & Presser, S. (2003). The science of asking questions. Annual Review of Sociology, 29(1), 65-88. https://doi.org/10.1146/annurev.soc.29.110702.110112
Schuman, H., & Presser, S. (1996). Questions and answers in attitudes surveys. Londres: Sage.
Schwarz, N., & Bless, H. (1992). Assimilation and contrast effects in attitude measurement: an inclusion/exclusion model. Public Advanced in Consumer Research, 19, 72-77. https://www.acrwebsite.org/volumes/7271/volumes/v19/
Schwarz, N., & Sudman, S. (Eds.). (1992). Context Effects in Social and Psychological Research. Nueva York: Springer.
Schwarz, N., Hippler, H.-J., & Noelle-Neumann, E. (1992). A Cognitive Model of Response Order Effects in Survey Measurement. En N. Schwarz & S. Sudman (Eds.), Context Effects in Social and Psychological Research (pp. 187-202). Nueva York: Springer.
Smyth, J. D.; Israel, G. D., Newberry III, M. G., & Hull, R. G. (2019). Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings. Field Methods, 31(3), 260-276. https://doi.org/10.1177/1525822X19860648
Sudman, S., & Bradburn, N. (1982). Answering questions. San Francisco: Jossey Baas.
Sykes, W., & Goinville, G. (1985). Telephone interviewing on a survey of social attitudes. A comparison with face-to-face procedures. Londres: SCPR.
Treier, S., & Hillygus, D. S. (2009). The Nature of Political Ideology in the Contemporary Electorate. Public Opinion Quarterly, 73(4), 679-703. https://doi.org/10.1093/poq/nfp067
Tourangeau, R., Couper, M. P., & Conrad, F. G. (2013). "Up Means Good": The Effect of Screen Position on Evaluative Ratings in Web Surveys. Public opinion quarterly, 77(Suppl 1), 69-88. https://doi.org/10.1093/poq/nfs063
Yu, J. H., Albaum, G., & Swenson, M. (2003). Is a Central Tendency Error Inherent in the Use of Semantic Differential Scales in Different Cultures?. International Journal of Market Research, 45(2), 213-228. https://doi.org/10.1177/2F147078530304500203
Truebner, M. (2021). The Dynamics of Neither Agree Nor Disagree Answers in Attitudinal Questions. Journal of Survey Statistics and Methodology, 9(1), 51-72. https://doi.org/10.1093/jssam/smz029
Wetzelhütter, D. (2020). Scale-Sensitive Response Behavior!? Consequences of Offering versus Omitting a ‘Don’t Know’ Option and/or a Middle Category. Survey Practice, 13(1). https://doi.org/10.29115/SP-2020-0012
Derechos de autor 2021 Vidal Díaz de Rada
Esta obra está bajo licencia internacional Creative Commons Reconocimiento-NoComercial 4.0.
Todas las publicaciones de la Revista Española de Sociología se realizarán bajo una licencia abierta Creative Commons de Reconocimiento 4.0 Internacional (CC BY 4.0). Dicha licencia establece que los autores son los poseedores de los derechos de propiedad intelectual de sus trabajos, que pueden redistribuirse a cambio de un reconocimiento adecuado. Para más información de la licencia Creative Commons, consultar aquí.
Una vez aceptado un artículo para su publicación, la Revista Española de Sociología solicitará al denominado "autor para la correspondencia" la aceptación de una licencia obligatoria Creative Commons incluida en un acuerdo o contrato de publicación.
