Identifying the influence of the order of presentation of items in closed questions in surveys with parallel testing, survey experiment and split-half
DOI:
https://doi.org/10.33448/rsd-v14i4.48625Keywords:
Order effect, Primacy and recency effect, Parallel test, Survey experimentation, Split half.Abstract
This article aims to describe how the order of presentation of items in nominal scales influences decision making at the time of choice. By applying a survey and collecting data through a questionnaire self-completed by 800 students in public schools, a database was created. To identify the order and primacy and recency effect was applied a specific questionnaire with inverse sequential allocation of items in the same sample group. To identify the order and primacy and recency effect was applied a specific questionnaire in an parallel test with inverse sequential allocation of items in the same sample group. In order to measure decision-making under different circumstances, a questionnaire with random ordering of items was applied to an experimental group. To measure the influence of order on dichotomous scales, a presentation in standard mode and another in reverse mode was applied to an experimental group using the split half method. The results show that there is a tendency to choose the first items presented or the primacy effect. However, the choice of items is a combination of several influences, such as question and item content, psycholinguistic aspects of the items, specific framing, interval of the survey application in relation to parallel test, type of scale such as dichotomous or polyatomic, etc., which makes it difficult to identify clearly the influence of order or primacy or recency effect.
Downloads
References
Anderson, N. H. (1965). Primacy effects in personality impression formation using a generalized order effect paradigm. Journal of Personality and Social Psychology, 2(1), 1-9. https://doi.org/10.1037/h0021966
Araujo, E. A. C. de, Andrade, D. F. de, & Bortolotti, S. L. V. (2009). Teoria da resposta ao item. Revista da Escola de Enfermagem da USP, 43, 1000-1008. https://doi.org/10.1590/S0080-62342009000500003
Bizer, G. Y., Larsen, J. T., & Petty, R. E. (2011). Exploring the valence‐framing effect: negative framing enhances attitude strength. Political Psychology, 32(1), 59-80. https://doi.org/10.1111/j.1467-9221.2010.00795.x
Buchanan, E. M., Foreman, R E., Johnson, B. N., Pavlacic, J. M., Swadley, R. L., & Schulenberg, S. E. (2018). Does the delivery matter? Examining randomization at the item level. Behaviormetrika, 45(2), 1-22. https://doi.org/10.1007/s41237-018-0055-y
Crano, W. D. (1977). Primacy versus recency in retention of information and opinion change. The Journal of Social Psychology, 101(1), 87-96. https://doi.org/10.1080/00224545.1977.9923987.
Eberlein, A. (2011). Lehr-Lern-Labor Videoanalyse zweidimensionaler Bewegungen. (Dissertação de Mestrado). Würzburg: Universidade de Würzburg.
Eisinga, R. N., Grotenhuis, M. Te, & Pelzer, B. J. (2013). The reliability of a two-item scale: Pearson, Cronbach or Spearman-Brown? International Journal of Public Health, 58(4), 637-642. https://doi.org/10.1007/s00038-012-0416-3
D. R. Zhang, Z. H. Li, X. C. Chen, Z. X. Wang, X. C. Zhang, X. M. Meng, S. He, & X. P. Hu (2003). Functional comparison of primacy, middle and recency retrieval in human auditory short-term memory: an event-related fMRI study. Cognitive Brain Research, 16(1), 91-98. https://doi.org/10.1016/S0926-6410(02)00223-9
Dieckmann, A., & Voss, T. (2004). Die Theorie rationalen Handelns. Stand und Perspektiven. In A. Dieckmann & T. Voss (eds.). Rational-choice-Theorie in den Sozialwissenschaften (pp. 13-29). München: Oldenburg. ISBN 348656644X
Gaines, B. J., Kuklinski, J. H., & Quirk, P. J. (2007). The logic of the survey experiment reexamined. Political Analysis, 15(1), 1-20. https://doi:10.1093/pan/mpl008
Galera, C., & Fuhs, C. C. L. (2003). Memória visuo-espacial a curto prazo: os efeitos da supressão articulatória e de uma tarefa aritmética. Psicologia: Reflexão e Crítica, 16(2), 337-348. https://doi.org/10.1590/S0102-79722003000200014
Gruber, T. (2011). Transiente Gedächtnissysteme (sensorisches Gedächtnis und Kurzzeitgedächtnis). In T. Gruber (ed.) Gedächtnis (pp.15-38). Heidelberg: Springer. https://doi.org/10.1007/978-3-662-56362-5_2.
Haubensak, G. (1985). Der Begriff des absoluten Urteils. In G. Haubensak (ed.). Absolutes und vergleichendes Urteil (pp.1-14). Heidelberg: Springer. https://doi.org/10.1007/978-3-642-70319-5_1
Henkel, K., & Almeida, J. (2021). A imaginabilidade visual de itens nas perguntas de pesquisas de opinião em comunicação política. Comunicação & Sociedade, 43(3), 111-135. https://doi.10.15603/2175-7755/cs.v43n1p1-7
Henkel, K., & Almeida, J. (2022). Valores e cultura política de jovens eleitores. Belém: UFPA.
Henkel, K., & de Almeida, J. (2023). Editoração de dados errôneos em surveys. Revista Gestão & Tecnologia, 23(2), 81-104.
Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109-133. https://doi.org/10.1007/BF02291393
Kahneman, D., & Tversky, A. (1986). Rational choice and the framing of decisions. The Journal of Business, 59(4), 251-278. https://www.jstor.org/stable/2352759
Kogler, Sandra (2010). Automatische Itemgenierung bei Aufgaben zum visuellen Gedächtnis. (Dissertação de Bacharelado). Viena: Universidade Viena.
Krebs, D., & Hoffmeyer-Zlotnik, J. H. P. (2010). Positive first or negative first? Effects of the order of answering categories on response behavior. Methodology, 6(3), 118-127. https://doi.org/10.1027/1614-2241/a000013
Levin, I. P., Schneider, S. L., & Gaeth, G. J. (1998). All frames are not created equal: a typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76(2), 149-188. https://doi.org/10.1006/obhd.1998.2804
Loiacono, E., & Wilson, V. (2020). Do we truly sacrifice truth for simplicity: comparing complete individual randomization and semi-randomized approaches to survey administration. AIS Transactions on Human-Computer Interactio, 12(2). 45-69. https://doi.org/10.17705/1thci.00128
Lord, F. M. (1964). Nominally and rigorously parallel test forms. Psychometrika, 29(4), 353-345. https://doi.org/10.1007/BF02289600
Mack, B. (1970). Die Beziehung zwischen der Beantwortungszeit von Fragebogen-Items und der Neigung der Beantworter sozial unerwünschte Antworten zu bejahen bzw. zu unterdrücken. (Tese de Doutorado). Hamburgo: Universidade Hamburgo.
Matos, C. A., & Trez, G. (2012). A influência da ordem das questões nos resultados de pesquisas surveys. Revista de Administração FACES Journal, 11(1), 151-172. https://doi.org/10.21714/1984-6975FACES2012V11N1ART1136
Murdock, B. B. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64(5), 482-488. https://doi.org/10.1037/h0045106
Meyer, P., & Müller-Spitzer, C. (2013). Überlegungen zur Visualisierung von Wortbildung in elektronischen Wörterbüchern. In A. Klosa (ed.). Wortbildung im elektronischen Wörterbuch (pp. 255-279). Tübingen: Narr. ISBN 978-3-8233-6737-6
Petersen, T. (2002). Das Feldexperiment in der Umfrageforschung. Frankfurt: Campus. ISBN 9783593370149
Pfister, H.-R., Jungermann, H., & Fischer, K. (2017). Zielkonflikte. In H.-R. Pfister, H. Jungmann & K. Fischer (eds.) Die Psychologie der Entscheidung – eine Einführung (pp. 83-114). Berlin: Springer. https://doi.org/10.1007/978-3-662-53038-2
Pina, M. (2012). Falsas memórias e formação de impressões: efeito de primazia. (Dissertação de Mestrado). Lisboa: Universidade de Lisboa. http://hdl.handle.net/10451/6924
Pronk, T., Molenaar, D., Wiers, R. W., & Murre, J. (2022). Methods to split cognitive task data for estimating split-half reliability: a comprehensive review and systematic assessment. Psychonomic Bulletin & Review, 29, 44-54. https://doi.org/10.3758/s13423-021-01948-3
Rajaratnam, N., Cronbach, L. J., & Gleser, G. C. (1965). Generalizability of stratified-parallel tests. Psychometrika, 30(1), 39-56. https://doi.org/10.1007/BF02289746
Siemeling, C. (2020). Grammatikalität und Akzeptabilität: Methodische Überlegungen zur Erhebung von Sprecherurteilen am Beispiel des europäischen Portugiesisch. In C. Ossenkop & G. Veldre-Gerner (eds.). Fehler – Abweichung – Variation.
Sniderman, P. M., & Grob, D. B. (1996). Innovations in experimental design in attitude surveys. Annual Review of Sociology, 22, 377-399. https://doi.org/10.1146/annurev.soc.22.1.377
Sprachnormen aus linguistischer und didaktischer Sicht (pp. 107-129). Berlin: Editora Peter Lang. https://doi.org/10.3726/b17238
Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. New York: Academic Press. https://doi.org/10.2307/1957697
Schwarz, N., Oyserman, D., & Peytcheva, E. (2010). Cognition, communication, and culture: implications for the survey response process. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, L. Lyberg, P. P. Mohler, B.-E. Pennell, T. W. Smith (eds.). Survey methods in multinational, multiregional, and multicultural contexto (pp. 175-190). New Jersey: John Wiley & Sons. https://doi.org/10.1002/9780470609927.ch10
Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9(1), 59-71. https://doi.org/10.1177/1745691613514
Veld, W. M.v.d., & Saris, W. E. (2005). The nature of measurement error in panel data. Estimating opinion stability in panel surveys. In Anais…Conference of the European Survey Research Association, 1. p. 1-12. Barcelona, 2005.
Warrens, M. J. (2015). On Cronbach’s alpha as the mean of all split-half reliabilities. In R. E. Millsap, D. M. Bolt, L. A. v. d. Ark, & W.-C. Wang (eds.). Quantitative Psychology Research (pp. 293-300). Heidelberg: Springer. https://doi.org/10.1007/978-3-319-07503-7
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Karl Henkel; Jimnah de Almeida
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
