The Second Dimension of Educational Quality: A Comparative Study in Seven Countries in Latin America
Main Article Content
Abstract
INTRODUCTION. The educational quality assessments made it possible to compare education systems in the last 20 years. PISA test results are the most commonly used as reference. In Latin America, there are seven countries that have recurrently participated in this test since the year 2000: Argentina, Brazil, Chile, Colombia, Mexico, Peru, and Uruguay. METHODOLOGY. In this study, we analyze the evolution of the above-named countries in the PISA test, and other national and international assessments. We examine the goals, methodological biases, and the comparative results of these tests. RESULTS. The results of the educational quality assessments contradict each other and have variations based on methodological issues that are relatively unexplored. The PISA tests have introduced a methodological change in its 2015 edition that modify the results of previous editions and made the results incomparable to those of the first editions (2000 and 2003). This raises important questions regarding the fact that the educational systems that seem to have improved, have actually not, or at least not to such extent. The paper introduces a comparison of the results of different assessments in which those countries participated, showing some cases that have improved their educational quality like Peru—which had a lower starting point— and Chile— which has shown the highest improvement among the analyzed countries during the 2000s. DISCUSSION. The study concludes by revealing a necessity to discuss a “second dimension of the educational quality” that requires further analyses of the standardized assessment results —not just skim reading— taking into consideration the various educational indicators and sources in a complimentary manner. That second dimension acts as an epistemological filter to understand the evolution of educational systems in Latin America and to avoid a misreading or a biased interpretation of the improvements achieved in the quality of education.
Downloads
Article Details
References
Alexander, R. (2001). Culture and Pedagogy. International Comparisons in Primary Education. Londres: Blackwell.
Ball, S. J. (2003). The teacher's soul and the terrors of performativity. Journal of Education Policy, 18, 215-228. doi: 10.1080/0268093022000043065
Berliner, D. (2011). Rational responses to high stakes testing: the case of curriculum narrowing and the harm that follows. Cambridge Journal of Education, 41, 287-302. doi: 10.1080/0305764X.2011.607151
Bos, M. S., Elías, A., Vegas, E. y Zoido, P. (2016). PISA: América Latina y el Caribe. ¿Cuánto mejoró la región? [Nota 2, Banco Interamericano de Desarrollo, Centro de Información para la Mejora de los Aprendizajes]. Recuperado de https://publications.iadb.org/bitstream/handle/11319/7992/America-Latina-en-PISA-2015-Cuanto-mejoro-la-region.pdf?sequence=4
Carnoy, M. (2007). Cuba’s Academic Advantage. Stanford: Stanford University Press.
Carnoy, M. y Rothstein, R. (2013). International tests show achievement gaps in all countries, with big gains for U.S. disadvantaged students. Washington DC: Economic Policy Institute.
Carnoy, M., Khavenson, T., Fonseca, I., Costa, L. y Marotta, L. (2015). Is Brazilian education improving? Evidence from PISA and SAEB. Cadernos de Pesquisa, 45(157), 450-485. doi: 10.1590/198053143331
Chen, Y. F. y Jiao, H. (2014). Exploring the utility of background and cognitive variables in explaining latent differential item functioning: An example of the PISA 2009 Reading assessment. Educational Assessment, 19(2), 77-96. doi: 10.1080/10627197.2014.903650
Cosgrove, J. (2015). Changes in Achievement in PISA from 2000 to 2009 in Ireland: beyond the test scores. The Irish Journal of Education, xl, 29-44. doi: 10.1186/2196-0739-2-2
Cowen, R. (2011). “CODA”. En Pereyra, Kotthoff y Cowen (eds.), PISA Under Examination: Changing Knowledge, Changing Tests, and Changing Schools (pp. 259-264). Holanda: Sense Publishers.
Eivers, E. (2010). PISA: Issues in implementation and interpretation. Irish Journal of Education/Iris Eireannach an Oideachais, 38, 94-118. Recuperado de http://www.jstor.org/stable/20789130
Eklöf, H. (2010). Skill and will: test-taking motivation and assessment quality. Assessment in Education: Principles, Policy & Practice, 17(4), 345-356. doi: 10.1080/0969594X.2010.516569
Fernández Cano, A. (2016). A Methodological Critique of the PISA Evaluations. Relieve, 22(1). doi: 10.7203/relieve.22.1.8806
Ferrer, G. (2006). Sistemas de evaluación de aprendizajes en América Latina. Balances y desafíos. Santiago de Chile: Educar Chile.
Ferrer, G. y Fiszbein, A. (2015). “¿Qué ha sucedido con los sistemas de evaluación de aprendizajes en América Latina?”. Documento de Antecedentes. Comisión para la Educación de Calidad para Todos. El Diálogo Interamericano. Washington DC: World Bank Group.
Finn, B. (2015). Measuring motivation in low-stakes assessments. ETS Research Report Series, 1-17. doi: 10.1002/ets2.12067
Fischman, G. E., Marcetti Topper, A., Silova, I., Goebel, J. y Holloway, J. L. (2018). Examining the influence of international large-scale assessments on national education policies. Journal of Education Policy, 34(4), 470-499, doi: 10.1080/02680939.2018.1460493
Freitas, P., Catela Nunes, L., Balcão Reis, A., Seabra, C. y Ferro, A. (2016). Correcting for sample problems in PISA and the improvement in Portuguese students’ performance. Assessment in Education: Principles, Policy & Practice, 23(4), 456-472. doi: 10.1080/0969594X.2015.1105784
Fuller, B., Henne, M. K. y Hannum, E. (2008). Strong States, Weak Schools: The Benefits and Dilemmas of Centralized Accountability. Bingley, Reino Unido: Emerald Group Publishing Limited.
Gneezy, U., List, J. A., Livingston, J. A., Sadoff, S., Qin, X. y Xu, Y. (2017). Measuring Success in Education: The Role of Effort on the Test Itself. NBER Working Paper No. 24004. doi: 10.3386/w24004
Grek, S., Lawn, M. y Ozga, J. (2009). Fabrication, circulation and use of a supra-national instrument of regulation based on knowledge – Education Sector: Production of OECD’s Programme for International Student Assessment (PISA). Edimburgo: Centre for Educational Sociology, University of Edinburgh.
Hanushek, E. y Woessmann, L. (2012). Schooling, educational achievement, and the Latin American growth puzzle. Journal of Development Economics, K(2), 497-512. doi: 10.1016/j.jdeveco.2012.06.004.
Hopfenbeck, T. N., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J. y Baird, J. (2017). Lessons Learned from PISA: A Systematic Review of Peer-Reviewed Articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research, K(3), 333-353. doi: 10.1080 /00313831.2016.1258726
Huerta Cuervo, M. R. (2010). Iberoamérica en PISA 2006: Informe regional del Grupo Iberoamericano de PISA. Gestión y Política Pública, 19(2), 420-430. Recuperado de http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S1405-10792010000200008
Instituto Colombiano para la Evaluación de la Educación (2017). Informe Nacional de Resultados. Colombia en PISA 2015. Colombia: Ministerio de Educación de Colombia.
Instituto Nacional de Evaluación Educativa (2016). México en PISA 2015. México: INEE. Recuperado de http://publicaciones.inee.edu.mx/buscadorPub/P1/D/316/P1D316.pdf
Jodouin, H. (2014). Differential omission rates: Exploring factors that influence omission rates in a large-scale Pan-Canadian assessment (tesis doctoral). Department of Applied Psychology and Human Development, Ontario Institute for Studies in Education, University of Toronto.
Klein, R. (2011). Uma re-análise dos resultados do PISA: problemas de comparabilidade. Ensaio: Avaliação e Políticas Públicas em Educação, 19(73), 717-742, doi: 10.1590/S0104-40362011000500002
Kreiner, S. y Christensen, K. B. (2014). Analyses of model fit and robustness. A new look at the PISA scaling model underlying ranking of countries according to reading literacy. Psychometrika, 79(2), 210-231. doi: 10.1007/s11336-013-9347-z
Lewis, S. y Lingard, B. (2015). The Multiple Effects of International Large-Scale assessment on Education Policy and Research. Discourse, 36(5), 621-637. doi: 10.1080/01596306.2015.1039765.
Lindblad, S., Pettersson, D., y Popkewitz, T. S. (2015). International comparisons of school results: A systematic review of research on large-scale assessments in education. Estocolmo: Swedish Research Council.
LLECE-UNESCO (2014). “Primera entrega de resultados”. TERCE: Tercer estudio regional, comparativo y explicativo. Santiago de Chile: OREALC-UNESCO.
Lüdtke, O., Robitzsch, A., Trautwein, U., Kreuter, F. e Ihme Marten, J. (2007). Are there test administrator effects in large-scale educational assessments? Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 3(4), 149-159. doi: 10.1027/1614-2241.3.4.149
Meyer, H. D. y Benavot, A. (2013). PISA, power, and policy: The emergence of global educational governance. Wallingford, Reino Unido: Symposium Books.
Ministério da Educação do Brasil (2016). Brasil no PISA 2015: análises e reflexões sobre o desempenho dos estudantes brasileiros. São Paulo: Fundación Santillana.
Mislevy, R. J. (1995). Test theory and language-learning assessment. Language Testing, 12(3), 341-369. doi: 10.1177/026553229501 200305
Morris, A. (2011). Student Standardised Testing: Current Practices in OECD Countries and a Literature Review. París: OECD Education Working Papers, No. 65, OECD Publishing. Recuperado de http://dx.doi.org/10.1787/5kg3rp9qbnr6-en
Niemann, D., Martens, K. y Teltemann, J. (2017). PISA and its Consequences: Shaping Education Policies through International Comparisons. European Journal of Education, 52(2), 175-183. doi: 10.1111/ejed.12220
OECD (2015). PISA 2015: Technical Standards. París: OCDE Publishing.
OECD (2016a). PISA 2015 Results (Volume I): Excellence and Equity in Education. París: OCDE Publishing. Recuperado de http://dx.doi.org/10.1787/9789264266490-en
OECD (2016b). Programme for International Student Assessment. Results from PISA 2015. Country Note: Colombia. OCDE Publishing. Recuperado de https://www.oecd.org/pisa/PISA-2015-Colombia.pdf
OECD (2018). Competencias en Iberoamérica: análisis de PISA 2015. OCDE Publishing & Fundación Santillana –for Spanish edition. Recuperado de https://www.segib.org/wp-content/uploads/Competencias-en-Iberoamerica-Analisis-de-PISA-2015.pdf
Pintrich, P. R. y De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33-40. doi: 10.1037//0022–0663.82.1.33
Pons, X. (2017). Fifteen Years of Research on PISA Effects on Education Governance: A Critical Review. European Journal of Education, 52(2), 131-144. doi: 10.1111/ejed.12213
Rautalin, M., Alasuutari, P. y Vento, E. (2018). Globalisation of education policies: does PISA have an effect? Journal of Education Policy, 34(4), 500-522, doi: 10.1080/02680939.2018.1462890
Ravela, P. (2003). ¿Cómo aparecen los resultados de las evaluaciones educativas en la prensa? Washington DC: PREAL.
Rivas, A. (2015). América Latina después de PISA: Lecciones aprendidas de siete países. Buenos Aires: CIPPEC.
Solano-Flores, G. y Milbourn, T. (2016). Capacidad evaluativa, validez cultural y validez consecuencial en PISA. Relieve, 22(1), doi: 10.7203/relieve.22.1.8281
Swaffield, S. y Thomas, S. (2016). Educational Assessment in Latin America. Assessment in Education: Principles, Policy & Practice, 23(1), 1-7. doi: 10.1080/0969594X.2016.1119519
Taubman, P. M. (2009). Deconstructing the Discourse of Standards and Accountability in Education. Nueva York y Londres: Routledge, Taylor and Francis Group.
Wise, S. L. y DeMars, C. (2005). Low examinee effort in low-stakes assessment: Problems and possible solutions. Educational Assessment, 10(1), 1-17. doi: 10.1207/s15326977ea1001_1
Wolff, L. (2004). Educational Assessments in Latin America: the state of art. International Association of Applied Psychology, 53(2), 192-214, doi: 10.1111/j.1464-0597.2004.00168.x
Wolff, L. y Smith, J. K. (1995). The consequence of consequence: motivation, anxiety, and test performance. Applied Measurement in Education, 8(3), 227-242.
Wuttke, J. (2007). “Uncertainties and Bias in PISA. PISA according to PISA”. En Hopmann, Brinek y Retzl (eds.), Does PISA keep what it promises? (pp. 241-263). Viena: Hopmann. Recuperado de https://ssrn.com/abstract=1159042