https://doi.org/10.4438/1988-592X-RE-2023-400-569
Francesc Pedró Garcia
https://orcid.org/0000-0001-5335-8100
Instituto Internacional de la UNESCO para la Educación Superior en América Latina y el Caribe (UNESCO IESALC)
Abstract
International comparative studies in education are considered by their proponents as relevant sources of evidence for the improvement of public policies in education and are frequently referenced by policymakers. However, the increase in comparative evidence, especially thanks to the OECD's PISA program, does not seem to have translated into significant improvements in the quality of educational policies, as relatively few countries have improved their results. The tradition of public policy analysis sheds light on the main difficulties encountered by comparative studies to be used appropriately in the public policy formulation process. From this perspective, three of these barriers are analyzed: the perverse nature of the educational problems to which we are trying to respond, the communication barriers between researchers and producers of evidence and policymakers, as well as issues of the latter's capacity to handle the evidence; and, finally, the implementation gap. The analysis of these three barriers is completed with some final considerations on improving the dialogue between political science and public policymaking in education.
Keywords: comparative education, comparative studies, education policy, evidence, political science, public policies.
Resumen
Los estudios comparativos internacionales en educación son considerados por sus proponentes fuentes relevantes de evidencia para la mejora de las políticas públicas en educación y son, con frecuencia, referenciados por los decisores políticos. Sin embargo, el aumento de evidencias comparativas, singularmente gracias al Programa PISA de la OCDE, no parece haberse traducido en mejoras significativas en la calidad de las políticas educativas pues son relativamente pocos los países que han mejorado sus resultados. La tradición de análisis de políticas públicas permite arrojar luz sobre las dificultades principales con las que tropiezan los estudios comparativos para ser utilizados apropiadamente en el proceso de formulación de políticas públicas. Desde esta perspectiva, se analizan tres de ellas: la naturaleza perversa de los problemas educativos a los que se intenta dar respuesta; las barreras de comunicación entre los investigadores y productores de evidencias y los decisores políticos, además de problemas de capacidad de manejar las evidencias por parte de estos últimos; y, finalmente, la brecha de implementación. Su análisis se completa con unas consideraciones finales acerca de cómo mejorar el diálogo entre la ciencia política y las políticas públicas en educación.
Palabras clave: educación comparada, estudios comparativos, política educativa, evidencias, ciencia política, políticas públicas.
The culture of evidence seems to be absent in the professional decisions of teachers and school administrators who rarely use research findings in their decision-making about which strategies or programs to adopt (Dagenais et al., 2012; Morrison et al., 2014). Education probably constitutes the field most hit by the discourse of evidence, understood as a pressure that largely comes from external actors who resonate poorly with most existing lessons among education professionals (Krejsler, 2013). In countries like the United States or England, where local authorities and school boards are responsible for making decisions about school programs and interventions, recourse to evidence, particularly through experimental evaluations, could facilitate decision-making processes (Slavin, 2021). Following this wake, a few European countries have created centers or programs to collect and disseminate evidence on the effectiveness of different educational programs among teachers and school managers, particularly (Eurydice, 2017). This is not an easy task: some of these public initiatives have also been closed or have become sustained by private entities in England (Pellegrini & Vivanet, 2021); in Spain, the La Caixa Foundation, in collaboration with the Education Endowment Foundation has recently launched a similar initiative1.
In line with what is happening in other sectors of public activity, such as in particular agriculture, health care, or science and technology (Cheung & Xie, 2021), it is worth asking whether evidence can also be a resource in the process of formulating public policies aimed at solving educational problems. A typical case is the quality of learning and its equity or educational inclusion for a country or an autonomous jurisdiction, particularly through regulatory, consultation, or funding mechanisms. Of course, it can be argued that the nature of public policies is different from that of programs2, as is their formulation process. Still, political science has spent decades analyzing the opportunities and barriers for public policies in all sectors to be informed by evidence. For various reasons, the study of education has long been a neglected topic in political science. In recent times, however, scholarly interest in the field has rapidly increased (Busemeyer & Trampusch, 2011). To what extent international comparisons might play an equivalent role in informing policy to experimentation in decisions about interventions and programs (Mølstad & Pettersson, 2019). This would be to go beyond the well-analyzed process of education policy borrowing, with excellent cases such as those of school inspection in Israel and Turkey (Nir, Kondakci, & Emil, 2018), the Danish school reform of 2013 (Karseth, Sivesind, & Steiner-Khamsi, 2022) or the UK-inspired assessment in Hong Kong (Yan & Brown, 2021).
The main international comparative source of evidence on education, the OECD's PISA Program, has existed for more than two decades. Throughout these years, the Program has disseminated several policy recommendations based precisely on the analysis of the results, such as, for example, the relevance of school autonomy and the critical role of pedagogical leadership, not to mention the low effectiveness of investments in educational technology, or the relative value of low student-teacher ratios or higher teacher salaries (Schleicher, 2018). If there is evidence about what works and why, then how to explain that on an international scale, progress remains scarce and that so few countries whose results improve and, at the same time, so many others worsen, and that the majority remain stable (OECD, 2020)?
Political science research has shed light on why it is so difficult to promote comparative evidence in public policymaking processes (Cairney & Oliver, 2017a). But examples of application to the education sector are very scarce; not only have political scientists tended to neglect the sector in their analyses, but also education scholars have made little effort to look to political science for answers to these problems (Jakobi, Martens, & Wolf, 2009).
From this perspective of dialogue between political science and public policies in education, this contribution focuses on analyzing three fundamental reasons that could explain the paradox that, despite having more and more comparative evidence, educational policies fail to improve results at the national level or at the level of the competent autonomous jurisdiction with the capacity to formulate its policies. The first is that the nature of the educational problems we are trying to respond to is so complex that it requires public policy designs that are no less complex and so contextualized that it is impossible to find in comparative evidence more than a source of information. In other words, what is presented as evidence of educational policy would not be so because it would only offer partial, atomized, or incomplete information, fundamentally focused on programs and projects. The second is that there are communication barriers between researchers and producers of evidence and policymakers, as well as problems in the latter's ability to manage the evidence. The third is that even assuming that none of the above reasons is true, there is an implementation gap: policies would be correctly formulated based on the available evidence, but they would never be properly or fully implemented for various reasons, including conflicts of agenda between the different agents, in particular teachers and families, but also the different levels of government and public administrations.
Each of these three reasons is discussed below. Their analysis is completed with some final considerations on improving the dialogue between political science and public policy in education.
Although traditional sectors of public activity, such as school education, have well-defined intervention methodologies and dominant assumptions about their dynamics, many emerging policy issues do not generate the same agreement on methods or interactions between variables to be considered (Simpson, 2019). Even for educational problems that have been addressed for some time, such as, for example, the impact of family support on school outcomes, the internal complexity of such issues may have become more apparent, or the opening of policy areas to a broader set of policy actors may have led to alternative conceptualizations to the traditional ones. For example, governments have been formulating and implementing school policies for decades. Still, school policies have been subsumed into the more diffuse realm of education policy to incorporate other subsectors with which they interact and with different visions. Instead of being limited to the school sector, this sphere of action is now also concerned with issues related to early childhood education or university education, both being relatively recent areas of government intervention and limited by the proverbial autonomy of providers (Pedró Garcia, 2021).
One way of parameterizing the complexity of educational problems is to use the concept of wicked problems. This concept comes from the literature on systems theory and planning (Rittel & Webber, 1973). (Rittel & Webber, 1973) and was developed to describe the emergence of a set of problems, such as poverty, that challenge the ability of governments to formulate public policies effectively. These problems share some common traits that can be summarized in three main areas (Termeer, Dewulf, & Biesbroek, 2019). First, they are poorly defined and linked to other problems. In addition, the solutions for those problems are not easy to find and are connected to the same actors that cause the problems. And finally, it seems impossible to know, ex-ante, what would constitute a good solution. While each of the characteristics is important, the general argument underlying them is that an increasing number of problems facing governments and societies cannot be solved effectively through the traditional procedures governments have typically used to find solutions. In short, these are problems for whose resolution there is a lack of evidence because their very nature prevents the parameterization of solutions based on the available evidence (Turnbull & Hoppe, 2019).
It is important to distinguish between wicked and complicated problems (Peters & Tarpey, 2019). The latter category of the issues may have several moving parts, such as actors, but the relationships between them are linear and largely predictable. Indeed, all policy problems are complicated at a certain point, with multiple interests and usually multiple veto points on which decisions must be made. A clear example is the issue of equalizing teacher salaries with those of other public professionals with equivalent entry requirements: it is not simple to resolve, and there are several players, but, to some extent, the behaviors of the finance ministry, the teachers' unions, and the education ministry are predictable. On the other hand, Wicked problems involve several actors but have more uncertain and non-linear connections between the variables that make up the policy domain. The complexity is evident in a political issue as apparently simple as concertation or chartering, i.e., whether or not private schools in a country where they are deeply rooted should be able to receive public funds in exchange for what quid pro quo. However, the reaction of public schools and unions to such an initiative can evolve if, as is usually the case, the negotiation for concertation takes place in the middle of salary negotiations, as has been the case in several European countries such as Spain and France. In this sense, the discussion of wicked problems can be linked to that of the so-called “intractable political disputes” (Susskind & Field, 1996). (Susskind & Field, 1996). In this case, a problem is considered intractable much less because of its technical characteristics and the uncertain interactions of variables than because of the political preferences and associated policy frameworks of the actors involved. Easier problems, from the perspective of the wicked problems literature, may be intractable from a more political stance. Wicked problems can be described using six attributes. (Sternberg & Frensch, 2014):
Policy problems such as improving school learning outcomes can certainly be described as wicked problems, and, indeed, some analysts have developed the concept of “wicked super problems”3 to describe climate change and other extremely twisted contemporary policy problems (Levin, Cashore, Bernstein, & Auld, 2012). And still, other analysts have also emphasized problem complexity as a more generic way of describing policy problems that do not easily fit into the usual linear conceptions of public policymaking and governance (Duit & Galaz, 2008; Klijn, 2008).
Political problems in education are difficult to conceptualize and even more difficult to solve. To this extent, they fit perfectly within the definition of wicked problems offered by political science. Given this reality, comparative evidence can be useful in providing information on other countries' previous experiences. In doing so, they can help to break down the problem into its various components and to visualize alternative solutions, whether they are feasible within the regulatory framework itself. But it is hard to imagine that comparative evidence can do more than inform in the context of wicked problems. When comparative evidence, which explains differences generated in the past, is used to normativize the present and prescribe courses of action, it inevitably oversimplifies the definition of problems, neglects the particularities of each national context, or feigns a conceptual leap to promote certain educational policies. Such approaches have been criticized on countless occasions concerning the work of the OECD and the World Bank, in particular (Takala, Kallo, Kauko, & Rinne, 2018; Zapp, 2021) and even more specifically in the case of policy recommendations based on the results of PISA (Pedró, 2012; Sjøberg & Jenkins, 2022), including their even more striking application to the case of developing countries (Auld, Rappleye, & Morris, 2019).
Assuming the perverse nature of problems in educational policy and the intrinsic difficulty of making comparative evidence a solid base on which to base policy formulation, its informative relevance is beyond doubt. A better knowledge of comparative evidence on the part of politicians and managers would improve their decision-making capabilities without constraining their actions. In this regard, there is a notable tradition of policy studies on evidence-policy gaps, in which scholars describe their attempts to overcome the barriers between the production of evidence by researchers and its use by policymakers. The most frequently reported barriers (Owen, Watkins, & Hughes, 2022) relate to problems in effectively disseminating high-quality information, namely, lack of time, support, resources, and incentives for researchers to engage in dissemination. These studies suggest that evidence is often not presented at the right time and that researchers cannot quickly anticipate the demand for information to solve a very specific problem. In addition, policymakers lack the research skills needed to understand the evidence. More generally, one could say that researchers, on the one hand, and policymakers, on the other, have different scientific and political cultures even in sectors marked by science and technology, as is the case in the public health sector (Cairney & Oliver, 2017b).
The most frequently suggested solutions to these barriers highlight the limits of this theoretical analysis. For example, to address the supply-side problem, studies emphasize the need for improved dissemination to ensure that policymakers pay attention to and understand the best evidence (Oliver, Innvar, Lorenc, Woodman, & Thomas, 2014). But unfortunately, few studies recognize that policymakers will not share the sense that there is a hierarchy of evidence. Instead, too many assume that better dissemination will cause policymakers to think, like researchers, that evidence alone is persuasive or it is worth the redundancy, self-evident.
Most political theories explore the implications of two basic ideas: that policymakers are constrained by bounded rationality (Simon H., 1976) and that they share power with many actors in complex policymaking systems (Cairney, P., 2016). In part, bounded rationality is related to policymakers' inability to gather and consider all evidence relevant to policy problems. Instead, they employ two routes: rational, which pursues clear objectives and prioritizes specific sources of information, and irrational, which relies on emotions, hunches, beliefs, and habits to make decisions quickly.
The main problem with many education policy studies is that they focus on the first route. They identify the problem of uncertainty and incomplete information and try to solve it by creating hierarchies of evidence and improving the provision of comparative data to policymakers through policy recommendations that often do not indicate financial feasibility or political viability. They ignore the role of negotiation and persuasion in reducing ambiguity. We must begin by recognizing politicians' tendency to base their judgments on their well-established beliefs and routes based on their values, emotions, and familiarity with information. From there, we need to think about how to reduce ambiguity, persuade politicians to frame a problem primarily in a certain way, and thereby demand evidence that will help solve that problem (Dekker & Meeter, 2022).
In education, many models of research impact are based on strategies that make minimal reference to policy formulation, namely, identification of the research question, development of a research methodology, implementation of data collection, analysis, and synthesis, interpretation of results, and development of research recommendations and, subsequently, for both policy and practice. In this patrician model, the process is owned and controlled by the researchers, who then advise or disseminate their work to policymakers. Under this logic, the proposed solution to improve the use of comparative evidence is to develop scientific competence in governments. Many studies assume it is realistic to produce a captive audience of policymakers willing to invest the time necessary to prioritize and understand the available evidence. This approach is at odds with the less rigid ways in which many forms of evidence are used by policymakers (Cairney, P., 2016).
Second, very few studies acknowledge the role of values in policy. Instead, an often implicit and untested assumption is that policymaking should be as evidence-based as medicine, which is at odds with the most common starting point, in the study of politics, of producing a democratic system that translates conflicting social values and preferences into policy solutions. Of course, a political system based on value judgments and evidence may be desired. Still, the trade-offs between these goals must be recognized and addressed, and the production of evidence is also an inherently value- based process. Of the few existing analyses of this issue in the education sector, an investigation into how members of the UK Parliament used the available evidence concerning the policy decision that led to the Selective Schools Expansion Fund, a policy designed to allow the former 163 selective grammar schools to apply for additional funding to expand their pupil numbers, stands out. It became clear from the research that, ultimately, the values espoused by the majority were more determinative than the evidence made available to them by the OFSTED agency (Bainbridge, Troppe, & Bartley, 2022).
The third gap in the use of comparative evidence is the recognition of the gap between the evidence of what has worked and the reality of the context in which a policy inspired by comparative evidence is implemented. There is a growing awareness that policies do not succeed or fail on their own merits but that their progress also depends on the implementation process. Unfortunately, the normatively attractive top-down view of policy and its implementation is based on three questionable assumptions: a chronological order in which expressed intentions precede action; a linear causal logic according to which objectives determine instruments and instruments determine outcomes; and a hierarchy in which policy formulation is more important than implementation (Hupe, 2015). Yet, despite several decades of criticism, it is a model that still retains some popularity among policymakers and is probably the one on which comparative researchers in education rely.
The classic concept of the policy implementation gap (Gunn, L.A., 1978). (Gunn, L.A., 1978) has been complemented in recent years by complex systems thinking informed by unpredictability, nonlinearity, and adaptability (Rapport et al., 2018). Here, the factors shaping and influencing policy implementation are complex, multifaceted, and multilevel, with public policies invariably resembling wicked problems that are resistant to change, have multiple possible causes, and with potential solutions that vary across place and time depending on the local context (Rittel & Webber, 1973).
There is currently a great deal of interest in the notion of policy failure (Volcker, 2014), but, as McConnell has pointed out (2015, p. 231), failure lies at the end of a success-failure spectrum where it is characterized by outright non-compliance. Such a situation will be unusual. As he observes, “failure is rarely unequivocal and absolute… even policies that have been known as classic policy failures also produced small, modest successes”.
Four general factors can be identified that contribute to policy failure even when they claim to be supported by international comparative evidence: overly optimistic expectations; implementation in dispersed governance; bad collaborative policymaking; and the vagaries of the political cycle. Each of these is discussed below.
One might think that the most ambitious and costly policies - the large projects - would be the most carefully assessed for risk. However, “over-optimism” was the title given to an influential review of failure in large government projects in the UK by the National Audit Office (National Audit Office, 2013). This problem is not confined to the UK: a comparative study by the OECD (OECD, 2015a), for example, also notes that successful implementation is a constant challenge for government centers. This is the case when policies require a long-term approach. A study by the Institute for Government in the United Kingdom of four such policy areas -fighting poverty, climate change, international development, and homelessness- identified three common features that complicate implementation (Ilott, Randall, Bleasdale, & Norris, 2016).: costs and benefits are unevenly distributed over time - there is a large time lag between implementation and positive outcomes; they tend to be intellectually controversial, politically contentious, and difficult to execute; and causes and effects span different government agendas across several administrations or departments.
The French policy of class size reduction in priority areas is a good example of this unbridled optimism. Initiated in 2017, taking as its starting point a single comparative study on the impact of class size reduction, its low impact and high costs demonstrate the intrinsic difficulties of a simple policy aimed only at modifying one parameter of school provision (Pellegrini & Vivanet, 2021).
Policies formulated at the national level may face the challenge of ensuring some degree of consistency in their implementation at the subnational level. This process is especially complicated when the sub-national level has some degree of independent political authority, as is increasingly the case in education (Gamage & Zajda, 2009; Sausman et al. (2016), when it draws on the concept of local universality to describe the process by which general standards, products, or guidelines are shaped and adapted to fit local contexts and enacted in practices. However, it is less clear how central authorities can respond to this reality, especially when it occurs in ways hidden from policymakers' views.
Even when governance is concentrated rather than dispersed, implementation will still depend heavily on local context: the literature on complex systems has made it abundantly clear that an intervention that is successful in one place does not necessarily deliver the same results elsewhere (Braithwaite, Churruca, Long, Ellis, & Herkes, 2018) as has been pointed out many times in Comparative Education (Mølstad & Pettersson, 2019). All these links with the literature that for decades has been dealing with receptive and non-receptive contexts to change, pioneered by Pettigrew et al. (1992), and emphasizes the need for policymakers to confront the messy engagement of multiple actors with diverse sources of knowledge. (Davies, Nutley, & Walter, 2008).
Added to this is the complication that those acting at the top levels cannot succeed without knowing what is happening at or near the front line. Such is the premise of the bottom-up school of thought on policy implementation and echoes the notion of Lipsky (1980) of the “street-level bureaucrat” whose discretionary power may prove decisive in determining the success or failure of a policy. One of the salient features of many policies- especially those requiring direct contact with the public, such as education- is that middle-level personnel, particularly the school inspectorate, have considerable contact with outside agencies and often enjoy discretionary powers that give them de facto autonomy from their managers. Although many of the decisions of these agents may seem small individually, they can radically reshape strategic policy intent (Hudson, Hunter, & Peckham, 2019).
One of the biggest recent reversals of fortune in the education sector illustrates the importance of understanding external factors: the sustained improvement in the performance of disadvantaged pupils in London's public schools around 2005-2014. This remarkable success is a puzzle because the improvement was not predicted and resists explanation from commonly understood factors. For example, demographic changes cannot explain the improvement. Instead, it appears that more resources, a successful teacher recruitment campaign and new buildings have played a supportive, if not decisive, role and that new institutions focused on school management helped (Blanden, Greaves, Gregg, Macmillan, & Sibieta, 2015).
Policy development has tended to take place in separate administrative departments or ministries, even though most interventions will almost certainly have broader implications affecting external parties. Moreover, despite growing academic interest in developing ideas and tools to promote inter- organizational partnerships, improvements have been patchy at best and limited (Gazley, 2017). The weakness of collaborative policymaking and the failure to establish common ground for public problem-solving through constructive management of differences remains one of the key reasons for subsequent implementation difficulties.
Except for the simplest of tasks, policy design requires ongoing collaboration with a range of stakeholders at multiple political, policymaking, managerial, and administrative levels, as well as the involvement of local implementing agents, municipalities where they have competencies or decentralized entities, as well as end-users, students, and families, and, of course, front-line staff, school managers and teachers, and a range of local service agencies such as, for example, educational resource centers. Ansell et al. (2017) emphasize the need for policies to be designed to connect stakeholders vertically and horizontally in the process of collaboration and joint deliberation. They argue this should not be equated with a long and cumbersome search for unanimous consent; rather, it is a search for sufficient common ground on which to proceed, without which there will be ongoing conflicts over the legitimacy of the policies and the organization's mission. Therefore, policy design and implementation must become an integrated process rather than a series of discrete and distinct stages. Another question is whether policymakers are equipped with the necessary skills, competencies, capabilities, and capacities to address these systemic shortcomings and succeed in that endeavor (Williams, P., 2012).
The Pact provides an excellent example of this practice for Educational Excellence in French- speaking Belgium, an open process initiated in 2015 (Dachet & Baye, 2021). Due to the many different actors involved, it tends to take a compromise position between the purely evidence-based paradigm and the professional development considerations traditionally advocated by teachers and didactic specialists. By its definition, structure, and proposals, the Pact pays particular attention to the reform of both curricula and the structures of the educational system. We would also like to underline the will to bridge the gap between professionals and researchers by financing research carried out in schools in collaboration with teachers. Finally, the cooperation of all educational actors in the country, including families, is an innovative and valuable feature of the Pact's work. It has made it possible to (1) initiate a process of interaction between researchers, educational authorities, and practitioners; (2) identify promising educational programs in French-speaking Belgium that correspond both to researchers' recommendations and to standards and curricula; (3) create groups of experts in the field that can be used both in the evaluation of educational programs and their validation; and (4) have educationalists and educational science researchers jointly define minimum methodological standards for all categories of research.
Politicians tend not to be held accountable for the results of their policy initiatives: in the event of failure, they are likely to have moved on or gone away. One of the consequences is that the prospect of short-term results too easily attracts them. This can lead to pushing through policies as quickly as possible rather than engaging in the cumbersome, lengthy, and frustrating details of how things might work in practice. Evidence suggests that the political thrust needed to drive long-term policy development tends to dissipate over time (Norris, E., P. Bouchal, J. Rutter, & M. Kidson, 2014). The education sector is a classic example, as credited by the OECD. (2015b) by highlighting the virtual absence of public evaluations of education policies and reforms, with barely 10% of initiatives having undergone rigorous evaluation. The concern here is that policymakers are more likely to get credit for evidenced legislation than for implementation problems that have been avoided. Indeed, the latter is likely to be seen as the problem of others rather than themselves (Weaver, K., 2010). Therefore, it is not surprising that politicians at the head of ministries focus their efforts on new laws, on the one hand, and on material investments that have a very important symbolic value for voters (such as the delivery of digital devices or the opening of new schools).
As Cowen has rightly noted (2019), the emphasis on evidence from experiments allows policymakers to target interventions teachers must implement rather than policies for which they are responsible. The focus on empirical evidence favors teacher-level interventions rather than structural changes to the education system since the latter's effects are almost impossible to measure through experiments. For example, letting teachers teach mathematics with certain didactics can be evaluated experimentally, but not a structural reform of the educational system. This bias also has a positive side. Structural overhauls of the educational system carry great costs (both financial and mental) and dangers; this should be an argument for being more conservative when it comes to structural reorganizations than with classroom interventions. In addition, Cowen (2019) similarly points out that it could be solved by drawing on the full range of research techniques available when studying the potential benefits of structural changes in educational systems. This is, again, consistent with the maxim of always using the best available evidence.
The benefit of having an international evidence base is undeniable if the information is not confused with prescription. The example of the Pact for Educational Excellence in French-speaking Belgium is, to date, one of the few cases where the boundaries between the two are clear. And it is also an excellent example of a social dialogue on educational reform that makes all the existing international comparative evidence available to all stakeholders for each of the policy agenda elements. But unfortunately, it remains a unique example of the difficulties associated with the use of comparative evidence in the process of shaping education policy.
The brief analysis presented above of the three fundamental reasons why international comparative studies are not used as a solid basis for policy formulation is also implicitly a warning about the impossibility of their ever being used as a solid basis for policy formulation. Beyond the perverse nature of educational problems, or the difficulties of communication and policy implementation, comparative studies can only be considered one more source of information to support the formulation process. The risks of a prescriptive, technocratic approach are very clear: it aspires, either out of naivety or bad faith, to overlook the values, perspectives, and lived experiences of stakeholders and citizens directly or indirectly involved in these policies. Increasing evidence, even if it is of an international comparative nature, cannot alone solve wicked policy problems that, like educational ones, must be seen as based on competing viewpoints and value frameworks. Addressing these problems requires deliberation and debate about the nature of the issues and exploring alternative ways forward. This deliberative process of seeking solutions, with its recognition of the perspectives and values that frame the definition of the problems, is very different from the imposition of prescribed solutions on the grounds of international authority or experience-based answers that emerge from the growth of empirical knowledge.
Finally, the analysis carried out has been intended to be, at the same time, an example of how a closer approach of educational policy researchers to the theoretical and conceptual richness of public policy studies can be extremely fruitful and enriching so that policy theory serves, in the classic expression of Carney (2015) to have an impact on public policy. In the same way, it will also be useful for political science to access the background of a sector as dynamic and complex, not perverse, as that of education.
Ansell, C., Sørensen, E., & Torfing, J. (2017). Improving policy implementation through collaborative policymaking. Policy and Politics, 45(3), 467–486. https://doi.org/10.1332/030557317X14972799760260
Auld, E., Rappleye, J., & Morris, P. (2019). PISA for Development: How the OECD and World Bank shaped education governance post-2015. Comparative Education, 55(2), 197–219. https://doi.org/10.1080/03050068.2018.1538635
Bainbridge, A., Troppe, T., & Bartley, J. (2022). Responding to research evidence in Parliament: A case study on selective education policy. Review of Education, 10(1). https://doi.org/10.1002/rev3.3335
Blanden, J., Greaves, E., Gregg, P., Macmillan, L., & Sibieta, L. (2015). Understanding the improved performance of disadvantaged pupils in London. London: London School of Economics.
Braithwaite, J., Churruca, K., Long, J. C., Ellis, L. A., & Herkes, J. (2018). When complexity science meets implementation science: A theoretical and empirical analysis of systems change. BMC Medicine, 16(1), 63. https://doi.org/10.1186/s12916-018-1057-z
Busemeyer, M. R., & Trampusch, C. (2011). Review Article: Comparative Political Science and the Study of Education. British Journal of Political Science, 41(2), 413–443. https://doi.org/10.1017/S0007123410000517
Cairney, P. (2015). How can policy theory have an impact on policymaking? Teaching Public Administration, 33(1), 22–39.
Cairney, P. (2016). The politics of evidence-based policymaking. London: Palgrave Pivot.
Cairney, P., & Oliver, K. (2017a). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Research Policy and Systems, 15(1), 35, s12961-017-0192-x. https://doi.org/10.1186/s12961-017-0192-x
Cairney, P., & Oliver, K. (2017b). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Research Policy and Systems, 15(1), 35, s12961-017-0192-x. https://doi.org/10.1186/s12961-017-0192-x
Cheung, A. C. K., & Xie, C. (2021). Evidence-Based Reform in Education: Global Perspectives. ECNU Review of Education, 4(1), 3–6. https://doi.org/10.1177/2096531120984793
Cowen, N. (2019). For whom does “what works” work? The political economy of evidence-based education. Educational Research and Evaluation, 25(1-2), 81–98. https://doi.org/10.1080/13803611.2019.1617991
Dachet, D., & Baye, A. (2021). Evidence-Based Education: The (Not So Simple) Case of French- Speaking Belgium. ECNU Review of Education, 4(1), 164–189. https://doi.org/10.1177/2096531120928086
Dagenais, C., Lysenko, L., Abrami, P., Bernard, R., Ramdé, J., & Janosz, M. (2012). Use of research- based information by school practitioners and determinants of use: A review of empirical research. Evidence & Policy A Journal of Research Debate and Practice, 8, 285–309.
Davies, H., Nutley, S., & Walter, I. (2008). Why 'knowledge transfer' is misconceived for applied social research. Journal of Health Services Research & Policy, 13(3), 188–190. https://doi.org/10.1258/jhsrp.2008.008055
Dekker, I., & Meeter, M. (2022). Evidence-based education: Objections and future directions. Frontiers in Education, 7, 941410. https://doi.org/10.3389/feduc.2022.941410
Duit, A., & Galaz, V. (2008). Governance and ComplexityEmerging Issues for Governance Theory. Governance, 21(3), 311–335. https://doi.org/10.1111/j.1468-0491.2008.00402.x
Eurydice. (2017). Support mechanisms for evidence-based policymaking in education. Eurydice report. Brussels: Publications Office of the European Union.
Gamage, D. T., & Zajda, J. (2009). Decentralisation and School-Based Governance: A Comparative Study of Self-Governing School Models. En J. Zajda & D. T. Gamage (Eds.), Decentralisation, School-Based Management, and Quality (pp. 3–22). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-90-481-2703-0_1
Gazley, B. (2017). The Current State of Interorganizational Collaboration: Lessons for Human Service Research and Management. Human Service Organizations: Management, Leadership & Governance, 41(1), 1–5. https://doi.org/10.1080/23303131.2015.1095582
Gunn, L. A. (1978). Why is implementation so difficult? Management Services in Government, 33(4), 169–176.
Hudson, B., Hunter, D., & Peckham, S. (2019). Policy failure and the policy-implementation gap: Can policy support programs help? Policy Design and Practice, 2(1), 1–14. https://doi.org/10.1080/25741292.2018.1540378
Hupe, P. (Ed.). (2015). Understanding street-level bureaucracy. Bristol: Policy Press.
Ilott, O., Randall, J., Bleasdale, A., & Norris, E. (2016). Making policy stick. Tackling long-term challenges in government (p. 79). London: Institute for Government.
Jakobi, A. P., Martens, K., & Wolf, K. D. (2009). Education in Political Science: Discovering a Neglected Field. Routledge.
Karseth, B., Sivesind, K., & Steiner-Khamsi, G. (Eds.). (2022). Evidence and Expertise in Nordic Education Policy: A Comparative Network Analysis. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-91959-7
Kingdon, John W., E. Stano. (1984). Agendas, alternatives, and public policies. Boston: Little, Brown.
Klijn, E.-H. (2008). Complexity Theory and Public Administration: What's New? Public Management Review, 10(3), 299–317. https://doi.org/10.1080/14719030802002675
Krejsler, J. B. (2013). What Works in Education and Social Welfare? A Mapping of the Evidence Discourse and Reflections upon Consequences for Professionals. Scandinavian Journal of Educational Research, 57(1), 16–32. https://doi.org/10.1080/00313831.2011.621141
Levin, K., Cashore, B., Bernstein, S., & Auld, G. (2012). Overcoming the tragedy of super wicked problems: Constraining our future selves to ameliorate global climate change. Policy Sciences, 45(2), 123–152. https://doi.org/10.1007/s11077-012-9151-0
Lipsky, M. (1980). Street Level Bureaucracy: Dilemmas of the Individual in Public Services. Russell Sage Foundation. Recuperado de https://www.jstor.org/stable/10.7758/9781610447713
McConnell, A. (2015). What is policy failure? A primer to help navigate the maze. Public Policy and Administration, 30(3-4), 221–242. https://doi.org/10.1177/0952076714565416
Mølstad, C. E., & Pettersson, D. (Eds.). (2019). New Practices of Comparison, Quantification and Expertise in Education: Conducting Empirically Based Research (1.a ed.). Abingdon, Oxon; New York, NY: Routledge, 2019.: Routledge. https://doi.org/10.4324/9780429464904
Morrison, J. R., Ross, S. M., & Corcoran, R. P. (2014). Fostering Market Efficiency in K-12 Ed-tech Procurement (p. 170). Baltimore: Johns Hopkins University.
National Audit Office. (2013). Over-Optimism in Government Projects. London: National Audit Office.
Nir, A., Kondakci, Y., & Emil, S. (2018). Travelling policies and contextual considerations: On threshold criteria. Compare: A Journal of Comparative and International Education, 48(1), 21–38. https://doi.org/10.1080/03057925.2017.1281102
Norris, E., P. Bouchal, J. Rutter, & M. Kidson. (2014). Doing Them Justice: Lessons from Four Cases of Policy Implementation. London: Institute for Government.
OECD. (2015a). Delivering from the Centre: Strengthening the Role of the Centre of Government in Driving Priority Strategies. Paris: OECD Publishing.
OECD. (2015b). Education Policy Outllok. Making Reform Happen. Paris: OECD Publishing. OECD. (2020). PISA 2018 Results What Students Know and Can Do. Paris: OECD Publishing. Recuperado de https://www.oecd.org/pisa/publications/pisa-2018-results-volume-i-5f07c754-en.htm
Oliver, K., Innvar, S., Lorenc, T., Woodman, J., & Thomas, J. (2014). A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Services Research, 14(1), 2. https://doi.org/10.1186/1472-6963-14-2
Owen, K. L., Watkins, R. C., & Hughes, J. C. (2022). From evidence-informed to evidence-based: An evidence building framework for education. Review of Education, 10(1). https://doi.org/10.1002/rev3.3342
Pedró, F. (2012). Deconstruyendo los puentes de PISA: Del análisis de resultados a la prescripción política. Revista Española de Educación Comparada, (19), 139–174. https://doi.org/10.5944/reec.19.2012.7581
Pedró, F. (2021). ¿Quién le pone el cascabel al gato?: Un análisis comparativo de las agencias de garantía de la calidad de la educación superior. Revista Española de Educación Comparada, (37), 129–152. Recuperado de https://dialnet.unirioja.es/servlet/articulo?codigo=7712088 Pellegrini, M., & Vivanet, G. (2021). Evidence-Based Policies in Education: Initiatives and Challenges in Europe. ECNU Review of Education, 4(1), 25–45. https://doi.org/10.1177/2096531120924670
Peters, B. G., & Tarpey, M. (2019). Are wicked problems really so wicked? Perceptions of policy problems. Policy and Society, 38(2), 218–236. https://doi.org/10.1080/14494035.2019.1626595
Pettigrew, A., Ferlie, E., & McKee, L. (1992). Shaping strategic change The case of the NHS in the 1980s. Public Money & Management, 12(3), 27–31. https://doi.org/10.1080/09540969209387719
Rapport, F., Clay-Williams, R., Churruca, K., Shih, P., Hogden, A., & Braithwaite, J. (2018). The struggle of translating science into action: Foundational concepts of implementation science. Journal of Evaluation in Clinical Practice, 24(1), 117–126. https://doi.org/10.1111/jep.12741
Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169. https://doi.org/10.1007/BF01405730
Sausman, C., Oborn, E., & Barrett, M. (2016). Policy translation through localisation: Implementing national policy in the UK. Policy & Politics, 44(4), 563–589. https://doi.org/10.1332/030557315X14298807527143
Schleicher, A. (2018). World Class. How to build a 21st-century school system. Paris: OECD Publishing.
Simon H. (1976). Administrative Behavior (3.a ed.). London: Macmillan.
Simpson, A. (2019). The evidential basis of “evidence-based education”: An introduction to the special issue. Educational Research and Evaluation, 25(1-2), 1–6. https://doi.org/10.1080/13803611.2019.1617979
Sjøberg, S., & Jenkins, E. (2022). PISA: A political project and a research agenda. Studies in Science Education, 58(1), 1–14. https://doi.org/10.1080/03057267.2020.1824473
Slavin, R. E., Cheung, A. C. K., & Zhuang (庄腾腾), T. (2021). How Could Evidence-Based Reform Advance Education? ECNU Review of Education, 4(1), 7–24. https://doi.org/10.1177/2096531120976060
Sternberg, R. J., & Frensch, P. A. (2014). Complex Problem Solving: Principles and Mechanisms. Psychology Press.
Susskind, L., & Field, P. (1996). Dealing with an Angry Public: The Mutual Gains Approach To Resolving Disputes. Simon and Schuster.
Takala, T., Kallo, J., Kauko, J., & Rinne, R. (2018). One Size for All? Policy Advice of the World Bank and the OECD on Quality Assurance and Evaluation of School Education in Russia, Brazil, and China1. En A. W. Wiseman & P. M. Davidson (Eds.), Cross-nationally Comparative, Evidence-based Educational Policymaking and Reform (pp. 301–319). Emerald Publishing Limited. https://doi.org/10.1108/S1479-367920180000035009
Termeer, C. J. A. M., Dewulf, A., & Biesbroek, R. (2019). A critical assessment of the wicked problem concept: Relevance and usefulness for policy science and practice. Policy and Society, 38(2), 167–179. https://doi.org/10.1080/14494035.2019.1617971
Turnbull, N., & Hoppe, R. (2019). Problematizing 'wickedness': A critique of the wicked problems concept, from philosophy to practice. Policy and Society, 38(2), 315–337. https://doi.org/10.1080/14494035.2018.1488796
Volcker, P. A. (2014). Vision without Execution Is Hallucination. Public Administration Review, 74(4), 439–441. Recuperado de https://www.jstor.org/stable/24029423
Weaver, K. (2010). But Will It Work? Issues in Government Studies, (32), 1–17.
Williams, P. (2012). Collaboration in Public Policy and Practice. Bristol: Policy Press.
Yan, Z., & Brown, G. T. L. (2021). Assessment for learning in the Hong Kong assessment reform: A case of policy borrowing. Studies in Educational Evaluation, 68, 100985. https://doi.org/10.1016/j.stueduc.2021.100985
Zapp, M. (2021). The authority of science and the legitimacy of international organisations: OECD, UNESCO and World Bank in global education governance. Compare: A Journal of Comparative and International Education, 51(7), 1022–1041. https://doi.org/10.1080/03057925.2019.1702503
Contact address: Francesc Pedró Garcia. Instituto Internacional de la UNESCO para la Educación Superior en América Latina y el Caribe (UNESCO IESALC). Edificio Asovincar. Av. Los Chorros con Calle Acueducto, Altos de Sebucán. Caracas, 1071, Venezuela, e-mail: f.pedro@unesco.org
_______________________________
1 https://educaixa.org/es/repositorio-evidencias-educativas
2 A public policy is a set of objectives, decisions, and actions of a government to solve problems that both citizens and the government itself consider a priority at a given time. These actions and decisions involve multiple actors, sectors, or levels of government. Public policy is materialized in programs and projects, for the execution of which resources are allocated. (Kingdon, John W., 1984).
3 These problems have the basic characteristics of wicked problems but have additional ones that make them even more problematic for the public sector (Levin, Cashore, Bernstein and Auld, 2012). Perhaps the most important of these problems is that the capacity to solve them is being exhausted. Specifically, these problems are characterized by the existence of a tipping point that, once reached, will have meant a fundamental change in the nature of the issue and there may be no capacity for a solution as, for example, climate change.