Evaluation of the educational quality in cycle 0-3:
state of the question

Evaluación de la calidad educativa en el ciclo 0-3:
estado de la cuestión

DOI: 10.4438/1988-592X-RE-2021-394-506

Andrea Otero-Mayer

Consuelo Vélaz-de-Medrano

Eva Expósito-Casas

Universidad Nacional de Educación a Distancia (UNED)

Abstract

Introduction: the debate towards universalization of Early Childhood Education (ECE) from cycle 0-3 requires ensuring not only an accessible and affordable school place, but the quality of it. This work addresses a situation diagnosis of quality assessment in ECE, to help make visible what has been done and what to do in Spain. Method: documentary type design, which includes a type regulatory review compared to descriptors and categories of analysis, as well as a systematic review (SR) of literature (meta-synthesis) in depth. Databases are queried in the SR using keywords and categories. Results: the regulations, when it exists, are very heterogeneous in typology and application of the evaluation. It does not refer to a system of quality indicators. The LOMLOE (2020) lays new foundations for moving forward. Only one pilot study (2005) and 5 studies (most recently 2012) have been identified, which evaluates globally and with validated instruments the quality of children’s education in Spain, and none apply in cycle 0-3. Discussion: Mathiesen and Dominguez (2006) show that, while the relationship between coverage and quality in cycle 3-6 is supported by sufficient evidence, it appears that this is not true in cycle 0-3, which is consistent with the results obtained in this study and places us away from scientific activity in other countries. Conclusions: work is urgently needed on a system of process quality indicators and results in the two cycles of this stage, and of valid and reliable instruments to achieve comparable results, as is done in other countries. The need for greater territorial cooperation is evident to ensure that the right to ECE is not only sufficient and accessible, but of quality for all.

Key words: Early childhood education, Educational quality, Educational evaluation, Educational indicators, Evaluation methods, Measuring instruments.

Resumen

Introducción: el debate hacia la universalización la Educación Infantil (EI) desde el ciclo 0-3 exige garantizar, no solo una plaza escolar accesible y asequible, sino la calidad de la misma. Este trabajo aborda un diagnóstico de situación de la evaluación de la calidad en EI, para contribuir a visibilizar lo realizado y lo por hacer en España. Método: diseño de tipo documental, que incluye una revisión normativa de tipo comparado con descriptores y categorías de análisis, así como una revisión sistemática (RS) de la literatura (meta-síntesis) en profundidad. En la RS se consultan bases de datos utilizando palabras clave y categorías. Resultados: la normativa, cuando existe, es muy heterogénea en tipología y aplicación de la evaluación. No refiere a un sistema de indicadores de calidad. La LOMLOE (2020) pone nuevas bases para avanzar. Solo se han identificado un estudio piloto (2005) y 5 estudios (el más reciente de 2012), que evalúan de manera global y con instrumentos validados la calidad de la educación infantil en España, y ninguno se aplica en el ciclo 0-3. Discusión: Mathiesen y Domínguez (2006) muestran que, si bien la relación entre la cobertura y la calidad en la etapa 3-6 está avalada por suficiente evidencia, parece que esto no se cumple en la etapa 0-3, lo que concuerda con los resultados obtenidos en el presente estudio y nos sitúa lejos de la actividad científica en otros países. Conclusiones: es urgente trabajar en un sistema de indicadores de la calidad de procesos y resultados en los dos ciclos de esta etapa, y de unos instrumentos válidos y fiables que permitan obtener resultados comparables, como se hace en otros países. Se evidencia la necesidad de mayor cooperación territorial para garantizar que el derecho a la EI no solo sea suficiente y accesible, sino de calidad para todos.

Palabras clave: educación de la primera infancia, calidad de la educación, evaluación de la educación, indicadores educativos, método de evaluación, instrumentos de medida.

Introduction

The debate about universal (free of charge) access to Early Childhood Education (henceforth, ECE) has increased as its impact on child development and women joining the job market has been demonstrated. Various national, international and supranational bodies and institutions have expressed their concerns about inequity in access and quality, and have highlighted this concern as one of their main courses of action. As happenned with the universalization of compulsory education, the first step was to guarantee access, and later ensuring its quality, but at this time both must be simultaneous. This would fulfil the European Social Rights Pillar: “Children have the right to enjoy affordable and good quality education and care at early childhood” (Unión Europea, 2017, p.19), also the Convention on the Rights of the Child (ONU, 1989) and lastly with the Objective 4.2. of the United Nations Sustainable Development Programme (ONU, 2015); by 2030, all children will have access to quality development, care and pre-primary education. Certainly, investing in a quality first stage is more effective and efficient than tryung to compensate for inequalities in later stages of education (Heckman 2017; OCDE, 2017; Save the Children, 2019).

The importance of early education in child development is supported by strong evidence: 87% of the brain is formed during the first three years of a child’s life (UNICEF, 2014); neural connections are affected by both health and nutritional aspects, as well as the type of interaction between the child and people and objects within their environment (Herrerea, Mathiensen and Domínguez, 2006). In this period, basic social skills and some key behaviours are also developed. Likewise, the European Council (Consejo Europeo, 2018) states that in the 0-3 cycle public policies are needed as tools to prevent and combat poverty and social exclusion, break the intergenerational cycle of disadvantage and promote social mobility. This brief lapse in time influences the rest of our lives.

According to data from the Ministry of Education and Vocational Training (MEFP, 2019a), enrolment in pre-school education in Spain has doubled in the last 10 years, going largely from having a care function to fulfilling an educational one, although it remains essential for family work/life balance (Llorent, 2013).

State, regional and local administrations are working to guarantee the access to the first cycle of education, but there is still plenty of room for improvement. While the net enrolment rate in the second cycle is close to 100 per cent, in the first cycle it is much lower; in children under one it is 12.3 per cent, 40.2 per cent at children´s first year, and 60 per cent for two year olds (MEFP, 2020b). There are also remarkable differences among Autonomous Communities from 18.3% in Murcia to 53.8% in the Basque Country. If school enrolment rate in 2 years is considered, it reaches 92.9% in the Basque Country, 71.5% in Madrid and below 30% in the Canary Islands (MEFP, 2020a). The Government of Spain made a commitment in the LOMLOE (2020) to promote a sufficient and accessible supply of the first cycle offering within a period of eight years, and significant progress is also seen at regional level.

In regards to ownership, Spain has 9952 primary schools, of which 52.5% are private and in 73% of the public schools the ownership belongs to the local government (MEFP, 2020a). In the unequal disribution between Autonomous Commnuties, it is worth mentioning the case of the Canary Islands, where almost 80% of the schools are private or, as a complete opposite, Navarra, where only 20% of the schools are private (MEFP, 2019b).

The MEFP is responsible for the basic regulations on the curriculum and minimum requirements of schools, and its development corresponds to local education administrations. Only Galicia has the particularity of sharing these responsibilities between the Ministry of Education and the Ministry of Social Policy.

In general, in public or government funded private centres the second cycle is free of charge -with different formulas-, unlike the first cycle, so the new Education Law (LOMLOE, 2020) takes a step forward and incorporates the “24th Additional Provision”. The General State Budget corresponding to the temporary application of this Law will progressively incorporate the credits to make the second cycle of early years education, referred to in Article 15.2, free of charge”. Furthermore, the Third Additional Provision, which states that “within one year from this Act’s approval, the Government, in collaboration with the Education Administrations, will draw up an eight-year plan for the extension of the first cycle of early years education so as to move towards an adequate and affordable public supply with equity and quality, and to ensure its educational value. Its gradual introduction will tend to extend its free access, prioritising those students at risk of poverty and social exclusion and low school enrolment rate”.

Delving into the subject of this article, theoretical and empirical evidence shows that quality ECE has a positive impact on the development of children under 6 years old, while low quality education may have no effect or even be harmful (Clarke-Steward, 1987; Felfe, Nollenberger and Rodríguez Planas, 2012; Heckman and Mossos, 2014; Larrea, López de Arana, Barandiaran and Vitoria, 2010; Lera, 2007; Rolla and Rivadeniera, 2006; Melhuish et al., 2015; NICHD ECCRN, 2006; Phillips, Scars and McCartney, 1987). Some longitudinal studies (Dickinson, 2006; Pungello et al., 2010; McLean, Sparapani, Toste and McDonald-Connor, 2016; Heckman, 2017) show that high-quality educational interventions during the first years can have long-lasting effects: in subsequent school success, preventing school failure and lowering dropout rates.

The concept of quality in education is not unique. “The very notion of educational quality is an example of a multifaceted and polymorphic concept which, taking an analytical perspective, manifests itself in a plurality of components or quality factors: however, none of them is sufficient to comprehend the concept” (Borrell, Morrás and López, 2005, p.518). While this is true, there are widely endorsed conceptualizations.

Ruopp and Travers´ study (1982) marked the beginning of the quality of early years education research and is widely used today (Valverde-Forttes, 2015). The authors divide the quality dimensions into two groups of variables: the structural ones, whose regulation is financially expensive but simpler-i.e. classroom size, teacher/student ratio, training, teachers’ experience and salary-and those related to the educational process -i.e. the quality of the environment and relationships in the classroom, methodology or curriculum. Specifically, the assessment of the environment (processes) is very widespread at the institutional level in countries such as USA (Zellman and Perlman, 2008) or Scotland (Bradshaw, P. Hinchliffe & Scholes, 2020) and serves as a predictor of quality in childhood education (Sylva et al., 2006).

As for the European Commission, they have adopted a Council Recommendation (Consejo Europeo, 2018) on supporting Member States’ efforts to improve access and quality at this level of education. It includes a “quality framework” that lists five key components:

■ Access to early years education and care

■ Training and working conditions for personnel responsable for early years education and care

■ Appropriate curriculum and governance definition

■ Financing

■ Systems monitoring and evaluation

It is certain that only through monitoring and evaluation we will be able to analyse their strengths and weaknesses and suport the improvement process. However, it is necessary to foresee the inherent complexity of the evaluation at the early years education stage, due to the reasons indicated by the Evaluation Institute (INE, 2008).

Lack of previous reference studies that allow us to start from a solid and proven theoretical and practical basis.

– Difficulty in applying assessment tools to large samples of children at this stage.

– Complexity of its application, in which it is necessary to have expert and experienced professionals in this educational stage or, where appropriate, to train the teachers by making them participants in the procedures and evaluation system.

This will undoubtedly influence the available evidence on the evaluation of the quality of ECE in Spain.

Since Spain is a highly decentralised country, it is also necessary to review the current regulation in the autonomous communities and cities.

Based on the above, the following research questions are posed:

■ What are the regulations regarding the evaluation of the quality of early years education in Spain?

■ What theoretical and factual evidence is there regarding the evaluation of the quality of early years education, and what methodologies and tools are being used?

General objective:

To analyse the current state of the question on the evaluation of ECE quality, with special reference to the firste cycle (0-3) in Spain.

Specific objectives shall be:

1. To conduct a comparative review and summary of the regulations on the evaluation of the quality of education at ECE level, with special reference to the first cycle, published by the MEFP and the Autonomous comunities and cities

2. To conduct a review and a comparative summary of the most used methodologies to assess the quality of ECE at national and international levels, with special reference to the tools used in the first cycle.

Methodology

The study is carried out using a methodological design in documentary style (Payne and Payne, 2004), which includes a normative review of comparative type, as well as an in depth systematic review of the literature (meta-synthesis).

Regulation review

In order to perform the comparative study (Raventós, 1991; Velloso & Pedró, 1991) between January and March 2020, a review of the regulations referring to the Autonomous Communities and Cities which govern the evaluation of the first cycle of this stage is carried out. BOE (the Spanish official state bulletin), the official bulletins of the different communities, the websites of the Education Councils and the official reports available that collect regulations have been consulted. The review is limited to the regulations in force in January 2020. The descriptors used are: “evaluation of early childhood education”, “supervision of early childhood education”, “quality in early childhood education”, “Educational inspection”, “external/internal evaluation” and “teaching practice”, and the terms used in the regulations are analysed. Subsequently, the four phases of the comparative method were carried out (Bereday, 1968; Hilker, 1964); descriptive, interpretive, juxtapositionally and comparative.

In the Autonomous Communities and Cities in which the evaluation is regulated, three categories were analysed: implementation (internal/external), who is responsible for carrying it out, and its mandatory nature. The information and data obtained were then compared, and the similarities and differences found in the territories were analysed. The documentation used in this comparative study was the regulations published by the autonomous administrations and by the MEFP, for Ceuta and Melilla, in January 2020 (Table I).

TABLE 1. Comparative review of the quality of ECE in the Autonomous Communities and Cities.

Autonomous Communities/Cities	Revised reference regulation
Andalucía	Decree 149/2009, of 12 May
Aragón	Order of 28 March 28, 2008
Asturias	Decree 113/2014, of 3 December Decree 27/2015, of 15 April
Baleares	Decree 60/2008, of 2 May
Canarias	Decree 201/2008, of 30 September Order of 5 February,2009
Cantabria	Decree 143/2007, of 31 October Decree 144/2007, of 31 October
Castilla y León	Decree 12/2008, of 14 February Order EDU/958/2007, of 25 May
Castilla-La Mancha	Decree 88/2009, of 7 July Law 7/2010, of 20 July
Cataluña	Decree 101/2010, of 3 August Decree 282/2006, of 4 July
Valencian Comunity	Order of 22 March, 2005 Decree 253/2019, of 29 November
Extremadura	Decree 4/2008, of 11 January
Galicia	Decree 330/2009, of 4 June
Madrid	Order 680/2009, of 19 January
Murcia	Resolution of 16 June, 2010
Navarra	Foral Decree 80/2008, 30 June Foral Order 63/2013, of 5 July
Basque Country	Decree 237/2015, of 22 December
La Rioja	Decree 49/2009, of 3 July
Ceuta	Order EDU/1965/2010, of 14 July
Melilla	Order EDU/1965/2010, of 14 July

Source: Own elaboration

Systematic review of the literature

Following the guidelines of Gough, Oliver and Thomas (2013), a systematic review has been carried out to identify and evaluate studies on the evaluation of the quality of ECE, with special reference to the first cycle. It was conducted from January to March 2020, using the databases Dialnet, ERIC, Scopus and Web of Science. Key words and categories (Table II) are used to specify the search, together with Boolean operators and truncations1.

TABLE II. Categories and key words used in the systematic review of literature

Categories	Key words
Educational stage	Toddler, pre-k, early childhood, preschool. Educación infantil.
Measurement	Measurement, assess*, evaluation, questionnaire. Evaluación, cuestionario, instrumento.
Geographic location	Spain, spani*, Autonomous Communit*. Comunidad* autónoma*, españ*.
Other	Education*. Quality. Calidad

Source: own elaboration

Subsequently, inclusion and exclusion criteria are applied to the identified studies:

Inclusion:

a) Peer-reviewed articles or books in SPI publishers.

b) Studies in Spain (the entire country or some Autonomous Communities).

c) International, but in all cases including Spain.

d) Both cycles of ECE.

e) Tools for quality assessment at this stage.

Exclusion:

a) Articles measuring an aspect of quality, a programme or subject.

b) Articles whose research is not in Spain or does not include Spain.

c) Articles that do not use tools for evaluation.

Results

Regulation of the evaluation of Early Childhood Education in the
autonomous communities and cities

The LOE (2006), and the LOMLOE (2020) that amends it, establish that the Inspection and the diagnostic evaluation of the system and the centres, are the way to improve the quality of the education and training systems. Its implementation and development is the responsibility of the autonomous education administrations. Beyond this, after the review it is concluded that in Spain there are no specific basic regulations governing quality assessment in the first cycle of ECE, although the new Education Law provides the basis for developing a system of quality indicators

Organic Law 3/2020 of 29 December, amending Organic Law 2/2006, of 3 May, on Education (LOMLOE), mandates ECE in the Spanish educational system. The LOE (2006) already collected basic elements of the quality of this stage, maintained by the LOMCE (2013) and now by the LOMLOE (2020). Title I, Chapter I, establishes as general principles that “...is the educational stage with its own identity that caters for children from birth to six years of age” (Art.12.1); “...its purpose is to contribute to the physical, affective, social and cognitive development of students”, to which the LOMLOE (2020) adds artistic development, as well as education in civic values for coexistence (Art.12.3); and it maintains that “...to respect the mothers and fathers or legal guardians fundamental responsibility at this stage, nursery schools will cooperate closely with them (art. 12.4).”The Government, in collaboration with the Autonomous Communities, shall determine the educational content of the first cycle of ECE in accordance with the provisions of this chapter. It shall also regulate the qualification requirements of its professionals and those to be met by the centres providing this cycle, in any case relating to the pupil-teacher ratio, the facilities and the number of school places (Art. 14.7). The curriculum and requirements of the centres in the autonomous cities of Ceuta and Melilla and Spanish educational establishments abroad are regulated by the Ministry of Education and Vocational Training (MEFP).

LOMLOE also strengthens its preventive and compensatory nature by specifying: “The programming, management and development of ECE shall, in any case, be aimed at offsetting the effects of cultural, social and economic inequalities on children’s learning and development, as well as early detection and early attention to specific educational support needs (Art. 12.5). Also noteworthy is the regulation in Article 14 (pedagogical planning and principles) of working methods in both cycles, which “...will be based on emotionally positive learning experiences, activities and play will be applied in an atmosphere of affection and trust, to enhance their self-esteem and social integration and the establishment of a secure attachment” (14.6). In addition, it emphasises its mission: “The educational nature of each cycle will be reflected in a pedagogical proposal by all centres that provide early childhood education (Art. 14.2.) and the transition between stages “To ensure the continuity of the training process and a transition and positive evolution of the student body, the necessary continuity between this stage and primary education will be reflected in curriculum development, which will require close coordination between teachers at both stages. To this end, at the end of the stage, the tutor will issue a report on the development and needs of each student” (Art. 14.8). Another important novelty of the articles of the Law 2020 to control the quality of the offer stands out, adding a general principle: “Schools that regularly have an intake of children between the ages of zero and six during the school calendar must be authorised by the educational authorities as nursery schools” (Art.12.2.) and a requirement in Article 15.1.: “... All establishments shall be approved by and supervised by the corresponding Education Administration”.

Regulations between autonomous communities are mixed. In three of them there is no regulation of the evaluation (Galicia, Madrid and Murcia), and in others it is covered by normatives regulating the functions of specific bodies (Institutes or Centres) for the evaluation of their educational system, such as Aragon, Catalonia, Valencian Community and Basque Country.

Table III presents a comparison of the regulations of those territories with specific reference to the evaluation of the 0-3 cycle. The comparative analysis focuses on the type, nature, agent and application of the evaluation. In nine Communities and in Ceuta and Melilla an external and internal evaluation is regulated -where the teacher is responsible of making a self-assessment report of their teaching practice and to elevate it to Inspection. Aragon, the Canary Islands, the Basque Country and La Rioja only regulate external evaluation. Navarre and Valencia establish that the evaluation is voluntary, both external (schools can apply to be evaluated) and internal.

TABLE III. Characteristics of the evaluation in the first cycle of Early Childhood Education in the Autonomous Communities and Cities regulating it.

CC. y CCAA	Type	Nature	Agent	Proceedure
Andalucía	Internal External	Mandatory Mandatory	School (only public ownership) Educational Inspection	Self-evaluation No reported
Aragón	External	Mandatory	Educational Inspection	No reported
Asturias	Internal External	Mandatory Mandatory	Schools Educational Inspection	Self-evaluation No reported
Baleares	Internal External	Mandatory Mandatory	Schools Educational Inspection	No reported Evaluation plan from centres
Canarias	External	Mandatory	Educational Inspection	No reported
Cantabria	Internal External	Mandatory Mandatory	Teaching staff Educational Inspection	Self-evaluation No reported
Castilla y León	Internal External	Voluntary Mandatory	Schools Educational Inspection	Self-evaluation No reported
Castilla La Mancha	Internal External	Mandatory Mandatory	Centres Educational Inspection	Self-evaluation No reported
Valencian Community	Internal External Int/Ext	Mandatory Mandatory Voluntary	Centres Educational Inspection Educational Inspection	Self-evaluation No reported Quality management system
Extremadura	Internal External	Mandatory Mandatory	Teachers Educational Inspection	Self-evaluation No reported
Navarra	External Int/ext	Mandatory Voluntary	Educational Inspection Educational Inspection	No reported Quality management system
Basque Country	External	Mandatory	Educational Inspection	No reported
Rioja	External	Mandatory	Educational Inspection	No reported
Ceuta Melilla	Internal External	Mandatory Mandatory	Schools Educational Inspection	Self-evaluation No reported

Source: own elaboration

Table III reveals the variability in the regulation of quality assessment at this first stage. In the rest of Europe the situation is very similar: one third of European education systems lack regulations on the internal evaluation of childhood education centres for children under 3 years old, and most countries monitor the entire childhood education system based on each centre individual findings (Eurydice, 2019).

Systematic review of the literature

The final selection of the papers that are part of this review, has taken place after the systematic analysis of the relevant studies according to the established criteria. The first selection of texts resulted in two books and 285 articles.

The books present a model and a pilot study, and the analyses of this study respectively, and focus on the second cycle of childhood education. The first, entitled Evaluation Model for Early Childhood Education (INECSE, 2005), presents a proposal for the evaluation of childhood education that includes three dimensions: assessment of learners’ outcomes at the end of the stage, educational processes developed in the classroom, and the school and family contexts. It aims to be a starting point towards a stable framework for evaluation at this stage that serves as a reference for further studies and encourages the development of external evaluation during the first years of schooling. The second book, entitled Evaluation of ECE in Spain. Report of the 2007 pilot study (INE, 2008), is the report of the previous pilot study. It presents a framework for the external evaluation of childhood education. It includes an in-depth reflection on the curriculum that has allowed the construction of the evaluation tools validated in the pilot study. The extensive study was not conducted.

In respect of the 285 articles, by applying the selection/exclusion criteria to the reading of their summary, 264 pieces of work were discarded. Of the remaining 21 articles, the in-depth reading led to the rejection of 16, because they did not meet the above criteria, although 10 of them met it at least partially. The papers discarded did not make a comprehensive assessment of the quality of education of the cycle or stage, nor did they use a validated evaluation tool, or assessed aspects as relevant as communication and the families´ involvement in a thorough manner. The sample of selected articles is presented in Table IV -they present studies that meet all the criteria- and in Table V -those that fulfill them only partially-, which provides a first view of the research landscape on the subject.

It should be noted that, to the date of writing this article, only 5 studies have been identified that overall evaluate the quality of ECE in Spain with validated tools (Table IV).

All of them have used the Early Childhood Environment Rating Scale (ECERS, 1980), two of them the Observation of Preschool Activities (OAP, 1995) and/or the Early Childhood Environment Rating Scale-Revised version (ECERS-R, 1998) and in one case the Caregiver Interactions Scale (CIS, 1989). A first limitation for our purpose is that the four tools are developed for the 3-6 cycle. In addition, its publication date (ECERS-R) is more than 20 years old. In the three cases not using ECERS-R, they use a version of the tools prior to the last revised version. It also calls to our attention that the most recent study is from 2012, so it does not record the years in which ECE has evolved the most. Finally, the results show that the quality of the second cycle is low or minimal (Table IV).

TABLE IV. Articles that evaluate the quality of Early Childhood Education in Spain globally and with validated instruments.

Author	Cycle	Participants	Application	Instruments	Results
Sandstrom (2012)	2º (3-6)	25 classrooms	Andalucía	ECERS, ECERS-R, OAP	Low quality
Lera (2007)	2º (3-6)	380 classrooms	Spain	ECERS, ECERS-R, OAP	Minimal quality
Lera (1996)	2º (3-6)	59 classrooms	Seville	ECERS, OAP	Low quality
Tietze, Cryer, Bairrão, Palacios & Wetzel (1996)	2º (3-6)	80 classrooms (in Spain)	Austria, Germany, Spain, USA and Portugal	ECERS, CIS	Spain: lowest punctuation in the countries sample
Calder (1996)	2º (3-6)	Minimum 5 classrooms per ountry	España, EEUU, Italia y Suecia	ECERS	Low quality

Source: own elaboration

Selected articles partially meeting the established selection criteria are presented in Table V and complete our review.

TABLE V. Articles/studies only partially meeting the selection criteria

Author	Cycle	Participants	Applications	Techniques /Instruments	Aspects of quality assessed
Sobrino Morrás, & Rivas Borrell (2011)	1	1 centre	Basque Country	Protocol	Educational programme (teacher training, identification with teaching methods, the importance of the school climate and communication in the parent-teacher team).
Pineda Herrero, Moreno Andrés, Úcar & Belvis Pons (2008)	1 and 2	1000 centres	Spain	Surveys, interviews and forums	In-service teacher training
Izagirre, Hernández & García (2014)	1	325 teachers	Basque Country	Questionnaire Cuidando_0-2 Validated and ad hoc	Teaching attitude towards child autonomy
Vendrell Mañós, Geis Balagué, Anglès Virgili, & Dalmau Montalá (2019)	1 and 2	150 teachers	Barcelona	Validated questionnaire and ad hoc	Perception of the importance of free play
Parra, Gomariz, Hernández-Prados & García-Sanz (2017)	Not required	1.953 families	Spain	Validated questionnaire and ad hoc	Families` involvment in the educational process
Melgar Alcantud (2015)	Not required	Case study	Spain	Questionnaires, communicative stories, semi-structured interviews, discussion groups and communicative observations.	Families´ participation at school
Paz-Albo Prieto (2018)	Not required	120 tutors	Spain	Validated questionnaire	Families´ participation
Muela, Larrea, Miranda, & Barandiaran (2019)	2	1.001 children, 54 parents and 94 teachers	Basque Country	Preschool Outdoor Environment Measurement Scale (POEMS), participant observation, images, photographs and drawings, as well as conversations and interviews with children, parents and teachers	Families´ participation, teachers and children outdoor quality
Lebrero Baena & Fernández Pérez (2009)	1 and 2	250 centres	Spain	Validated ad hoc questionnaire applied to teachers on the quality aspects they consider important	Students´ family circumstances, needs of specialized teachers, non-teaching staff, funding and material resources, climate, institutional objectives and planning
de Miguel & García (2004)	2º	96.206 pupils	Madrid	Validated ad hoc questionnaire	Parents´ involvment, activities, children-teachers relationships, personal, basic skill, social, cognitive and motor development.

Source: own elaboration

Finally, we note that the review of the scientific literature on the most widely used set of tools to assess quality in the first cycle of infant education (Table VI) shows that there are 3 of them with greater weight at the international level (ITERS, CLASS and GGA), while there are 2 that have only been applied in the USA (CCIS and ORCE).

TABLE VI. Comparison of evaluation instruments for the quality assessment of education in the first cycle of ECE

Instrument	Year of the latest version	Countries in which they have been applied	Translated to Spanish	Validated in Spain	Good reliability results	Age
ITERS Infant/Toddler Rating Scale	2017	Among others: USA, Chile, Brasil, Colombia, Ecuador, Finland, Norway, Germany, Italy	No. Applied to former versions	No	La Paro, Williamson & Hatfield (2014).	0-36 months
CLASS Clasrromm Assessment Scoring System	2012	USA, Chile, Ecuador, China, Finland, Germany, Portugal	Yes	No	Hu, Fan, Gu, & Yang (2016)	Infant 0-18 months Toddler 18-36 Pre-k 3-5 years old
GGA Guías Globales de Evaluación ACEI	2011	Among others: USA, India, Greece, Italy	Yes	No	Hardin, Bergen, & Hung (2013)	0-6 years old
CCIS Child Caregiver Interaction Scale	2010	USA	No	No	Carl (2017).	0-5 years old
ORCE Observational Record of the Caregiving Environment	2000	USA	No	No	De Schipper, Tavecchio & Van IJzendoorn (2008).	0-5 years old

Source: own elaboration

Conclusions

ECE in Spain is undergoing a process of change. Both the State and the Autonomous Communities’ regulations show a strong trend towards the universalisation of the first cycle of pre-school education and the concern to improve its quality. This calls for a rapid reaction on the regulatory side.

In response to this situation, the present study has sought to highlight the state of the evaluation of ECE in Spain, with special attention to the first cycle, and the evaluation tools employed.

Herrera, Mathiesen and Domínguez (2006) show that, although the relationship between coverage and quality in stage 3-6 is supported by sufficient evidence, this is not fulfilled in stage 0-3, which concurs with the results obtained in the present study.

The picture drawn through the regulatory review is not satisfactory, since there is no system of agreed quality indicators, nor convergence in the criteria on the type of evaluation needed, procedures, agents and controllers in the autonomous administrations. There are even Communities without specific regulations. These practices differ substantially from European and international policies or recommendations advocating homogeneous regulations to ensure quality standards. The Council of Europe has established a framework for quality assessment (2018). The OCDE (2019) claims that most countries must develop minimum standards and practices for the evaluation of the education of children under 3 years old, ensuring a certain level of consensus and homogeneity. As per Save the Children (2019), they highlight the current absence of a systematic record of reliable and valid data on the quality of education in stage 0-3. Furthermore, if we dissect the latest meta-analyses -such as the Engaging Young Children (OCDE, 2018)- it is evident that there are studies from several European countries, but none from Spain. There is only one pilot study from 2005 conducted by the Evaluation Institute (INECSE; 2005; INE, 2008). This reinforces the need to propose a system of periodic and specific evaluation for the whole of this stage that assures the quality and favours the continuous improvement of the educational process.

In contrast to the lack of common criteria identified in the regulatory analysis of the case of Spain, it is worth mentioning a promising policy carried out in the USA, where the Department of Health and Social Services has established the National Early Childhood Quality Assurance Centre. It has developed a system of qualification and quality improvement at this stage, the Quality Rating and Improvement System (QRIS) (Zellman and Perlman, 2008) in order to evaluate, improve and communicate its level of quality.

The review of the literature also reveals that, unfortunately, we do not have sufficient factual evidence in Spain to support the guidelines of an evaluation system at this stage, which requires a rapid effort in this area of research from the responsible bodies and research. Only 5 studies have been identified to be evaluating, in a comprehensive manner and with validated tools, the quality of ECE in Spain, and none of these studies focused specifically on the 0-3 cycle, which distances us from scientific activity in other countries. In fact, the WoS database contains 20 academic publications with the words “ECERS” and “United States/States/USA/America” and 11 articles if we switch to the “ITERS” tool (developed to assess quality from birth to three years old).

In addition, the poor quality of ECE found in the five articles mentioned above deserves special mention. These results are especially worrying when considering the studies of Belsky, (1988), Howes, Phillips and Whitebook (1992) and Rolla and Rivadeneira (2006), in which the existence of a negative correlation between poor quality care and education and child development was observed, highlighting cognitive and socio-emotional aspects.

The findings of this study show that we are at a favourable moment to follow the momentum of other countries and begin an appropriate evaluation process that accompanies the extension of stage 0-3. This evaluation will provide reliable and valid information to develop improvement strategies so that we can offer and guarantee the right to quality education from early childhood that brings us closer to the Sustainable Development Goals -ODS- (UN, 2015). More specifically, target 4.2 of 2030: “Ensure that all girls and boys have access to early childhood care and development services and quality pre-school education (...)”.

As suggestions for improvement and progress in this line of research, it is proposed to take into consideration the initiatives of other countries, to intensify territorial cooperation between administrations and to develop a system of indicators and tools for the collection of agreed, valid and reliable information which can be used at State and Autonomous Community level for diagnostic purposes. Its impact on the design of public policies for ECE from a rights-based approach, which goes beyond the services approach, will also bring us closer to meeting the requirements of the SDGs and the European framework for quality and equal opportunities. Having comparable results will allow us not only to have a solid reference that will drive us to improve individually, but also to share the results and be able to foster good practices between each other. The existence of participatory and articulated structures in Spain -such as the National Institute of Educational Evaluation -formerly the National Institute of Evaluation- and the Institute of Curriculum Development foreseen in the LOMLOE- must grant its due importance to the quality of ECE.

Bibliographic references

Belsky, J. (1988). The effects of infant day care reconsidered. Early Childhood Research Quarterly. 3, 237-272.

Bereday, G. (1968). El método comparativo en pedagogía. Barcelona: Herder.

BOCM Decreto 28/2019, de 9 de abril, del Consejo de Gobierno, por el que se regula la financiación del primer ciclo de Educación Infantil en la Comunidad de Madrid.

Borrell, S., Morrás, &., & López, F. (2005). La evaluación como garantía de calidad en educación preescolar. Revista Española De Pedagogía, 63(232), 511-528.

Bradshaw, P. Hinchliffe, S. &Scholes, A. (2020). Schotish Study of Early Learning and Childcare. NatCen ISBN:9781839600494

Calder, P. (1996). Methodological reflections on using the Early Childhood Environment Rating Scale as a measure to make cross-national evaluations of quality. Early Child Development and Care, 126(1), 27-37.

Carl, B. (2017). Child Caregiver Interaction Scale (CCIS) (Carl, 2010) Compared to Arnett CIS (Arnett, 1989): An Updated Measure to Assess Quality Child Caregiving. International Journal of Education and Social Science 4(8), 31-40

Clarke-Stewart, A. (1987): Predicting child development from child care forms and

features: The Chicago Study, in D. PHILLIPS (Ed.), Quality in Child Care:waht does

research tell us? (21-41).Washington DC: NAEYC.

Comisión Europea/EACEA/Eurydice. (2019). Cifras clave de la educación y atención a la primera infancia en Europa – Edición 2019. Informe de Eurydice. Luxemburgo: Oficina de Publicaciones de la Unión Europea

Consejo Europeo (2018). Integrated early childhood development policies as a tool for reducing poverty and promoting social inclusion - Council Conclusions. Luxemburgo: Publications Office of the European Union

Dickinson, D.K. (2006). Cognitive and Linguistic Building Blocks of Early Literacy. En Handbook of Early Literacy Research, Vol. 2. D.K. Dickinson & S.B. Neuman, Eds. New York: Guilford Publications.

Eurydice/Comisión Europea (2020). Cifras clave de la educación y atención a la primera infancia en Europa. Luxemburgo. Oficina de Publicaciones de la Comisión Europea.

Eurydice (2020). Evaluación de la calidad en la Educación Infantil, Primaria y Secundaria en España. [en línea]. Recuperado de https://eacea.ec.europa.eu/national-policies/eurydice/content/quality-assurance-early-childhood-and-school-education-70_es

Felfe C., Nollenberger, N. & Rodríguez-Planas, N. (2012): Can’t Buy Mommy’s Love? Universal Childcare and Children’s Long-Term Cognitive Development, IZA Discussion Paper No. 7053

Gough, D., Oliver, S., & Thomas, J. (2013). An Introduction to Systematic Reviews. Los Angeles, CA: Sage.

Hardin, B. J., Bergen, D., & Hung, H-F. (2013). Investigating the psychometric properties of the ACEI global guidelines assessment (GGA) in four countries. Early Childhood Education Journal, 41(2), 91-101.

Heckman, J. (2017). Early Childhood Education: Quality and Access Pay Off. Chicago: The Heckman Equation

Heckman, J., & Mosso, S. (2014). The Economics of Human Development and Social HU

Herrera, M. O., Mathiesen. M. E., & Domínguez, P. (2006). Evaluación de entornos educativos en centros parvularios para menores de tres años: la Escala ITERS. Investigaciones en Educación, 6(1), 141-168.

Hilker, F. (1964). La pedagogie comparée. Introduction à son historie, sa théorie et sa practiqué. Paris: Institut Pédagogique National.

Howes, C., Phillips, D. & Whitebook, M. (1992). Thresholds of quality: Implications for the social development of children in center-based child care. Child Development. 63, 449-460.

Hu, B. Y., Fan, X., Gu, C., & Yang, N. (2016). Applicability of the Classroom Assessment Scoring System in Chinese Preschools Based on Psychometric Evidence. Early Education and Development, 27(5), 714–734.

INECSE (2005). Modelo de evaluación para la Educación Infantil. Madrid: Secretaría General Técnica. Ministerio de Educación.

Instituto de Evaluación (2008). Evaluación de la educación infantil en España Informe del estudio piloto 2007. Madrid: Secretaría General Técnica. Ministerio de Educación.

Izagirre, E. H., Hernández, S. O., & García, M. B. O. (2014). Actitudes docentes y autonomía en Educación Infantil 0-2: Un estudio exploratorio en la Comunidad Autónoma del País Vasco (CAPV) Educators’ Attitudes and Autonomy in Childhood Education Age 0-2: An exploratory Study in the Autonomous. Revista de Educación, 365, 150-176.

Junta de Andalucía (2020). Consejo de Gobierno. Ley de medidas de apoyo a la Educación Infantil [en línea]. Recuperado de http://www.juntadeandalucia.es/presidencia/portavoz/educacion/141401/ConsejodeGobierno/ConsejeriadeEducacion/LeydeMedidasdeApoyoalaEducacion/Infantil0a3anos/Primaria/FP

La Paro, K. M., Williamson, A. C., & Hatfield, B. (2014). Assessing quality in toddler classrooms using the CLASS-Toddler and the ITERS-R. Early Education and Development, 25(6), 875-893.

Larrea, I., Lopez de Arana, E., Barandiaran, A., & Vitoria, J. R. (2010). La implicación del niño de 0 a 3 años en las experiencias del aula. International Journal of Developmental and Educational Psychology, 2(1),157-167.

Lebrero Baena, M. P. & Fernández Pérez, M. D. (2009). Indicadores organizativos para la calidad en Educación Infantil. Acción Pedagógica, 18(1), 66-77.

Lera, M.J. (1996). Education under five in Spain: a study of preschool classes in Seville. European Journal of Psychology of Education, XI(2), 139-151.

Lera, M. J. (2007). Calidad de la Educación Infantil: instrumentos de evaluación. Revista de Educación, 343, 301-323.

Ley Orgánica 3/2020, de 29 de diciembre, por la que se modifica la Ley Orgánica 2/2006, de 3 de mayo, de Educación. Boletín Oficial del Estado (BOE) 340, de 30 de diciembre de 2020.

Ley Orgánica 2/2006, de 3 de mayo, de Educación, Boletín Oficial del Estado (BOE) 106, de 4 de mayo de 2006.

Llorent, V. (2013). La educación infantil en Alemania, España, Francia e Inglaterra: Estudio comparado. Revista Española De Educación Comparada, 0(21), 29-58.

McLean, L., Sparapani, N., Toste, J. R., & McDonald-Connor, C. (2016). Classroom quality as a predictor of first graders’ time in non-instructional activities and literacy achievement. Journal of School Psychology, 56, 45-48. doi:10.1016/j.jsp.2016.03.004

Melgar Alcantud, P. (2015). Grupos interactivos en educación infantil: Primer paso para el éxito educativo. Intangible Capital, 11(3), 316-332.

Melhuish, E., Ereky–Stevens, K., Petrogiannis, K., Ariescu, A., Penderi, E., Rentzou, K., Tawell, A., Leseman, P. & Broekhuisen, M. (2015). A Review of Research on the Effects of Early Childhood Education and Care on Child Development. CARE Project Report.

De Miguel, C. R., & García, M. G. (2004). Modelo explicativo de factores vinculados a la calidad en educación infantil. Revista de Investigación Educativa, 22(2), 497-518.

Ministerio de Educación y Formación Profesional (2019a). Las cifras de la educación en España. Estadísticas e indicadores. Edición 2019. Madrid: Secretaría General Técnica.

Ministerio de Educación y Formación Profesional (2019b). Estadísticas de las enseñanzas no universitarias. Madrid: Secretaría General Técnica.

Ministerio de Educación y Formación Profesional (2020a). El primer ciclo de Educación Infantil en las CCAA a través de la revisión normativa. Madrid, Secretaría General Técnica.

Ministerio de Educación y Formación Profesional (2020b): Sistema Estatal de Indicadores de la Educación 2020. Madrid: Secretaría General Técnica.

Muela, A., Larrea, I., Miranda, N., & Barandiaran, A. (2019). Improving the quality of preschool outdoor environments: getting children involved. European Early Childhood Education Research Journal, 27(3), 385-396.

National Institute Of Child Health And Human Development Early Child Care Research (NICHD). (2006). Child care effect sizes for the NICHD Study of early child care and youth development. American Psychologist, 61, 99-116.

OCDE. (2019). Providing Quality Early Childhood Education and Care: Results from the Starting Strong Survey 2018, TALIS, OCDE Publishing, Paris, https://doi.org/10.1787/301005d1-en.

OCDE. (2018). Engagin Young Children: lessons from Research about Quality in early Childhood Education and Care, Starting Strong, OCDE Publishing, Paris.

OCDE (2017). Starting Strong 2017: Key OCDE Indicators on Early Childhood Education and Care. París: OCDE.

ONU. (2015). Objetivos de Desarrollo Sostenible: Agenda 2030. Nueva York: Asamblea General de Naciones Unidas.

ONU.(1989). Convención sobre los derechos del niño. Nueva York: Asamblea General de Naciones Unidas

Parra, J., Gomariz, M-Á., Hernández-Prados, M.-Á. & García-Sanz, M. (2017). La participación de las familias en educación infantil. RELIEVE, 23(1), 1-26 doi: http://doi.org/10.7203/relieve.23.1.9258

Payne, G. & Payne, J. (2004). Key concepts in social research. London: Sage Publications.

Paz-Albo Prieto, J. (2018). Enhancing the quality of early childhood education and care: ECEC tutors’ perspectives of family engagement in Spain. Early Child Development and Care, 188(5), 613-623.

Phillips, D., Scarr, S. & Mccartney, K. (1987): Dimensions and effects of child care quality: The Bermuda Study. In D. Phillips (Ed.), Quality in child care: what does research tell us? Washington, D.C.:NAEYC.

Pineda Herrero, P., Moreno Andrés, M., Úcar, X., & Belvis Pons, E. (2008). Derecho a la calidad: Evaluación de la formación permanente en el sector de la educación infantil en España. Revista De Educación, 347, 101-126.

Pungello, E., Kainz, K., Burchinal, M., Wasik, B., Sparling, J., Ramey, C., & Campbell, F. (2010). Early Educational Intervention, Early Cumulative Risk, and the Early Home Environment as Predictors of Young Adult Outcomes Within a High-Risk Sample. Child Development, 81(1), 410-426.

Raventós, F. (1991). Metodología Comparativa y Pedagogía Comparada. Barcelona: Boixareu Universitaria.

Rolla, A., & Rivadeneira, M. (2006). ¿Por qué es importante y cómo es una educación preescolar de calidad?. En foco, 76, 1-16.

Ruopp, R. R. & Travers, J. (1982). Janus faces day care: Perspective on quality and cost. In E.F. Zigler & E.W. Gordon (Eds.), Day care: Scientific and social policy issues, 72-101. Boston: Auburn House.

Sandstrom, H. (2012). The characteristics and quality of pre-school education in Spain. International Journal of Early Years Education, 20(2), 130-158.

Save the Children. (2019). Donde todo empieza. Educación infantil de 0 a 3 años para igualar las oportunidades. Madrid: Save the Children España.

Sylva, K., Siraj-Blatchford, I., Taggart, B., Sammons, P., Melhuish, E., Elliot, K., & Totsika, V. (2006). Capturing quality in early childhood through environmental rating scales. Early childhood research quarterly, 21(1), 76-92.

De Schipper, J. C., Tavecchio, L. W., & Van IJzendoorn, M. H. (2008). Children’s attachment relationships with day care caregivers: Associations with positive caregiving and the child’s temperament. Social Development, 17(3), 454-470.

Sobrino Morrás, A. & Rivas Borrell, S. (2011). Determining quality of early childhood education programmes in Spain: A case study. Revista De Educación, (355), 193-194.

Tietze, W., Cryer, D., Bairrão, J., Palacios, J., & Wetzel, G. (1996). Comparisons of observed process quality in early child care and education programs in five countries. Early Childhood Research Quarterly, 11(4), 447-475.

UNICEF. (2014). Building Better Brains: New Frontiers in Early Childhood Development. UNICEF: Nueva York.

Unión Europea. (2017). Pilar europeo de derechos sociales. Luxemburgo: Oficina de publicaciones de la Unión Europea. doi:10.2792/506887

Valverde-Forttes, P. (2015). Ambientes de calidad en la infancia temprana. Pensando Psicología, 11(18), 141-151. doi: http://dx.doi.org/10.16925/ pe.v11i18.1224

Velloso, A. & Pedró, F. (1991). Manual de educación comparada. Vol. 1 Conceptos básicos. Barcelona: PPU.

Vendrell Mañós, R., Geis Balagué, À., Anglès Virgili, N., & Dalmau Montalá, M. (2019). Percepción de los maestros sobre el derecho al juego libre en educación infantil y educación primaria. Estudio desarrollado en Barcelona (España). Bordón. Revista de Pedagogía, 71(4), 151-165. D

Contact address: Andrea Otero-Mayer. Universidad Nacional de Educación a Distancia, Facultad de Educación. Departamento de Métodos de Investigación y Diagnóstico en Educación II. C/ Juan del Rosal, 14, 28040, Madrid. Despacho 2.26.

---