CLIL lesson planning in Education student teachers:
a case study

La programación de lecciones AICLE en estudiantes de grado de Educación: un estudio de caso

https://doi.org/10.4438/1988-592X-RE-2023-399-561

Magdalena Custodio Espinar

https://orcid.org/0000-0001-5314-1606

Universidad Pontificia Comillas

José Manuel García Ramos

https://orcid.org/0000-0002-5623-0900

Universidad Complutense de Madrid

Abstract

The content and language integrated learning (CLIL) approach, introduced by the European Union to promote the teaching of foreign languages and second languages, requires adequate training of future CLIL teachers. This pretest-posttest study analyzes the competence to plan CLIL lessons of a sample of students in pre-primary and primary education degrees before (N=56) and after (N=50) taking a CLIL course that is part of their studies. The objective is to analyze the effectiveness of the course by studying the differences between both groups with the CIPMA questionnaire (Cuestionario de Integración de los Principios Metodológicos AICLE), a questionnaire, already validated, that specifically measures teachers’ competence to plan CLIL lessons. These quantitative results were completed with an analysis of the CLIL lesson plans that both groups designed as part of the course. From the quantitative analysis carried out, it can be deduced that both groups of students acquire the competence to plan their CLIL lessons and that, despite some differences found in some of the areas of their lesson plannings, the CLIL course provides them with the strategies to adapt the methodological principles of the approach to the idiosyncrasies of their respective educational stages. Despite the limitations of the study, it is concluded that the integration of methodological training in CLIL, together with linguistic training, in both pre-primary and primary education degrees, is effective and necessary for the acquisition of the basic knowledge to plan lessons under this approach.

Keywords: pre-primary education, primary education, CLIL, lesson planning, higher education, training.

Resumen

El aprendizaje integrado de contenido y lengua (AICLE), introducido por la Unión Europea para el fomento de la enseñanza de lenguas extranjeras y segundas lenguas, exige una adecuada formación de los futuros maestros AICLE. Este estudio pretest-posttest analiza la competencia para programar lecciones de AICLE de una muestra de estudiantes de educación infantil y primaria antes (N=56) y después (N=50) de cursar una asignatura de AICLE que forma parte de sus estudios de grado. El objetivo es analizar la eficacia del curso mediante el estudio de las diferencias entre ambos grupos con el CIPMA (Cuestionario de Integración de los Principios Metodológicos AICLE), cuestionario ya validado que mide específicamente la competencia para programar lecciones AICLE de los docentes. Estos resultados cuantitativos se completaron con un análisis de las programaciones de AICLE que ambos grupos diseñaron como parte del curso. De los análisis cuantitativos realizados se desprende que tanto el grupo de infantil como el de primaria adquieren la competencia para programar AICLE y que, a pesar de algunas diferencias encontradas en algunas de las áreas de sus programaciones, el curso AICLE les dota de las estrategias para adaptar los principios metodológicos del enfoque a la idiosincrasia de sus respectivas etapas educativas. Pese a las limitaciones del estudio, se concluye que la integración de la formación metodológica en AICLE, junto con la lingüística, en los grados de educación, tanto de infantil como de primaria, es eficaz y necesaria para la adquisición de los conocimientos básicos para programar lecciones bajo este enfoque.

Palabras clave: educación infantil, educación primaria, AICLE, programación, educación superior, formación.

Introduction

Content and language integrated learning approach (CLIL) has been promoted by the European Union (EU) since the beginning of this century (European Commission, 1995) for the teaching of foreign languages, second languages and third languages. Hence, this educational approach is present in the Spanish regulations and curricula of compulsory educational stages (Guillén Díaz & Sanz Trigueros, 2019). In addition, the recent curriculum for pre-primary education, which is not compulsory, also makes explicit reference to the “use of methodologies for integrated learning of content and foreign languages” (Royal Decree 95/2022, Second Additional Provision, p. 14567).

The educational approach implicit in this regulation is aimed at avoiding compartments and offering a framework for the production of globalized, open and flexible teaching materials likely to be adapted to the realities of students at different stages. Therefore, although there are many agents involved in curriculum development, as stated by Beacco et al. (2016), it is ultimately the teachers who are responsible for choosing and implementing the learning experiences and activities that are meaningful for the students, allowing them to achieve the learning goals.

This curricular framework is, therefore, potentially ideal for the development of CLIL programs at all educational stages from the perspective of multilingual and multicultural competences development, which are promoted by the policy for language learning in the EU, as described in the Common European Framework of Reference for Languages (CEFR) (Council of Europe, 2001). These competences are of special interest in the development of CLIL proposals and have been specified in the Companion Volume (Council of Europe, 2020), which introduces descriptors of the mediation processes and of both linguistic and non-linguistic resources to evaluate those competences, among others. As Piccardo et al. (2019) explain, the CEFR sees students as social agents who mobilize all their skills, the general ones (that is, personal, not linguistic), as well as strategies for the fulfillment of specific tasks, which results in a proportional improvement of those skills and strategies.

In relation to teacher education for the implementation of CLIL methodologies, there are several frameworks of reference that describe the competencies and areas of training for CLIL teachers. Among them, there are those elaborated by Marsh et al. (2010) and Bertaux et al. (2010), published within the development of the EU language policy. Likewise, there are works by Spanish authors that describe the competencies of the CLIL teacher (Pavón & Ellison, 2013; Madrid & Madrid, 2014; Pérez-Cañado, 2015). All of them include the competence to plan lessons as a necessary skill for the CLIL teacher. However, in addition to these skills, CLIL lesson planning in pre-primary and primary education, in the current context, requires the promotion of the aforementioned multilingual and multicultural skills and, therefore, demands for specific training of the teachers responsible for its design (Torres Zúñiga & Carrasco Flores, 2020). Beacco et al. (2016) summarize this training as follows: knowledge about plurilingualism, repertoire and the corresponding acquisition processes; knowledge about the linguistic dimensions of all learning processes; the ability to activate transfer strategies from one language, one skill or one subject to another; reflexivity in the learning process, in particular, the learning of grammar; a differentiated approach to language rules; mediation and openness to the other and to mobility; other concepts of evaluation (in particular, plurilingual and intercultural transversal competences).

This professional profile of the CLIL teachers, capable of designing their own curricular programs from a transversal, plurilingual and pluricultural perspective, makes it necessary from an academic point of view, as stated by Delicado and Pavón (2015), Palacios et al. (2018) and Estrada (2021), among others, that bilingual teacher training initiatives in higher education focus not only on language skills (Torres Zúñiga & Carrasco Flores, 2020), but also on the methodological competence of future CLIL teachers. In the words of Palacios et al. (2018), the specific methodological treatment required by bilingual teaching implies, therefore, a specific training for teachers (p. 147). Specifically, there is a fundamental competence that this cross-curricular approach demands from CLIL teachers: the understanding of the role played by the linguistic dimension in its synergies with the different areas or subjects with which it is integrated (Beacco et al., 2015; Morton, 2016). The work of Guillén Díaz and Sanz Trigueros (2019) is a clear example of the difficulty for CLIL teachers to implement curricula in practice from the perspective of bilingual education. Specifically, their studies point out the difficulties to determine and formulate the objectives of bilingual education.

Although there are specializations in bilingual teaching in some education degrees of both state and private universities, for example, Andalusia (Zayas and Romero, 2017) and Madrid (López-Hernández, 2021), initial training in CLIL is far from being widespread in education degrees (Custodio-Espinar, 2019a, Jover et al., 2016; Palacios et al., 2018), despite the complexity involved in this teaching approach. The study by Cortina-Pérez and Pino Rodríguez (2021), on the level of competence as a CLIL teacher perceived by pre-primary education students at their university, suggests that future teachers do not yet feel prepared to become CLIL teachers in the pre-primary classroom. In addition, the authors highlight that the level of communication skills about CLIL is a predictor of their knowledge about the approach.

In short, the duality between language and content that CLIL demands from both learners and teachers requires universities to address, from the point of view of teacher training, both foreign language learning (Palacios et al., 2018) and training in the challenge of integrating the foreign language with the content (Morton, 2016). Regarding the linguistic competence, Sierra Macarrón and López Hernández (2015) claim that initial training of these teachers must meet the academic requirements derived from their university status, but, in addition, teacher educators must be able to model their teaching based on what the professional profile of bilingual teachers demands (p. 18). In line with this requirement, Torres Zúñiga and Carrasco Flores (2020) claim the need to adapt the teaching of the foreign language to the professional needs of the target context through a language approach (mainly English) for specific purposes (ESP). In relation to pedagogical competence, Bolarín Martínez et al. (2019) summarize the methodological needs of the bilingual classroom as follows: active methodologies, cooperative classroom management, emphasis on all types of linguistic and non-linguistic communication, lesson planning, coordination, evaluation and knowledge about language acquisition.

This work focuses on one of these methodological requirements, the competence to plan CLIL lessons. This competence implies the management of a series of methodological principles, described in Custodio-Espinar (2019b), which demand both linguistic and methodological skills. Specifically, the study analyzes the competence in the following fundamental areas of CLIL programming: language treatment, content and language integration, methodological strategies, attention to diversity, resources and evaluation. These areas correspond to the dimensions and subdimensions of the CIPMA (Cuestionario de Integración de los Principios Metodológicos AICLE) (Custodio-Espinar & García Ramos, 2020), used in this study to measure the competence to integrate CLIL methodological principles in the lesson plans of a sample of pre-primary and primary education students. Likewise, the study aims to analyze some specific aspects of these areas that are present in the CLIL lesson plannings of these students.

Method

Research method, objectives and variables

This study, which uses a mixed methodology, is a pretest-posttest study to measure the impact that a CLIL course has on the level of competence to plan CLIL lessons of future pre-primary and primary education teachers.

The lesson plannings that both groups designed as part of the CLIL course they take in the fourth year of the degree were analyzed to triangulate these quantitative results. The subject consists of six modules: M1 Bilingual education and bilingual schools, M2 Theories behind CLIL, M3 A CLIL lesson plan, M4 Scaffolding learning in CLIL, M 5 Assessment and Evaluation, M6 Skills and resources for the CLIL classroom. The analysis carried out was of qualitative content to verify the presence and frequency of the curricular areas of each stage, the textual genres linked to the final production tasks designed by the students and the learning standards of the official curricula of both educational stages, included in their lesson plannings, and their meaning within the context of CLIL programming.

The objectives of the research are:

• To analyze the effectiveness of a CLIL course for the training of future CLIL teachers of pre-primary and primary education.
• To compare the competence to plan CLIL lessons of CLIL students of pre-primary and primary education degrees.
• To know what characterizes the CLIL lesson planning of pre-primary education students compared to their peers in primary education.
• To define the training needs of future CLIL teachers in relation to the competence to plan CLIL lessons.

The variables of the quantitative study are:

• The dependent variables (DV) are five:

• Global: Total score in the CIPMA.

• Specific: Score in each of the four dimensions of the CIPMA: D1 CLIL core elements, D2 Methodology, D3 Resources and D4 Evaluation.

• The main independent variables (IV) are two:

• Pretest-posttest: pre and post measurement in the two groups in the five DV.

• Stage of education: pre-primary and primary.

Other secondary independent variables analyzed are perceived level of English (B1/B2/C1/C2), perceived academic performance in subjects taught in English (low/medium/high), perceived level of CLIL training (not at all/little/sufficient/ quite a lot/a lot), desire to receive training (EFL training/ CLIL training/ training in both/ no training), interest in being a CLIL teacher (yes/no/maybe) and type of school placement during the internships (bilingual/non-bilingual). It should be noted that, although the variable English level is perceived, the students know their actual level because they take placement exams at the beginning of the year to confirm their assignment to the different level groups, from A2 to C2. All the subjects of the degree studied in English, as part of the English language major, are divided in level groups in years 1 and 2 of both degrees. Subsequently, in the third and fourth years, these subjects are grouped into two levels that go from B1 to B2.1 and from B2.2 to C2 depending on the number of students in the different levels.

With respect to the qualitative analysis, the variables analyzed are the areas of the curriculum, the textual genres linked to the curricular content and the learning standards organized in lower order and higher order cognitive skills to compare the cognitive demand in the CLIL lesson plannings of both stages. These variables have been selected because they allow assessing the integration of content and language in CLIL programming.

Research hypotheses

The main hypothesis of the research is the following: the study of a CLIL course, in the degrees of pre-primary and primary education, allows improving the competence to plan CLIL lessons of these potential bilingual teachers.

The specific hypotheses that derive from the previous one are eight and revolve around the following main axes.

• Axis 1: CLIL lesson planning.

• H1 The CLIL course improves the competence to plan CLIL lessons of the students of the education degrees.

• H2 No statistically significant differences are expected in the competence to plan CLIL lessons between undergraduate students of pre-primary education and undergraduate students of primary education who have taken the CLIL course.

• Axis 2: Communicative competence in English and CLIL.

• H3 The higher level of perceived English improves the level of competence to plan CLIL lessons.

• H4 The higher level of perceived academic performance, in the subjects taught in English, improves the level of competence to plan CLIL lessons.

• H5 The higher level of perceived CLIL training improves the level of competence to plan CLIL lessons.

• Axis 3: Motivation of students towards the CLIL approach.

• H6 Greater interest in receiving training improves the level of competence to plan CLIL lessons.

• H7 Greater interest in being a CLIL teacher improves the level of competence to plan CLIL lessons.

• H8 Doing the internship in a bilingual school improves the level of competence to plan CLIL lessons.

With the aim of delving into the differences in the level of integration of CLIL methodological principles between the CLIL lesson plannings designed by students of both degrees, once the analyses to contrast the previous hypotheses were carried out, the qualitative analysis was performed to identify and clarify the differences and/or similarities between the CLIL lesson plannings of both groups.

Participants

The population of this study consists of the students of the pre-primary and primary education degrees designed from the Bologna process that involved the adaptation of these degrees to the European Higher Education Area (MECD, 2003). The sample is made up of CLIL students of pre-primary and primary education in the fourth year of the 2020-2021 academic year at Comillas Pontifical University. The sample was distributed in 56 students in the pretest and 50 in the posttest. Of them, in the pretest, 25 are from pre-primary and 31 from Primary, while in the posttest, 20 are from pre-primary and 30 from Primary. Therefore, it is a non-probability sampling, specifically a convenience sampling for a case study.

All students in both degrees study the specialization in English language, which consists of five subjects with the following credit load: English for Education I (12 ECTS), English for Education II (6 ECTS), Teaching English as a Foreign Language I (6 ECTS), Teaching English as a Foreign Language II (6 ECTS) and Content and Language Integrated Learning (5 ECTS).

In the posttest, 92% of the sample were women and their English level according to the CEFR (Council of Europe, 2001, 2020) was medium-high: level B1=8%, B2 =50% and C1=42%. These data correlate with the perceived level of the sample in relation to their performance in the subjects studied in English, with a cumulative percentage of 80% who perceived their performance as medium or high, between notable-sobresaliente 60% and sobresaliente 20%. With regard to the perceived level of CLIL training in the sample, once the subject was completed, the majority considered that it was sufficient (44%) or quite a lot (52%). In addition, 26% only wanted training in English, 20% only in CLIL, 48% in both, and 6% in none of them. Finally, a large majority of the sample carried out their internships in school placements that have implemented a bilingual education program (82%) and most expressed their interest in being a CLIL teacher in the future (60%) or considered this possibility (30%), while only 10% stated that they are not interested in being a CLIL teacher.

Instruments

The instrument used to collect the information in the quantitative study is the questionnaire called CIPMA (Appendix A), already validated in Custodio-Espinar and García Ramos (2020). The main data of the psychometric study are shown in Appendix B. In the qualitative study, the CLIL lesson plan template revised in Custodio-Espinar (2019b) (Appendix C) has been used. Table I shows the most relevant information about the questionnaire.

TABLE I. Description of the CIPMA

Dimensions	Subdimensions	N of items
Dimension 1 CLIL core elements	D1.1 Language treatment D1.2 4Cs integration	5
		5
Dimension 2 Methodology	D2.1 Attention to diversity D2.2 Methodological strategies	4
		5
Dimension 3 Resources		2
Dimension 4 Evaluation		2
Total of item-variables		23
Criterion-items		3

These 23 item-variables are measured on a 1-6 Likert scale, where 1 means never and 6 always. The questionnaire includes three criterion-items (CI) to measure criterial validity.

Regarding the template used to plan the CLIL lessons, its structure coincides with the four dimensions of the CIPMA and, therefore, allows triangulating the data obtained in both analyses. For an exhaustive description of the template and its relationship with the CIPMA, see Custodio-Espinar (2019b). Both instruments are directly related to the content of the CLIL course taken by the sample.

Data collection and analysis

The data for the quantitative analysis were collected at the beginning of the CLIL course, in February 2021, and at the end, in May-June of the same year. In both cases, the students responded to the same online version of the CIPMA. The data collected was analyzed with IBM SPSS 26 application. These analyses are descriptive and differential. The means and their standard deviation were used as descriptive statistics, and to measure the significance of the differences in the means, the Student’s t-test was used for independent samples and simple ANOVA with Tukey for the posterior contrasts of the groups. The significance level was set at 5%. For both statistics, the effect size (ES) was calculated, which was interpreted according to Cohen’s (1988) criteria.

The degree of reliability to check the global internal consistency of the instrument was measured by calculating Cronbach’s alpha. Finally, an analysis of the convergent validity of the instrument and its dimensions (Pearson-type correlations) was also carried out.

The CLIL lesson plannings were analyzed with NVivo application. They were completed as a task included in the final evaluation of the course, with a value of 30% in the final grade of the subject. The task consists of selecting curricular content from non-linguistic areas of their respective stages and designing CLIL lesson plannings in the template provided (Appendix C). The evaluation criteria for the task are included in Appendix D. Qualitative content analysis was chosen because it is intended to draw inferences and identify certain specific characteristics within the lesson plannings systematically and objectively. This analysis was carried out by one of the authors who is part of the four members of the teaching team of the subject and was chosen because it allows comparing the CLIL lesson plannings of both groups to identify and define their specific characteristics.

Results

Reliability and validity of the CIPMA in the sample

The reliability of the CIPMA in the sample was measured by calculating Cronbach’s alpha in the total scale, the 23 items, and in each of its four dimensions. In the pretest sample (N=56) the value of α is 0.983, very close to 1, which shows excellent reliability. In the posttest (N=50), this value is slightly lower, but still excellent, 0.939. In the dimensions, the result of this analysis is also very good in the pretest: D1: N=10; α is equal to 0.965; D2: N=9; α is equal to 0.970; D3: N=2; α is equal to 0.899; and D4: N=2; α is equal to 0.820. In the posttest sample, the alpha value is slightly lower in D1 (0.878) and in D2 (0.881), much lower in D3 (0.401), and in D4 it is also significantly lower than in the pretest (0.601).

Table II shows the Pearson-type correlations of the scale and its dimensions with the three criterion-items (CI) in the total responses before (N=56) and after (N=50) the CLIL course.

TABLE II. Pre-posttest criterion validity of the CIPMA

Questionnaire/ Dimensions	Corr. CI1 Pretest	Corr. CI2 Pretest	Corr. CI3 Pretest	Corr. CI1 Posttest	Corr. CI2 Posttest	Corr. CI3 Posttest
Scale	0.645**	0.251	0.557**	0.458**	0.328*	0.579**
Dimension 1	0.602**	0.254	0.487**	0.415**	0.315*	0.575**
Dimension 2	0.618**	0.217	0.555**	0.440**	0.284*	0.527**
Dimension 3	0.713**	0.337*	0.580**	0.368**	0.382**	0.421**
Dimension 4	0.588**	0.159	0.619**	0.344*	0.196	0.509**

** The correlation is significant at 0.01 level (bilateral)

* The correlation is significant at 0.05 level (bilateral)

These results show that there is a statistically significant linear correlation between the total score in the questionnaire and the 3 CI, since the significance has a p value ≤ 0.01 in all the correlations, except in CI2, which is not significant in the pretest, although it is at 5% in the posttest. Likewise, the correlations between the dimensions and IC1 and IC3 are significant at 1%, except for the IC1 of the posttest in D4, which is significant at 5%. The correlations with CI2 are not significant in the pretest, except in D3, nor in D4 of the posttest. In the posttest, however, D1 and D2 are significant at 5% and D3 at 1%.

Descriptive analysis

The result of the descriptive analysis of the dependent variables in pretest and posttest is shown in Table III.

TABLE III. Descriptive analysis of the dependent variables in the IV Pretest-posttest

Variables	IV Pretest			IV Posttest
DV	N	Mean	SD	N	Mean	SD
Scale	56	71.98	32.296	50	104.74	14.090
D1	56	32.13	14.550	50	46.32	6.478
D2	56	27.07	12.954	50	40.09	6.283
D3	56	7.17	3.369	50	9.84	1.521
D4	56	5.66	2.843	50	8.49	1.616

All the posttest means are higher and the standard deviation in the posttest responses is notably lower than that of the pretest. This is clearly seen in Graph I, which shows the distribution of the main dependent variable, the full scale (23 items), in the sample at both times of the study.

GRAPH 1. Distribution of the main dependent variable in the sample in the pretest and the posttest

Source: compiled by author

The results of the descriptive analyzes of the IV Stage of education are shown in Table IV.

TABLE IV. Descriptive analysis of the dependent variables according to the IV Stage of education

Variable	IV Stage of education Pretest					IV Stage of education Posttest
DV	N	Mean Pre-pr.	Mean Prim.	SD Pre-pr.	SD Prim.	N	Mean Pre-pr.	Mean Prim.	SD Pre-pr.	SD Prim.
Scale	PP=25 P=31	74.80	69.71	29.20	34.90	PP=20 P=30	102.80	105.00	17.10	13.04
D1	PP=25 P=31	33.20	31.26	13.22	15.70	PP=20 P=30	44.70	46.83	7.94	5.48
D2	PP=25 P=31	28.20	26.16	12.68	13.31	PP=20 P=30	39.90	39.83	7.03	6.04
D3	PP=25 P=31	7.40	6.90	3.04	3.65	PP=20 P=30	9.60	9.80	1.50	1.52
D4	PP=25 P=31	6.00	5.39	2.78	2.91	PP=20 P=30	8.60	8.53	1.70	1.89

It is noteworthy that all the means of the pre-primary education group are higher in the pretest and have a smaller standard deviation in the scores. However, in the posttest, the primary education group increases its means and outperforms the pre-primary group on the global scale (the sum of the four dimensions) and Dimension 1 CLIL core elements and significantly reduces the standard deviation in all variables.

Differential studies on the competence to plan CLIL lessons

The result of comparing the means of the sample in the CIPMA before and after studying the CLIL course is shown in Table V.

TABLE V. Differences in the level of competence to plan CLIL lessons according to the IV Pretest-posttest by groups

Hypothesis	Pre-primary group					Primary group
	Student t	Sig.	Mean diff.	Standard error	ES**	Student t	Sig.	Mean diff.	Standard error	ES**
H1 Scale	-3.79*	0.001	-28.00	7.379	-1.138	-5.20*	0.001	-35.29	6.790	-1.33
H1.1 D1	-3.42*	0.001	-11.50	3.360	-1.027	-5.14*	0.001	-15.58	3.032	-1.32
H1.2 D2	-3.69*	0.001	-11.70	3.168	-1.108	-5.14*	0.001	-13.67	2.662	-1.32
H1.3 D3	-2.96*	0.005	-2.20	0.744	-0.887	-4.03*	0.001	-2.90	0.719	-1.03
H1.4 D4	-3.66*	0.001	-2.60	0.710	-1.099	-5.00*	0.001	-3.15	0.630	-1.28

** Levene’s test is significant (p < 0.05), equal variances is not assumed

** Values to interpret the effect size (ES) (Cohen, 1988): d ≥ 0.20 small; d ≥ 0.50 median; d ≥ 0.80 large

The result of this analysis shows that the CLIL course improves both the global competence to plan CLIL lessons, as well as the competence in each of its dimensions, in the sample of pre-primary and primary education students. A large magnitude of the ES is appreciated in all the differences, greater than 0.80 for this statistic.

The result of the differential analyzes of the IV Educational stage are shown in Table VI.

TABLE VI. Significance of differences in the level of competence to plan CLIL lessons according to the IV Educational stage (Pre-primary/Primary), pre-posttest

Hypotheses DV	Levene (homog. of var.)	Pretest PP Mean	Pretest P Mean	Sig. of t	Levene (homog. of var.)	Posttest PP Mean	Posttest P Mean	Sig. of t
H2 Scale	Yes	74.80	69.71	0.562	No	102.80	105.00	0.628
H2.1 D1	Yes	33.20	31.26	0.624	No	44.70	46.83	0.303
H2.2 D2	Yes	28.20	26.16	0.563	Yes	39.90	39.83	0.972
H2.3 D3	Yes	7.40	6.90	0.588	Yes	9.60	9.80	0.649
H2.4 D4	Yes	6.00	5.39	0.428	Yes	8.60	8.53	0.899

These results show that there are no statistically significant differences in the competence to plan CLIL lessons between the pre-primary and primary groups, neither before nor after studying the CLIL subject.

The results of the analyses to contrast the secondary hypotheses of the study are summarized below (Table VII).

TABLE VII. Significance of the differences in the level of competence to plan CLIL lessons according to the secondary IV

Hypothesis/Variable	Test	Sig. Scale	Sig. D1	Sig. D2	Sig. D3	Sig. D4
H3 IV English level	One way ANOVA	0.094	0.032*	0.360	0.503	0.093
H4 IV Perceived academic achievement	One way ANOVA	0.139	0.070	0.248	0.498	0.355
H5 IV Perceived CLIL training level	One way ANOVA	0.081	0.089	0.197	0.047*	0.166
H6 IV Interest in further training	One way ANOVA	0.740	0.588	0.826	0.633	0.655
H7 IV Interest in being a CLIL teacher	One way ANOVA	0.181	0.372	0.055	0.573	0.762
H8 IV Type of school placement	Student t	0.102	0.514	0.038*	0.001*	0.348

* Mean difference is significant at 0.05 level

The table shows statistically significant differences in D1 CLIL core elements in H3 that studies the effect of the IV English level with a value of η²=0.14. This result indicates a large magnitude for this statistic (F). The post hoc tests show that the differences are between English levels B2 and C1, in favor of level C1 with a value of p=0.06 (Tukey). The significant differences in D3 Resources (H5), which measures the differences in the perceived level of CLIL training, have a value of η²=0.12. This SE result is also large according to Cohen (1988). These differences are between the groups that perceive “quite a lot” and “sufficient” CLIL training, in favor of the “quite a lot” group, with a value of p=0.04 (Tukey). Finally, the statistically significant differences in dimensions 2 Methodology and 3 Resources found in H8, which reflects the influence of the IV Type of school placement (bilingual/non-bilingual), are in favor of students who have done the internships in a bilingual school. Cohen’s d values of 0.79 and 1.24 respectively (D2 and D3) are equal or exceed the threshold of 0.80 indicating large magnitudes of the ES.

Results of qualitative content analysis of CLIL lesson plannings

Graph II allows to compare the word clouds generated from the areas and contents selected by the pre-primary education group and the primary education group in their lesson plannings, respectively.

GRAPH II. Word frequency clouds for the code “Areas of Pre-primary Education” (above) and “Areas of Primary Education” (below), with the criteria: all words, with derived words, minimum length three characters

Source: compiled by author

In the cloud of pre-primary students, the occurrence of terms related to the areas of “Knowledge of the environment” and “Knowledge of oneself and personal autonomy” in the foreground is notable. In the background, the presence of the area of “Languages” can also be seen in the references “Body language” and “Musical language”. Likewise, in the cloud resulting from the primary lesson plannings in this code, the areas of “Natural Sciences”, “Social Sciences” and “Artistic Education” stand out in the foreground. At this stage, in the background, the division of the “Artistic education” area into “Arts & Crafts” and “Music” can be seen equally highlighted together with the subject “Religion”, which is part of the curriculum of both degrees. The third level of occurrence clearly shows a preference in both stages for programming content from the area of “Natural Sciences”, called “Knowledge of the environment” in the pre-primary education curriculum.

With regard to the preferred textual genres in the CLIL lesson plannings analyzed, Graph III shows the comparison between the 25 lesson plannings designed by pre-primary students and the 34 by primary students with a total of 29 and 46 references, respectively.

GRAPH III. Textual genres chosen for the CLIL lesson plannings by pre-primary (top) and primary (bottom) students

Source: compiled by author

In this case, there is discrepancy between both groups. In pre-primary, genres that allow working the language from simple units such as adjectives and names, in the descriptions, or verbs in the imperative mode in instructions, are preferred. On the other hand, the CLIL lesson plannings of the primary group show a preference for presentations that imply the handling of textual genres simultaneously such as explanations and descriptions at the same time. In the third and fourth plane of the analysis, the richness and variety of genres worked on in the primary lesson plannings with respect to those of pre-primary is evident, with examples such as debate, recount, report, discussion, letter, script and questions.

Finally, the frequency analysis of the learning standards designed, which are organized in lower and higher order cognitive skills, shows the cognitive demand of the CLIL lesson plannings of each stage. Table VIII summarizes the percentages by category in the lesson plannings of the pre-primary stage.

TABLE VIII. Frequency analysis of learning standards in the pre-primary CLIL lesson plannings by cognitive categories

Lower order cognitive skill	Counting	Weighted percentage	Higher order cognitive skill	Counting	Weighted percentage
Classify	17	19.10	Create	19	46.34
Identify	15	16.85	Asses	7	17.07
Name	11	12.36	Compare	3	7.32
Compare	9	10.11	Appraise	2	4.88
List	7	7.87	Formulate	2	4.88
Recognize/se	6	6.74	Hypothesize	2	4.88
Describe	4	4.49	Adapt	1	2.44
Label	3	3.37	Articulate	1	2.44
Relate	2	2.25	Defend	1	2.44
Remember	2	2.22	Design	1	2.44
Understand	2	2.22	Distinguish	1	2.44
Use	2	2.22	Reflect	1	2.44
Act	1	1.12
Apply	1	1.12
Connect	1	1.12
Create	1	1.12
Distinguish	1	1.12
Explain	1	1.12
Locate	1	1.12
Model	1	1.12
Roleplay	1	1.12

Table IX shows the same results corresponding to the CLIL lesson plannings designed by the primary group.

TABLE IX. Frequency analysis of learning standards in the primary CLIL lesson plannings by cognitive categories

Lower order cognitive skill	Counting	Weighted percentage	Higher order cognitive skill	Counting	Weighted percentage
Identify	18	15.25	Asses	11	16.18
Recognise/ze	14	11.86	Compare	7	10.29
Classify	12	10.17	Design	6	8.82
Create	9	7.63	Examine	5	7.35
List	9	7.63	Reflect	5	7.35
Locate	6	5.08	Analyse/ze	5	7.35
Name	6	5.08	Create	4	5.88
Remember	6	5.08	Select	4	5.88
Describe	4	3.39	Appraise	3	4.41
Explain	4	3.39	Formulate	3	4.41
Match	4	3.39	Classify	2	2.94
Compare	3	2.54	Hypothesize	2	2.94
Organize	3	2.54	Test	2	2.94
Relate	3	2.54	Choose	1	1.47
Associate	2	1.69	Decide	1	1.47
Define	2	1.69	Defend	1	1.47
Understand	2	1.69	Differentiate	1	1.47
Use	2	1.69	Discuss	1	1.47
Apply	1	0.85	Distinguish	1	1.47
Design	1	0.85	Experiment	1	1.47
Extract	1	0.85	Judge	1	1.47
Infer	1	0.85	Value	1	1.47
Interpret	1	0.85
Label	1	0.85
Model	1	0.85
Plan	1	0.85
Point	1	0.85

Once again, there is greater diversity in the learning standards designed by the primary students, although it is clear that in both stages the lesson plannings are designed following a cognitive progression from low order thinking skills, the majority in both stages, particularly in pre-primary, to high order thinking skills.

Discussion and conclusions

The results of testing the hypotheses in the first axis on CLIL lesson planning reveal that both primary and pre-primary students improve their skills in this competence once the CLIL course is completed (H1). In addition, there are no significant differences between the two groups neither before nor after having completed the CLIL course (H2). This result supports the ability of teachers to implement the curricula under the CLIL approach, thus overcoming the difficulties pointed out by Guillén Díaz and Sanz Trigueros (2019), and satisfies the claim of Delicado and Pavón (2015), Palacios et al. (2018) and Estrada (2021) to balance linguistic and methodological training for the bilingual teaching of future teachers. It also satisfies the demand contemplated in the list of quality factors for bilingual education at university drawn up by Madrid and Julius (2017), who point out that training in L2 and didactic preparation in CLIL are two of the most valued aspects by teachers and students. Likewise, this result supports the studies by Palacios et al. (2018) and Torres Zúñiga and Carrasco Flores (2020) who highlight the need for guidelines from the administration and the institutions themselves to urge all higher education centers to offer bilingual education in the education degrees. This offer can stop the proliferation of specialized postgraduate studies in bilingual education and CLIL (Palacios et al., 2018; López-Hernández, 2021) and compensate for the lack of access to this training by all potential CLIL teachers, because, As López-Hernández (2021) points out, this indeterminacy opens a gap in the initial training of teachers between public and private universities.

With regard to the hypotheses of the second axis on communicative competence in English and CLIL (H3-H5), the fact that the level of English (H3) generates significant differences in dimension 1 on the fundamental principles of the approach emphasizes the importance of the integration of content and language both in the training of CLIL teachers and in their competence to plan lessons under this approach, as pointed out by Beacco et al. (2015) and Morton (2016). Studies such as the one by Pérez-Cañado (2016), about in-service CLIL teachers, also point to this direct relationship between teacher’s level of competence in the CLIL foreign language and their knowledge of the methodological principles of the approach. This result in H3 and the differences found in dimension 3 Resources, in the IV Perceived level of training in CLIL (H5), are in line with the demand of various authors to improve academic plans and attend not only to the improvement in the linguistic competence of the students, but jointly also to the improvement of their competence in the methodology of bilingual education (Delicado & Pavón, 2015; Estrada, 2021; Morton, 2016; Palacios et al., 2018).

On the other hand, the motivation factor of students towards CLIL, axis 3 (H6-H7), has not generated statistically significant differences in the competence to plan CLIL. Only the students who do their internships in bilingual schools (H8) significantly improve their competence in the dimensions of methodology and resources. This highlights the interest of linking academic learning with what really happens in the bilingual classroom, as a source of training in the bilingual classroom methodology described by Bolarín Martínez et al. (2019). In this sense, proposals such as that of Pérez Murillo and Steele (2017), which promote a collaboration model between the classrooms of bilingual schools and university classrooms of future teachers, would allow all potential CLIL teachers to learn about the reality of these bilingual schools from their in-service teachers, regardless of the type of school placement in which they do their internship.

The results of the qualitative content analysis of the CLIL lesson plannings of both groups show an adequate integration of the methodological principles of the approach, thereby endorsing the results of H2. From the point of view of the selection of curricular areas, both groups tend to select content from the areas of science, which is in line with the guidelines of the curricular regulations that suggest them as the more convenient areas for the teaching of CLIL due to their load in the timetable (Community of Madrid, 2020). Likewise, both groups show adequate competence when designing learning standards following a progression of cognitive demand from the lower to the higher order skills. However, the differences in the selection of textual genres between both groups are due to the limitations in the language of pre-primary students (basically receptive skills) and the need to link these genres to specific adequate tasks for this stage, as suggested by Piccardo et al. (2019). This shows that the CLIL course provides students with the necessary strategies to adapt the methodological principles of the approach to the idiosyncrasies of their respective stages.

Despite the limitations pointed out in this case study, it can be concluded that initial CLIL training such as the one described in this paper favors the development of some of the most important competencies described in CLIL teacher training frameworks (Marsh et al., 2010; Bertaux et al., 2010; Pavón & Ellison, 2013; Madrid & Madrid, 2014; Pérez-Cañado, 2015): the competencies to design teaching materials and lesson plannings based on the contents of the official curriculum in a foreign language (Objective 1). As claimed by Torres Zúñiga and Carrasco Flores (2020), the students of both groups have shown that they are able to design CLIL lesson plannings to promote the development of multilingual and multicultural skills through the integration of final tasks linked to textual genres adequate to the curricular areas (Objective 2). In addition, the lesson plannings of both groups follow an adequate cognitive progression. All this favors a language learning model such as the one promoted in the CEFR (Council of Europe, 2001; 2020) and guarantees that all potential CLIL teachers, regardless of the educational stage, have a certain degree of competence in the CLIL approach (Objectives 3 and 4).

As a prospective of this study, it is proposed to analyze the competence to plan CLIL in Education degrees with and without the specialization in English and/or specific CLIL subjects in order to confirm these results. It is also possible to go deeper in the analysis of students’ CLIL lesson plannings so as to gain an in-depth understanding of the treatment of text types and improve their reliability by including at least one more researcher in the qualitative analysis. Other alternatives to improve the training of CLIL teachers, such as the one proposed by Banegas and del Pozo Beamud (2020), suggest “encouraging CLIL teacher educators to investigate their practices in different settings through (auto)ethnography to present detailed descriptions and honest accounts of the challenges, successes and failures in CLIL research” (p. 11). This critical view of teacher training is key to improving the academic programs that are at the basis for developing the key competences of the future CLIL teacher.

References

Banegas, D. L., & del Pozo Beamud, M. (2020). Content and Language Integrated Learning: A Duoethnographic Study about CLIL Pre-Service Teacher Education in Argentina and Spain. RELC Journal. https://doi.org/10.1177/0033688220930442

Beacco, J. C., Byram, M., Cavalli, M., Coste, D., Cuenat, M. E., Goullier, F., & Panthier, J. (2016). Guide for the development and implementation of curricula for plurilingual and intercultural education. Council of Europe.

Beacco, J.C., Fleming, M., Goullier, F. Thürmann, E., & Vollmer, H. (2015). The Language Dimension in all Subjects. A Handbook for Curriculum Development and Teacher Training. Council of Europe. www.coe.int/t/dg4/linguistic/Source/Handbook-Scol_final_EN.pdf

Bertaux, P., Coonan, C. M., Frigols, M. J., & Mehisto, P. (2010). The CLIL Teacher’s Competence Grid. http://lendtrento.eu/convegno/files/mehisto.pdf

Bolarín Martínez, M. J., Porto Currás, M., & Lova Mellado, M. (2019). Implementarion of bilingual programs in primary education: teaching and evaluation strategies. Elia: Estudios de Lingüística Inglesa Aplicada, 1 Monográfico, 207-234.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Second Edition. LEA.

Comunidad de Madrid (2020). Madrid comunidad bilingüe 2020-2021. Consejería de Educación, Universidades, Ciencia y Portavocía.

Cortina-Pérez, B., & Pino Rodríguez, A. M. (2021). Analysing CLIL teacher competences in pre-service preschool education. A case study at the University of Granada. European Journal of Teacher Education, 1-19. https://doi.org/10.1080/02619768.2021.1890021

Comisión Europea (1995). Libro Blanco sobre la educación y la formación: “Enseñar y aprender. Hacia la sociedad cognitiva”. https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:1995:0590:FIN:ES:PDF

Consejo de Europa. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.

Consejo de Europa. (2020). Common European framework of reference for languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing. www.coe.int/lang-cefr.

Custodio-Espinar, M. (2018). Assessment grid for the CLIL lesson plan [Class handout]. Universidad Pontificia Comillas, E000004435.

Custodio-Espinar, M. (2019a). CLIL teacher education in Spain. In K. Tsuchiya & M.D. Pérez Murillo (Eds.), Content and language integrated learning in Spanish and Japanese contexts (pp. 313-337). Palgrave Macmillan, Cham.

Custodio-Espinar, M. (2019b). Los principios metodológicos AICLE (aprendizaje integrado de contenido y lengua). Fundación Universitaria Española.

Custodio-Espinar, M., & García Ramos, J.M. (2020). Medida de la competencia para programar AICLE y diagnóstico de las necesidades de formación docente. Bordón. Revista de Pedagogía, 72(1), 31-48. https://doi.org/10.13042/Bordon.2019.72250

Delicado Puerto, G., & Pavón Vázquez, V. (2015). Training primary student teachers for CLIL: innovation through collaboration. Pulso, 39, 35–57.

Estrada, J. L. (2021). Diagnóstico de necesidades formativas entre maestros AICLE en formación inicial. European Journal of Child Development, Education and Psychopathology, 9(1), 1-16. https://doi.org/10.32457/ejpad.v9i1.1402

Guillén Díaz, C., & Sanz Trigueros, F. J. (2019). Un dispositivo para una relación pragmática de los docentes con los objetivos de programas bilingües, desde un estudio en documentos de planificación. PUBLICACIONES, 49(5), 113-130.

Jover, G., Fleta, T., & González, R. (2016). La formación inicial de los maestros de educación primaria en el contexto de la enseñanza bilingüe en lengua extranjera. Bordón, Revista de Pedagogía, 68(2), 121–135.

López-Hernández, A. (2021). Initial teacher education of primary English and CLIL teachers: An analysis of the training curricula in the universities of the Madrid Autonomous Community (Spain). International Journal of Learning, Teaching and Educational Research, 20(3), 132-150. https://doi.org/10.26803/ijlter.20.3.9

Madrid Fernández, D., & Julius, S. M. (2017). Quality Factors in Bilingual Education at the University Level. Porta Linguarum: revista internacional de didáctica de las lenguas extranjeras, (28), 49-66.

Madrid Manrique, M., & Madrid Fernández, D. (2014). La formación inicial del profesorado para la educación bilingüe. Universidad de Granada.

Marsh, D., Mehisto, P., Wolff, D., & Frigols, M. J. (2010). European Framework for CLIL Teacher Education: A framework for the professional development of CLIL teachers. European Centre for Modern Languages.

Morton, T. (2016). Conceptualizing and investigating teachers’ knowledge for integrating content and language in content-based instruction. Journal of Immersion and Content-Based Language Education, 4(2), 144–167. https://doi.org/10.1075/jicb.4.2.01mor

MECD (2003). La integración del sistema universitario español en el espacio europeo de enseñanza superior. Documento-Marco. Ministerio de Educación, Cultura y Deporte.

Palacios, F. J., Gómez, M. E., & Huertas, C. A. (2018). Formación inicial del docente AICLE en España: Retos y claves. Estudios Franco-Alemanes, 10, 141-161.

Pavón Vázquez, V., & Ellison, M. (2013). Examining teacher roles and competences in content and language integrated learning (CLIL). Linguarum Arena, 4, 65–78.

Pérez-Cañado, M. L. (2015). Training teachers for plurilingual education: A Spanish case study. In D. Marsh, M. L. Pérez-Cañado & J. Ráez Padilla (Eds.), CLIL in Action: Voices from the Classroom (pp. 165–187). Cambridge Scholars Publishing.

Pérez-Cañado, M. L. (2016). Teacher training needs for bilingual education: in-service teacher perceptions. International Journal of Bilingual Education and Bilingualism, 19(3), 266–295. https://doi.org/10.1080/13670050.2014.980778

Pérez Murillo, M. D., & Steele, A. (2017). Initial Teacher Education for CLIL at Primary Education Level in Madrid: Key issues and challenges. In M. E. Gómez Parra, & R. Johnstone (Eds), Bilingual education: Educational trends and key Concepts. Spanish Ministry of Education.

Piccardo, E., North, B., & Goodier, T. (2019). Broadening the scope of language education: Mediation, plurilingualism, and collaborative learning: The CEFR companion volume. Journal of e-Learning and Knowledge Society, 15(1), 17-36. https://doi.org/10.20368/1971-8829/1612

Real Decreto 95/2022, de 1 de febrero, por el que se establece la ordenación y las enseñanzas mínimas de la Educación Infantil. Boletín Oficial del Estado, 28, de 2 de febrero de 2022. https://www.boe.es/buscar/act.php?id=BOE-A-2022-1654

Sierra Macarrón, L., & López Hernández, A. (2015). CLIL en la formación inicial del profesorado de Educación Infantil y Primaria: la experiencia del CES Don Bosco. Educación y futuro: revista de investigación aplicada y experiencias educativas, 32, 83-116.

Torres Zúñiga, L., & Carrasco Flores, J. A. (2020). La enseñanza del inglés en educación infantil en España: Implicaciones para la formación del profesorado. Docencia e Investigación: revista de la Escuela Universitaria de Magisterio de Toledo, 45(31), 5-23.

Zayas, F., & Romero, E. (2017). Challenges and opportunities of training teachers for plurilingual education. En J. Valcke & R. Wilkinson (Eds.), Integrating Content and Language in Higher Education. Perspectives on Professional Practice (pp. 1-10). Frankfurt am Main: Peter Lang.

Contact address: Magdalena Custodio Espinar, Universidad Pontificia Comillas, Instituto de Idiomas Modernos, postal code 28049, Madrid. E-mail: mcustodio@comillas.edu

APPENDICES

Questionnaire to measure the integration of CLIL methodological principles (Custodio-Espinar & García Ramos, 2020, pp. 15-16)

DIMENSION 1 CLIL CORE ELEMENTS
SUBDIMENSION 1.1 The language of CLIL
1. Do you regularly program activities to reinforce the grammatical structures required by the contents?	*1		2	3		4		5	6
2. Do you include English language assessment activities in your CLIL lessons?	1		2	3		4		5	6
3. Do you usually give your students oral and written comprehension activities of the texts in that order?	1		2	3		4		5	6
4. To what extent do you develop strategies for learning content-related vocabulary?	1		2	3		4		5	6
5. Do you develop resources to support the linguistic demands of the content you teach?	1		2	3		4		5	6
SUBDIMENSION 1.2 Integration of the 4Cs
6. Do you analyze the level of cognitive difficulty of the activities to adapt it, if necessary, to the level of competence of your students?	1		2	3		4		5	6
7. Do you adapt the oral and written texts to the level of linguistic knowledge of your students?	1		2	3		4		5	6
8. To what degree are your activities related to the curricular contents of the level / area you teach?	1		2	3		4		5	6
9. Do you use scaffolding strategies to encourage and guide interaction in the classroom?	1		2	3		4		5	6
10. Do you think that your activities are motivating and relevant for your students and allow them to create a final result that they can show and / or share?	1		2	3		4		5	6
DIMENSION 2 METHODOLOGY
SUBDIMENSION 2.1 Attention to diversity
11. Do you indicate in any way the relationship between activities and skills that students develop in your programming?		1	2	3		4		5	6
12. Do you carry out specific activities to know the learning styles of your students?		1	2	3		4		5	6
13. Do you use any cognitive taxonomy to define the learning standards that you use to evaluate the learning goals of your lesson plans?		1	2	3		4		5	6
14. Do you usually program different activities on the same content to adapt to the different levels of competence of your students?		1	2	3		4		5	6
SUBDIMENSION 2.2 Methodological strategies
15. To what degree do you include group or pair work strategies?		1	2	3		4		5	6
16. To what extent do your lesson plans include problem-solving strategies, discovery / project learning, etc.?		1	2	3		4		5	6
17. Do you include activities for self-assessment and co-assessment of students for the evaluation of the curricular contents?		1	2	3		4		5	6
18. In addition to exams and tests, do you regularly use assessment tools such as observation sheets, checklists, rubrics or similar to assess your students?		1	2	3		4		5	6
19. Do you provide strategies to clarify and help students reach conclusions on their own?		1	2	3		4		5	6
DIMENSION 3 RESOURCE
20. Do you include didactic materials that reproduce or are resources from real life when carrying out tasks?		1	2	3	4		5		6
21. Do you use ICT as a resource to promote interaction and autonomous learning?		1	2	3	4		5		6
DIMENSION 4 EVALUATION
22. To what extent do you simplify and / or reduce the content of the curricular areas when teaching it in English?		1	2	3		4		5	6
23. To what degree do you combine formative assessment strategies (to provide feedback and help students) and summative assessment (to mark them)?		1	2	3		4		5	6
GENERAL CONSIDERATIONS
24. Do you think you plan your teaching?		1	2	3		4		5	6
25. Do you think that good teaching planning influences student learning?		1	2	3		4		5	6
26. Do you think that your knowledge of the CLIL principles allows you to carry out lesson plans appropriate to the requirements of the CLIL approach?		1	2	3		4		5	6

* 1 Never 2 Rarely 3 Sometimes 4 Frequently 5 Almost always 6 Always

Summary of CIPMA’s psychometric data (Custodio-Espinar & García Ramos, 2020, pp. 5-10)

Expert validation

Kendal’s coefficient of concordance among the 15 judges, with respect to the three measured variables (logical order, number of items and content validity), shows a degree of agreement close to 0.5. In the analysis of the “doctors” and “non-doctors” variable, the degree of agreement is greater among the non-doctors (five judges) than among the doctors (ten judges) with a w value of 0.47 and 0.6, respectively.

After this analysis, the variables have been organized into five dimensions, each with two subdimensions (Table V).

TABLE V. Revised structure of the questionnaire after validation by experts

Dimension	Subdimension	N of items
D1 CLIL core elements	Integration of the 4Cs Language treatment	5 4
D2 Methodology	Methodological strategies Attention to diversity	5 3
D3 Activities and resources	Activities Resources	3 3
D4 Evaluation	Language and content evaluation Learning process evaluation	2 2
D5 Organization	Timetable Space	2 2
Total number of item-variables		31
Total number of demographic variables		14
Total number of criterion items		3
Total number of items		48

Factor validity

Four factor analyzes were performed, using a combination of Principal Components and Maximum Likelihood methods and Varimax and Oblimin rotation models. The fourth Factor Analysis of Principal Components and Oblimin rotation of the matrix of 27 original variable-items (once the items of dimension 5 were eliminated due to reliability problems, most likely due to the fact that this dimension measures aspects of the organization of schedules and classrooms that do not depend on the teacher) was the most adjusted in its structure (Table VI).

TABLE VI. Factor matrix of 27 variables rotated by direct Oblimin criterion. Kaiser normalization

Variables	Factors
	1	2	3	4	5	6
D2/28. Relationship between activities and skills in programming	0.724
D2/30. Activities to learn about learning styles	0.720
D2/29. Use of a cognitive taxonomy to define evaluation criteria	0.709
D2/31. Activities to suit different levels of proficiency	0.579
D1/23. Reinforcement of grammatical structures of the content		-0.791
D4/38. English language assessment activities in programming		-0.711
D1/21. Oral and written comprehension activities		-0.671
D1/22. Content vocabulary development		-0.501
D3/35. Preparation of resources to support language development		-0.434
D1/17. Analysis of cognitive demands			0.662
D1/20. Adaptation of oral and written texts			0.596
D3/32. Relationship between activities and curricular objectives			0.526
D1/18. Strategies to foster interaction			0.516
D3/33. Motivating activities, relevant to the final product			0.436
D1/16. Simplification/reduction of content				0.701
D4/40. Formative and summative assessment strategies				0.436
D2/25. Group/pair work strategies					0.789
D2/26. Development of learning by discovery					0.690
D4/39. Programming of self-assessment and co-assessment activities					0.490
D4/41. Use of formative assessment tools					0.489
D2/27. Support them to come to conclusions on their own					0.480
D3/37. Use of ICT as a resource for interaction and autonomous learning						0.793
D3/36. Use of real life resources						0.594

This fourth exploratory factor analysis presents a distribution of factor loads that adequately reflects the four main dimensions of the original questionnaire (Table VII).

TABLE VII. Latent structure of the 27 variables after the fourth factor analysis

Factor	Items	N of variables*
1. Methodology: Attention to diversity	28, 30, 29, 31	4
2. CLIL elements: Language treatment	23, 38, 21, 22, 35	5
3. CLIL elements: Integration of 4Cs	17, 20, 32, 18, 33	5
4. Evaluation	16, 40	2
5. Methodology: Methodological strategies	25, 26, 39, 41, 27	5
6. Resources	37,36	2

* The excluded items showed non-significant loadings (15, 19, 24, 34).

Table VIII shows the revised structure of the questionnaire after the fourth Factor Analysis.

TABLE VIII. CIPMA dimensions y subdimensions.

Dimension	Subdimension	Items	N of items
D1 CLIL core elements	Language treatment Integration of 4Cs	23, 38, 21, 22, 35 17, 20, 32, 18, 33	5 5
D2 Methodology	Attention to diversity Methodological strategies	28, 30, 29, 31 25, 26, 39, 41, 27	4 5
D3 Resources	Materials and ICT resources	36, 37	2
D4 Evaluation	Content Assessment Evaluation Strategies	16, 40	2
Total number of item-variables			23
Total number of demographic variables			16
Total number of criterion items			3
Total number of items			42

Reliability analysis

Cronbach’s α coefficient was used to determine the reliability as internal consistency of the questionnaire with the complete sample of 383 teachers. The reliability analysis of the latent structure, made up of 23 items distributed in six factors, which has resulted from the fourth Factor Analysis, shows a high and satisfactory degree of reliability of the total scale, with an α value equal to 0.86.

Criterial validity

The results are shown in Table IX (Pearson-type correlations).

TABLE IX. Convergent criterial validity of the six factors and the total sum of factors

Factor	Corr. CI1	Corr. CI2	Corr. CI3
1. Metodología: Atención a la diversidad	0.366**	0.133**	0.456**
2. Elementos AICLE: Tratamiento del lenguaje	0.235**	0.069	0.428**
3. Elementos AICLE: Integración de las 4 Ces	0.374**	0.142**	0.345**
4. Evaluación	0.224**	0.122*	0.206**
5. Metodología: Estrategias metodológicas	0.349**	0.096	0.414**
6. Recursos	0.320**	0.104*	0.293**
Suma de los seis factores	0.458**	0.156**	0.557**

*p < 0.05 y ** p < 0.01

The results show that there is a statistically significant linear correlation between the total score of the sum of the six factors and the three criterion-items, the correlations are higher with criterion-items 1 and 3. By factors, the correlations are also significant with criteria items 1 and 3, in all cases. With item-criterion 2, factors 2 and 5 do not show significant values.

CLIL lesson plan template (Custodio-Espinar, 2019b, pp. 493-495)

LESSON PLAN: title
CONTENT AREA: Level: Timing: Description (aim): Final product:
CONTENT
Content (subject content from the official curriculum). Include language content too! • CONTENT (curricular content) • LANGUAGE CONTENT (genre)			Contribution to key competences Learning strategies
COGNITION
Learning goals 1. Declarative knowledge 2. Procedural knowledge 3. Metacognition 4. Language			Learning outcomes or standards (cognitive skills that are put into practice from least to greatest demand)
CULTURE
Learning goals (idem)			Learning outcomes (standards) (idem)
COMMUNICATION Coyle, Hood and Marsh (2010)
Language of learning		Language for learning		Language through learning
CONTENT-OBLIGATORY LANGUAGE Key language: (language specific to content) Language content (the genre): e.g., genre/recount = time connectors ‘when, later, after...’ or time expressions to recount ‘used to’. Academic language: (linkers)		CONTENT-COMPATIBLE LANGUAGE Speech acts related to content The language necessary to develop the activities you are planning to learn the content Classroom language		New areas of meaning connected to the knowledge
PROCEDURE
Timing	Activities (T/ S role)				Grouping/spaces
SESSIONS/Time planned for each activity, in minutes RECEPTION (INPUT) TRANSFORMATION (INPUT) PRODUCTION SCAFFOLDING (OUTPUT)	What the teacher (T), language assistant (LA) and students (Ss) are expected to do.				How you expect Ss to interact? What are the criteria for choosing groups? Examples: Whole class, pairs, groups of 3, etc.
Materials: (Materials that will be used by teacher and students, including SCAFFOLDING, ICT tools, arts materials, dictionaries, etc., e.g., worksheets, links to videos, audios or websites, printout of MS Ppt slides, sample posters or visual aids, etc.)
Evaluation criteria (from the official curriculum) • To... Assessment tools: (Assessment tools that will be used to assess students. Include formative and summative assessment tools to be used by the teacher, the students and/or both). • Of language: (for interaction, for self/peer-assessment, ...) • Of content: (rubric, checklist, tests, ...) Others: a checklist for active observation/teacher’s self-evaluation, etc.
ATTENTION TO DIVERSITY (Use a taxonomy to plan some of the activities at lower and higher order cognitive level) LOTS HOTS

Evaluation criteria of the CLIL lesson plan task
(Custodio-Espinar, 2018)

STUDENT
TASK	A CLIL LESSON PLAN
ASSESSMENT	Criteria	MAXIMUM SCORE	STUDENT SCORE
CONTENT	The student knows how to integrate content and language in a CLIL lesson. • Content (form the curriculum) • Cognition (cognitive demand) • Communication (language demands analysis including language content) • Culture	3 points
	The student is able to plan and develop an integrated assessment strategy including formative assessment tools (checklists and rubrics). • Connection with learning goals and outcomes (cognitive demand) • Formative/Summative assessment tools	3 points
PROCESS	The student is able to illustrate understanding of the CLIL approach through the analysis of different resources and materials to teach curricular content in different subjects to primary students. • Activities • Scaffolding • ICT resources	3 points
	The student is able to develop team teaching awareness and competence to improve the teaching/learning process in the bilingual classroom. (FROM TEAM TEACHING RUBRIC)	1 point
LANGUAGE	-0.05 points /grammar or spelling mistake -0.10 points/sense mistake	- 0,0
FINAL MARK		10 points
COMMENTS: Group work planning: Individual planning: