Measuring the importance of pedagogical leadership
according to the stakeholders’ perception1

Medición de la importancia del liderazgo pedagógico de acuerdo con la percepción de los evaluadores

DOI:10.4438/1988-592X-RE-2021-394-500

Raúl González-Fernández

Ernesto López-Gómez

Universidad Nacional de Educación a Distancia (UNED)

Buratin Khampirat

Suranaree University of Technology (Thailand)

Samuel Gento

Universidad Nacional de Educación a Distancia (UNED)

Abstract

There are numerous studies on pedagogical leadership, but there is a lack of instruments for measuring pedagogical leadership based on the contextual and cultural characteristics of the Spanish educational system. The purposes of this study were to examine the factor structure of the Pedagogical Leadership Scale (PLEADS), to test the measurement invariance properties of the factor structure of the PLEADS across groups of principals, directing team members, and teachers; and to study the effect of the types of assessors and assesses on the determination of the importance of leadership. This study collected data from 2,107 stakeholders. Of these, 62.32% were females, and 37.64% were males (0.04% didn´t answer). The stakeholders assessed the importance of pedagogical leadership for 729 principals, 330 other directing team members, and 1,048 teachers. Multigroup confirmatory factorial analysis is performed to test the invariance of the factor structure. Multiple linear regression analyses were employed to assess the influence of the type of leader and stakeholder on the leadership score. Participants assessed pedagogical leadership of the three groups using a single factor structure. The results supported that PLEADS can be well applied to assess the perception of the importance of pedagogical leadership by stakeholders, and cross-group comparisons of the PLEADS can be made. These findings provide evidence of the validity, reliability, and invariance of the PLEADS in Spain, confirming that it can be applied in other countries.

Key words: Pedagogical leadership, stakeholder perspective, confirmatory factor analysis, measurement invariance, psychometrics, Spanish context.

Resumen

Existen numerosos estudios sobre liderazgo pedagógico, pero falta un instrumento para medir el liderazgo pedagógico en función de las características contextuales y culturales del sistema educativo español. Los propósitos de este estudio fueron examinar la estructura factorial de la Escala de Liderazgo Pedagógico (PLEADS) entre grupos de directores, miembros del equipo directivo y profesorado; y estudiar el efecto de los tipos de evaluadores y evaluados sobre la determinación de la importancia de liderazgo. Este estudio recopiló datos de 2.107 evaluadores, el 62.32% eran mujeres y el 37.64% eran hombres (0.04% no contestó). Evaluaron la importancia del liderazgo pedagógico para 729 directores, 330 otros miembros del equipo directivo y 1.048 maestros. Se realiza un análisis factorial confirmatorio multigrupo para probar la invariancia de la estructura factorial. Se emplearon análisis de regresión lineal múltiple para evaluar la influencia del tipo de evaluador y líder en la puntuación de liderazgo. Los resultados respaldaron que PLEADS se puede aplicar bien a la evaluación de la percepción de la importancia del liderazgo pedagógico por parte de las partes interesadas y se pueden hacer comparaciones entre grupos de medias en el PLEADS. Estos hallazgos evidencian la validez, confiabilidad e invarianza de la PLEADS en España, confirmando que se puede aplicar en otros países.

Palabras clave: Liderazgo pedagógico, partes interesadas, análisis factorial confirmatorio, invarianza de medición, comparación entre grupo, contexto español.

Introduction

Educational leadership plays a central role in the success of the sustainable development of a quality education system and student learning outcomes (Bush, 2015; Hallinger, 2019). Each process of school improvement strategies is a complex task (Duke et al., 2013; Holmes et al., 2013) that demands very effective and highly talented leadership that is very capable of communicating needs and addressing the issues of school problems promptly in the most appropriate way (Abbas & Asghar, 2010). Effective education leadership is, therefore, a global policy because it is the basis for strongly determining the direction and success of society both at the micro level of the schools and communities or the broader level (OECD, 2013; Vaillant, 2015).

Since student achievement in the 21st century has diverse dimensions, effective educational organizations for sustainable growth must be related to various leadership behaviours with greater responsibility that are specific to each area in order to achieve goals and outcomes (Male & Palaiologou, 2013). “Pedagogical leadership” is therefore essential in the improvement of the education system (Gergen & Hersted, 2016) to achieve the effective development of learning and teaching from within the organization. The mission of a pedagogical leader will enable the teaching staff to become more involved in working towards common goals related to improving the teaching quality and learning processes of students (Shaked & Schechter, 2016). This highlights the particularity of leadership that demonstrates the process of intentional influence for achieving the intended results by handling and inspiring others (González-Fernández et al., 2016; Hallinger & Heck, 2010; López-Gómez & González-Fernández, 2018).

Because pedagogical leadership has several formal actions (Bush, 2016; Pont, Nusche, & Hopkins, 2008) used to perform complex tasks (Holmes et al., 2013) that must be adapted to meet the different needs and limitations of each school context (Hallinger, 2016), formal leaders (e.g., principals) cannot single-handedly lead in an era with high levels of demands and accountability (Gunter et al., 2013; Spillane, 2005). Organizational effectiveness in the challenging environment of educational institutions requires the integration of a variety of skills from different perspectives, and areas of expertise, as well as the ability to manage a variety of stakeholder groups (Gunter et al., 2013; Vilkinas, 2009). Therefore, pedagogical leadership’s roles should be developed as a shared responsibility from the collaboration between the representative leaders at different levels in each role’s teamwork and distributed leadership (Bush & Glover, 2014; Tian et al., 2015), such as the principal (Day et al., 2010), other directing team members or the middle leaders (Harris et al., 2019), and teacher (Wenner & Campbell, 2016).

In the context of Spain, although there are numerous researches on pedagogical leadership (Bolívar et al., 2013), there are still limitations, especially in two important areas. First, there is a lack of instruments for measuring pedagogical leadership practices and behaviours that were developed based on the contextual and cultural characteristics and underlying values of the leaders of the Spanish educational system. Second, most of the questionnaires used in the nation were adapted from existing measurement tools from other contexts (e.g. Alvarez et al., 2018; Bolívar et al., 2017;García-Garnica, 2016; Lopez-Zafra et al., 2012; Pérez-García et al., 2018).Therefore, Gento (2002) developed a set of dimensions and a questionnaire for measuring pedagogical leadership based on the relevant theories and previous studies that is more appropriate for the context and culture of educational organizations in Spain. This instrument consists of eight dimensions covering leadership aspects in the context of Spanish educational organizations, namely, the charismatic, emotional, anticipatory, professional, participative, cultural, formative and administrative dimensions (Gento et al., 2015).

A review of the literature by Antoniou and Lu (2017) showed that numerous conceptual frameworks and indicators of functions of pedagogical leadership have been proposed, including methodological progress on the development and validation of instruments (e.g., multilevel analysis by McCarley et al. (2016), confirmatory factor analysis (CFA) by Antoniou and Lu (2017) and Oterkiil and Ertesvåg (2014), and invariance analysis by Hallinger and Lee (2014). However, the limitation found is that the number of studies testing the measurement properties of perceived pedagogical leadership in distributed leaders is limited.

A reliable, valid, and recognized measurement instrument is important to explore pedagogical leadership competencies. Therefore, testing the reliability and validity of survey instruments is a significant technique in the primary step in evaluating the quality of an instrument and reducing measurement errors (Carmines & Zeller, 1979; Pop & Khampirat, 2019). In addition, understanding the quality of instruments measuring leaders at all educational process stages according to the perceptions of the stakeholders and the results of the measurement invariance of the scale are extremely important. The findings can provide insights to administrators, policymakers, teachers, and educators that can be used in the design of recruitment processes and training pedagogical leadership by focusing interventions to improve the quality of leadership skills and build potential leadership for sustainable development (Pont, Nusche, & Moorman, 2008). Pedagogical leaders who possess strong leadership competencies use effective leadership strategies more in their duties while thoroughly creating methods to achieve high levels of learning and teaching quality (Male & Palaiologou, 2013; Wang et al., 2016). Besides, because the process of determining the importance of the assessment indicators to the leader by stakeholders is important, this research will also present a systematic approach for evaluating pedagogical leadership.

Purposes of the study

The purposes of this study were to (1) examine the factor structure of the PLEADS; (2) test the measurement invariance properties of the factor structure of the PLEADS across groups of principals, directing team members, and teachers; and (3) study the effect of the types of assessors and assesses on the determination of the importance of leadership.

Methods

Participants

According to the multiple assessor concept of 360-degree evaluation (Eichinger & Lombardo, 2003), the important of leadership should be evaluated by stakeholders who had experience with leader. The study participants were representative of the target population, as they were drawn from 18 regions in Spain that (a) were geographically distributed in the main regions of Spain; (b) represented different types of schools; and (c) there were various stakeholders that were from the diverse demographic background.

This study collected data from 2,107 assessors who were stakeholders and had experience with pedagogical leaders. Of these, 62.32% were females, and 37.64% were males (0.04% didn´t answer). The majority of the stakeholders were senior teachers (1,149, 54.53%), students (309, 14.67%), heads/principals (273, 12.96%), and students’ mothers/fathers (216, 10.25%). They assessed the importance of pedagogical leadership for 729 principals, 330 other directing team members, and 1,048 teachers who work in public (1,705, 80.92%), subsidized private (336, 15.95%), and nonsubsidized private (60, 2.85%) educational institutions in Spain. The demographic profiles both assessees and assessors are displayed in TABLE I.

TABLE I. Demographics Background of Assessees and Assessors.

Demographics	Type of Leader
	Principal		Other Directing Team member		Teacher		Total
	N	%	N	%	N	%	N	%
Leaders (Assessees)	729	34.60	330	15.66	1,048	49.74	2,107	100.00
Type of School
Public	599	82.17	241	73.03	865	82.54	1,705	80.92
Pivate aided (subsidized)	106	14.54	72	21.82	158	15.08	336	15.95
Private non aided (unsubsidized)	24	3.29	15	4.55	21	2.00	60	2.85
Don’t know	-	-	2	0.61	4	0.38	6	0.28
Total	729	100.00	330	100.00	1,048	100.00	2,107	100.00
Stakeholders (Assessors)
Gender
Female	439	60.22	204	61.82	670	63.99	1,313	62.32
Male	290	39.78	126	38.18	377	36.01	793	37.64
Don’t know	-	-	-	-	1	0.10	1	0.04
Total	729	100.00	330	100.00	1,048	100.00	2,107	100.00
Position
Student	99	13.58	46	13.94	164	15.65	309	14.67
Teacher	343	47.05	157	47.58	649	61.93	1,149	54.53
Head/Principal	140	19.20	36	10.91	97	9.26	273	12.96
Student´s Mother/Father	113	15.50	21	6.36	82	7.82	216	10.25
School Inspector/Supervisor	2	0.27	4	1.21	3	0.29	9	0.43
Teacher’s trainer	4	0.55	13	3.94	14	1.34	31	1.47
Other	17	2.33	48	14.55	36	3.44	101	4.79
Don’t know	11	1.51	5	1.52	3	0.29	19	0.90
Total	729	100.00	330	100.00	1,048	100.00	2,107	100.00

Measures

The questionnaire that was used to measure the importance of pedagogical leadership was developed by Gento et al. (2015) based on theoretical background and empirical data regarding the quality of pedagogical leadership. The questionnaire consisted of eight dimensions, as shown in TABLE II. Each dimension consisted of 10 items, and scoring was performed on a 9-point rating scale where “1” means “not important” and “9” means “very important” (Gento et al., 2015; González-Fernández et al., 2016). The Cronbach’s alphas (α) for the scales ranged from .85 to .94 (TABLE III), which exceeded the good and excellent value in terms of internal consistency for all groups (Nunnally, 1978).

TABLE II. Eight dimensions of pedagogical leadership instrument and their meaning.

Dimensions		Meaning
1	Charismatic	The leader (whether an individual or team) is attractive enough on a personal level to enable other people to feel comfortable, and is approachable enough to inspire other people to feel confident about having a close professional relationship.
2	Emotional	A leader should treat everybody in the educational institution or related to it with the greatest kindness, consideration, and acknowledgement, and at the same time, be mindful of each person’s dignity and show esteem and appreciation to all people.
3	Anticipatory	The ability to predict the most suitable strategies and activities to solve future challenges or problems. It will also mean foreseeing the possible consequences or effects that may result from the solutions or decisions to be implemented.
4	Professional	A leader should provide the impulse for the institution or entity and for its members to attain the highest educational quality possible.
5	Participative	The best way of encouraging individuals and groups to engage in intelligent and collaborative work is to motivate them to offer their cooperative effort in projects they are committed to, and to participate in the decision-making process throughout every phase. Collected empirical data generally show that in schools of quality, all members of the institution work together and that its quality is increased if the educational system acts in coordination with educational institutions.
6	Cultural	Leaders should promote the consolidation of the institution’s particular culture or specific cultural profile. Pedagogical leaders should, therefore, act with the required commitment in order to clarify, consolidate, defend, and spread the institution’s cultural profile.
7	Formative	They should take responsibility for their own continuous training and formation, and promote continuous training of the people working with them. The basic approach of this leadership dimension must, then, be the promotion of personal professional training and encouragement to obtain the best qualifications in order to carry out the tasks necessary to improve the quality of education and of the institution.
8	Administrative	In order to achieve institutions of true quality, bureaucratic activities should be kept to a minimum, or at least, take second place to educational concerns. These activities cannot be totally eliminated, but it is desirable to simplify them and to ensure that they do not overshadow the fundamental aim of achieving educational institutions of quality.

Source: Gento et al. (2015).

Procedure

Data collection was performed in Spain, in several phases from June 2015 to October 2018. Participants were asked to voluntarily answer the paper questionnaire, and they could withdraw from the study at any time. In addition, they were formally informed that their responses would be kept confidential and anonymous. A total of 2,184 questionnaires were received. After investigating and checking the quality of all responses, of the 2,184 responses, 77 were excluded due to missing more than 10% of the data. Therefore 2,107 questionnaires were used in this study.

The inclusion criteria were stakeholders as assessors, who were willing to participate in the survey. The exclusion criteria were stakeholders (namely, Student, Teacher, Head/Principal, Student´s Mother/Father, School Inspector/Supervisor, Teacher’s trainer) who were not agree to participate in the study or returned incomplete questionnaires, with more than 10% missing values for a particular variable.

Data analysis

The descriptive statistics and Cronbach’s alpha coefficients (α) were calculated with SPSS version 18.0. Confirmatory factor analysis (CFA) was used to assess the factor structure of the PLEADS of each of the three groups using Mplus 8.3. When all three measurement models acceptably fit the data, their measurement invariance for group effects was analyzed based on a forward approach (Dimitrov, 2010; Vandenberg & Lance, 2000). First, configural invariance (invariance of the model form), which investigated participants from different groups, conceptualizes the construct model to answer the survey items in the same way (Vandenberg & Lance, 2000). If configural invariance was supported, then the constructs have the same pattern of free and fixed loadings across groups (Putnick & Bornstein, 2016). Second, factor loading (metric) invariance was investigated by constraining all factor loadings to be equal across groups. This model indicates that the strength of the relationship between the latent factor and its indicators is the same across groups and can be interpreted as valid coefficients (Bollen, 1989). Third, intercept (scalar) invariance was established, that is, the regression equations linking the indicators to their latent factor were constrained to be equal across groups. Finally, if intercept invariance was supported, residual invariances (strict or invariant uniqueness) were tested by constraining the item residuals to be equivalent across the different groups. If residual invariance is not met, then at least one item residual is different across the groups (Putnick & Bornstein, 2016).

Various fit indexes were employed to assess the model fit. A relative chi-square (χ2/df) of 3:1 or less indicates a good fit (Kline, 2011). The comparative fit index (CFI > .90), the Tucker-Lewis index (TLI > .90) (Hu & Bentler, 1999), and the root mean square error of approximation (RMSEA < .06) (Schreiber et al., 2006) were also employed. Finally, a standardized root mean square residual (SRMR) less than .08 indicates an acceptable fit (Hu & Bentler, 1999). To compare competing models to select the best model and make a trade-off between the model fit and model complexity, the Akaike information criterion (AIC) and Bayesian information criterion (BIC) were used (Schoot et al., 2012; Vrieze, 2012). A lower AIC or BIC indicates a better fit and complexity (Schoot et al., 2012).

In the last step, multiple linear regression analyses were conducted to assess the influence of the type of leader and stakeholder on the leadership score.

Results

Descriptive statistics

The mean scores (M) and standard deviations (SD) of eight dimensions for the PLEADS are given in TABLE III. Underlying the rating of stakeholders, the mean importance values in each group of leaders were mostly in the good range (M = 7.54 to 8.00). The absolute values of the skewness (SK) and kurtosis (KU) for all dimensions (TABLE III) are less than the thresholds of 3 (Chou & Bentler, 1995; Kline, 2011) and 10 (Kline, 2011), respectively, supporting that the sample data were assumed to be drawn from a population that has a univariate normal distribution.

TABLE III. Descriptive statistics in each group of leaders.

Dimensions	Principal					Other Directing Team Member					Teacher
	M	SD	SK	KU	α	M	SD	SK	KU	α	M	SD	SK	KU	α
1. Charismatic	7.98	0.84	-1.28	1.88	.86	7.87	0.91	-1.83	6.72	.85	7.85	0.94	-1.63	5.13	.85
2. Emotional	8.00	0.95	-1.49	3.06	.91	7.89	0.96	-1.84	7.77	.90	7.84	1.10	-1.91	6.58	.92
3. Anticipatory	7.94	0.94	-1.34	2.42	.91	7.88	1.02	-1.91	6.69	.91	7.78	1.05	-1.59	4.46	.92
4. Professional	7.94	0.95	-1.24	1.57	.91	7.89	0.94	-1.51	4.04	.90	7.79	1.09	-1.55	4.30	.91
5. Participative	7.97	0.95	-1.32	2.55	.93	7.89	0.94	-1.53	5.32	.91	7.80	1.12	-1.65	4.64	.94
6. Cultural	7.88	0.95	-1.20	1.85	.91	7.74	0.96	-1.55	5.88	.89	7.67	1.10	-1.44	3.88	.91
7. Formative	7.71	1.07	-1.00	0.96	.92	7.62	1.01	-1.14	2.95	.87	7.61	1.23	-1.30	2.05	.92
8. Administrative	7.94	1.01	-1.66	4.85	.91	7.76	1.05	-1.19	2.65	.91	7.54	1.25	-1.35	2.81	.90

Confirmatory factor analysis for each group

In this stage, the construct validity of each measurement model for the three groups was assessed by using CFA to confirm the single-factor structure of the PLEADS before assessing the measurement invariance. The first section of TABLE IV shows an overview of the goodness-of-fit indexes of the measurement models of the three groups. The results indicated that all models demonstrated an acceptable fit to the empirical data: χ2/df < 3, CFI and TLI > .90, RMSEA < .08, and SRMR < .06. Based on the AIC and BIC, the model with the lowest these values has the best fit, and the best fitting model to these data was the principal’s leadership measurement model. All standardized factor loadings were significant (p < .01), indicating that all eight dimensions contributed significantly to the measurement of pedagogical leadership in the three groups (FIGURE I).

TABLE IV. Summary of the fit indexes and tests of the invariance of the pedagogical leadership measurement model between principals, other directing team members, and teachers.

Model	χ2	df	χ2/df	p	CFI	TLI	RMSEA (90% CI)	SRMR	BIC	AIC	Decision
CFA for pedagogical leadership
- Principal model (N = 279)	22.17	11	2.02	.02	.99	.99	.04 (.01 - .06)	.01	11484.51	11334.18	Good Fit
- Other directing team member model (N = 330)	20.07	7	2.87	.01	.99	.98	.08 (.04 - .12)	.01	5081.202	4942.12	Good Fit
- Teacher model (N = 1,048)	13.95	7	1.99	.05	.99	.99	.03 (.00 - .05)	.01	18690.95	18509.00	Good Fit
Multigroup
Model 1: Configural invariance	148.80	52	2.86	.00	.99	.99	.05 (.04 - .06)	.06	32774.18	32323.36	Accepted
Model 2: Factor loading invariance (metric)	131.62	51	2.58	.00	.99	.99	.05 (.04 - .06)	.06	2764.63	32308.17	Accepted
Model 3: Factor loading and intercept invariances (scalar)	100.87	36	2.80	.00	.99	.99	.05 (.04 - .06)	.05	2848.42	32307.43	Accepted
Model 4: Factor loading, intercept, and residual invariances (unique)	478.08	57	8.39	.00	.97	.96	0.10 (.10 - .11)	.15	33065.28	32642.63	Rejected

FIGURE I. Results of the CFA of the pedagogical leadership measurement model (Mplus 8.3 standardized estimates).

Note: *= p < .05, and **= p < .01

Testing measurement invariance

Multigroup CFA was used to cross-validate the one-factor model across the groups of principals, other directing team members, and teachers; and the results are shown in the second part of TABLE IV.

Model 1 tested the configural invariance, and all parameters were freely estimated to allow differences in all groups to generate a baseline unconstrained model. This model fit the data (χ2 = 148.80, df = 52, χ2/df = 2.86, p = .00, CFI = .99, TLI = .99, RMSEA = .05(95% CI = .04 - .06), and SRMR = .06).

The factor loadings invariance in Model 2 had excellent fit indexes (χ2 = 131.62, df = 51, χ2/df = 2.58, p = .00, CFI = .99, TLI = .99, RMSEA = .05 (95% CI = .04 - .06), and SRMR = .06).

The intercept invariance (Model 3), which additionally imposed equality constraints on item intercepts across the groups, provided a slightly lower fit than Model 2 (χ2 = 100.87, df = 36, χ2/df = 2.80, p = .00, CFI = .99, TLI = .99, RMSEA = .05(95% CI = .04 - .06), and SRMR = .05).

In the final test of measurement invariance (Model 4), constraining the residual variance, the results did not fit the data (χ2 = 478.08, df = 57, χ2/df = 8.39, p = .00, CFI = .97, TLI = .96, RMSEA = .10(95% CI = .10 - .11), and SRMR = .15), rejecting residual invariance. This indicated that the residual variance for every item differed across the three groups. The results of the AIC and BIC for comparing models are presented in TABLE IV, showing that the factor loading invariance was the best-fitting model for these data.

The final model of scalar invariance is depicted in FIGURE II. The invariance results show that all indicators load on the pedagogical leadership construct with standardized factor loadings (range from .76 to 1.00) and intercepts (range from 7.98 to 8.00) that are equal across the three groups for all dimensions. The corresponding unconstrained residuals were in the 0.11 – 0.45 range for principals, 0.07 – 0.48 range for other directing team members, and 0.17 – 1.08 range for teachers.

FIGURE II. Results of the scalar invariance model across the principal, other directing team member, and teacher groups (Mplus 8.3 unstandardized estimates).

Note: *= p < .05, and **= p < .01

Assessing the influence of the assessors and type of leader on the leadership score

TABLE V shows the results of the multiple linear regression analyses for the leadership importance score. The independent variables were principal, male, and student (the other groups were controlled).

The results showed that principal (b = .09, p < .01) had a significant positive effect on the leadership score whereas male (b = -.04, p < .05) and student (b = -.28, p < .01) had significant negative effects. This finding implied that “principal” was assessed higher than “other directing team member” and “teacher” while the perception of male and student assessors in leadership was lower than other groups.

TABLE V. Results of the multiple linear regression analysis of the leadership score.

Independent Variable	b	b	t-test
(Constant)	7.89		275.37**
Leader (Assessee)
Principal (1= Principal, 0= Not principal)	0.17	.09	4.13**
Stakeholder (Assessor)
Male (1= Male, 0= Female)	-0.08	-.04	-2.11*
Student (1= Student, 0= Not student)	-0.73	-.28	-13.44**

Note: *= p < .05, and **= p < .01; b = unstandardized coefficient, and β = standardized coefficient.

Discussion

The study investigated the measurement invariance in the PLEADS across three groups of leaders according to the perceptions of their stakeholders. The preliminary results indicate that the eight dimensions of the PLEADS show satisfactory internal consistency and reliability.

The results from the multigroup measurement invariance supported the configural model, which confirms that a similar theoretical factor was present in the three groups. This means that stakeholders’ perceptions of the leadership of these three groups were similarly conceptualized as a single factor structure. Metric invariance was also supported and was the best-fitting model, indicating that stakeholders’ perceptions of the leaders from the three groups attributed the same meaning to the latent construct measured by pedagogical leadership. This suggests that pedagogical leadership can be compared across these groups because a one unit of change in one group equals a one unit of change in another group.

In the scalar invariance step, the results implied that the meaning of the construct (the factor loadings) and the levels of the underlying items (intercepts) are similar across the three groups. That is, respondents who have the same score on the latent construct would obtain the same score on the indicator (observed variable) regardless of their group membership (Milfont & Fischer, 2010). Consequently, multiple groups comparisons of their scores on the factor means can be applied (e.g. using t-test, ANOVA) (Schoot et al., 2012) because the difference in the group means are not due to the measurement properties.

Moreover, the results indicated that there is variance in the residuals; therefore, this study assessed partial residual invariance, but it was not achieved. This finding indicated that even though the PLEADS measured the same scale units across groups, at least one item residual is different across the three groups. This may be due to the common experience of different cultures and differences in understanding questions (Malpass, 1977) of the leadership assessors. In addition, Cheung and Rensvold (2002) and Mullen (1995) explained that residual inequivalence across groups occurs because respondents belonging to one group are unfamiliar with a scale and its scoring formats compared with the respondents from another group and therefore respond to leadership scales inconsistently.

The third purpose was to examine how the perceptions of the importance of leadership varied among the type of assessees (principals) and assessors (males and students). Principals were rated stronger in terms of their leadership importance after controlling for other types of leaders, such as the assessee, indicating that principals are more recognized as leaders and regarded as more important than others. This finding was consistent with previous research conducted in various contexts (González-Fernández et al., 2020; Lahtero et al., 2017; Tian & Virtanen, 2019) and may be the consequence of a formal pedagogical leadership. Principals play a key role in educational change due to their potential to positively influence the conditions in which educational plans, programmes and processes are implemented, as well as to promote the professional development of teachers and enhance student learning (Baptiste, 2019; Fullan, 2014). Principals must develop a positive collaborative climate and leader identity that is related to the school culture (Cruz-González et al., 2019). Furthermore, principals have a closer relationship and a formal line of communication (“as a visible leader”) with the evaluators compared to most teachers and middle leaders (Tintoré et al., 2020). This is reflected in a more favourable perception of the importance of principals’ leadership (González-Fernández et al., 2020).

Among the assessor characteristics, being male and a student were found to be significantly and negatively associated with the leadership score, which suggested that males and students assessed the importance of leadership lower. This finding leads us to propose two explanatory hypotheses for this result. The first is that students do not perceive that leadership is something important compared to the perceptions of other groups, probably because the relevance of leadership has not been shared with them in the educational institution, and they believe that leadership is an individualistic phenomenon far from the student role. The effects of leadership are hardly visible to students, especially if they are not made explicit. The second hypothesis is that multidimensional pedagogical leadership is perceived as more important by females because it is conceived from a more relational and interactive perspective, and this distances it from a traditional approach focused on the hierarchical role and power of the leader, perhaps most valued by males. These interpretations must be contrasted in future studies on gender bias in leadership perceptions.

Implications and recommendations for future research

Regarding the practical implications, researchers should consider the possibility of implementing measurement invariance testing in the first step of comparing the differences in the perceptions of groups to ensure that respondents from different groups conceptualize a concept using the same set of items of measurement instruments (Yap et al., 2014). Future studies might investigate the measurement invariance of the PLEADS in different countries or at different times because the interpretation of a leadership construct can change over time or across cultures (Putnick & Bornstein, 2016). Further research is needed in this area to determine why males and students rate the importance of leadership lower. Another essential issue for future studies is that qualitative research should be conducted to identify the equivalence concerned with how “concepts of pedagogical leadership are interpreted across stakeholders or cultures”.

Limitations

This study was not without limitations. Although the PLEADS was developed based on theory and the reliability and validity of the scale were supported in this study; however, it is possible that various measurement dimensions or items may be loaded on other factors that differ from those originally intended, and the theoretical structure used to create the scale may be measuring the similar or different overall concepts in different contexts. Therefore, this research was suggested that when this scale is used in new samples, the exploratory factor analysis should be analyzed.

Furthermore, to support of the factor structure of this scale, future direction of research may continue to test the validation of PLEADS in different cultures and consider delving into the influence of independent and mediator variables (such as organizational climate, school level, etc.) on effective pedagogical leadership, as well as the impact of pedagogical leadership on student outcomes and solving the problem during the coronavirus pandemic. Despite this limitation, the authors consider that testing the measurement invariance of the PLEADS is essential when using data from multiple samples and comparing these latent variables across groups. Moreover, it is important for educators to understand what stakeholders perceive as important indicators or criteria for effective pedagogical leadership (Rosser et al., 2003).

Conclusion

Testing measurement invariance of the leadership scale is challenging and importance to produce before implementing the scale across groups to ensure that they are compared based on the same constructs (Chen, 2007) and also for legitimate assessmment of comparison of perceived differences among various groups (Byrne & van de Vijver, 2017).

The results in this study found support for factor loadings and intercepts invariances among principal, other directing team member, and teacher groups, but no support for residual invariance. Results indicated that (a) Spanish stakeholders assess pedagogical leadership along a single factor structure; (b) the pedagogical leadership scale can be well applied to assessment of the perception of pedagogical leadership importance by stakeholders; (c) cross-group comparisons of means on the pedagogical leadership importance can be made; and (d) most of the variation in leadership is related to the stakeholder’ perception and type of leader. Besides, the perception of multi stakeholders can be utilized to improve pedagogical leadership performance in developing educational institutions for sustainable growth.

References

Abbas, W., & Asghar, I. (2010). The role of leadership in organizational change: Relating the successful organizational change to visionary and innovative leadership. University of Gävle: Sweden.

Alvarez, O., Tomás, I., Estevan, I., Molina-García, J., Queralt, A., & Castillo, I. (2018). Assessing teacher leadership in physical education: The Spanish version of the transformational teaching questionnaire. Anales De Psicología / Annals of Psychology, 34(2), 405-411. https://doi.org/10.6018/analesps.34.2.291711

Antoniou, P., & Lu, M. (2017). Evaluating the measuring properties of the principal instructional management rating scale in the Chinese educational system: Implications for measuring school leadership. Educational Management Administration & Leadership, 46(4), 624-641. https://doi.org/10.1177/1741143217700282

Baptiste, M. (2019). No teacher left behind: The impact of principal leadership styles on teacher job satisfaction and student success Journal of International Education and Leadership, 9(1), 1-11.

Bolívar, A., Caballero-Rodríguez, K., & García-Garnica, M. (2017). Multidimensional assessment of school leadership: Keys to school improvement. Ensaio: Avaliação e Políticas Públicas em Educação, 25(95), 483-506. https://doi.org/10.1590/s0104-40362017002500780

Bolívar, A., López Yáñez, J., & Murillo, F. J. (2013). Liderazgo en las instituciones educativas: Una revisión de líneas de investigación. Revista Fuentes, 14, 19-61.

Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley & Sons.

Bush, T. (2015). Understanding instructional leadership. Educational Management Administration & Leadership, 43(4), 487-489. https://doi.org/10.1177/1741143215577035

Bush, T. (2016). School leadership and management in England: The paradox of simultaneous centralisation and decentralisation. Research in Educational Administration and Leadership, 1(1), 1-23.

Bush, T., & Glover, D. (2014). School leadership models: what do we know? School Leadership & Management, 34(5), 553-571. https://doi.org/10.1080/13632434.2014.928680

Byrne, B. M., & van de Vijver, F. J. R. (2017). The maximum likelihood alignment approach to testing for approximate measurement invariance: A paradigmatic cross-cultural application. Psicothema, 29(4), 539-551. https://doi.org/10.7334/psicothema2017.178

Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. In E. G. Carmines (Ed.), Sage university papers series on quantitative applications in the social sciences (pp. 107-117). London: Sage.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504. https://doi.org/10.1080/10705510701301834

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5

Chou, C.-P., & Bentler, P. M. (1995). Estimates and tests in structural equation modeling. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 37-55). Sage.

Cruz-González, C., Domingo Segovia, J., & Lucena Rodriguez, C. (2019). School principals and leadership identity: A thematic exploration of the literature. Educational Research, 61(3), 319-336. https://doi.org/10.1080/00131881.2019.1633941

Day, C., Sammons, P., Hopkins, D., Harris, A., Leithwood, K., Gu, Q., & Brown, E. (2010). 10 strong claims about successful school leadership. London: The National College for Leadership of Schools and Children’s Services.

Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121-149. https://doi.org/10.1177/0748175610373459

Duke, D. L., Carr, M., & Sterrett, W. (2013). The school improvement planning handbook: Getting focused for turnaround and transition. Plymouth: Rowman & Littlefield Education.

Eichinger, R. W., & Lombardo, M. M. (2003). Knowledge summary siries: 360-degree assessment. Human Resource Planning, 26(4), 34-44.

Fullan, M. (2014). The principal: Three keys to maximizing impact. New York: Jossey-Bass.

García-Garnica, M. (2016). Construction and validation of a questionnaire to measure effective practices of principals´ pedagogical leadership. Profesorado. Revista de Currículum y Formación del Profesorado, 20(3), 493-526.

Gento, S. (2002). Instituciones educativas para la calidad total (Educational institutions of total quality). Madrid: La Muralla.

Gento, S., Huber, G. L., González Fernández, R., Palomares-Ruiz, A., & Orden-Gutiérrez, V. J. (2015). Promoting the quality of educational institutions by enhancing educational leadership. US-China Education Review B, 5(4), 215-232.

Gergen, K. J., & Hersted, L. (2016). Developing leadership as dialogic practice. In J. A. Raelin (Ed.), Leadership-aspractice: Theory and application (pp. 178-197). London: Routledge.

González-Fernández, R., Gento, S., & Orden-Gutiérrez, V. J. (2016). Importancia de la dimensión formativa de los líderes pedagógicos. Revista Iberoamericana De Educación, 70(131-144). https://doi.org/10.35362/rie70091

González-Fernández, R., Khampirat, B., López-Gómez, E., & Silfa-Sención, H.-O. (2020). La evidencia del liderazgo pedagógico de directores, jefes de estudios y profesorado desde la perspectiva de las partes interesadas. Estudios Sobre Educacion, 39, 207-228. https://doi.org/10.15581/004.39.207-228

Gunter, H., Hall, D., & Bragg, J. (2013). Distributed leadership: A study in knowledge production. Educational Management Administration & Leadership, 41(5), 555-580. https://doi.org/10.1177/1741143213488586

Hallinger, P. (2016). Bringing context out of the shadows of leadership. Educational Management Administration & Leadership, 46(1), 5-24. https://doi.org/10.1177/1741143216670652

Hallinger, P. (2019). Science mapping the knowledge base on educational leadership and management from the emerging regions of Asia, Africa and Latin America, 1965–2018. Educational Management Administration & Leadership, 1741143218822772. https://doi.org/10.1177/1741143218822772

Hallinger, P., & Heck, R. H. (2010). Collaborative leadership and school improvement: Understanding the impact on school capacity and student learning. School Leadership & Management, 30(2), 95-110. https://doi.org/10.1080/13632431003663214

Hallinger, P., & Lee, M. (2014). Mapping instructional leadership in Thailand: Has education reform impacted principal practice? Educational Management Administration & Leadership, 42(1), 6-29. https://doi.org/10.1177/1741143213502196

Harris, A., Jones, M., Ismail, N., & Nguyen, D. (2019). Middle leaders and middle leadership in schools: Exploring the knowledge base (2003–2017). School Leadership & Management, 39(3-4), 255-277. https://doi.org/10.1080/13632434.2019.1578738

Holmes, K., Clement, J., & Albright, J. (2013). The complex task of leading educational change in schools. School Leadership & Management, 33(3), 270-283. https://doi.org/10.1080/13632434.2013.800477

Hu, L.-t., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.

Kline, R. B. (2011). Principles and practice of structural equation modeling. New York: Guilford Press.

Lahtero, T. J., Lang, N., & Alava, J. (2017). Distributed leadership in practice in Finnish schools. School Leadership & Management, 37(3), 217-233. https://doi.org/10.1080/13632434.2017.1293638

López-Gómez, E., & González-Fernández, R. (2018). La confianza relacional y la creación de capital profesional: Claves del liderazgo en las instituciones educativas. In A. Medina & E. Pérez (Eds.), Formación de líderes y directivos para el desarrollo sustentable de las Organizaciones e Instituciones (pp. 39-53). Madrid: UNED-Universitas.

Lopez-Zafra, E., Garcia-Retamero, R., & Martos, M. P. B. (2012). The relationship between transformational leadership and emotional intelligence from a gendered approach. The Psychological Record, 62(1), 97–114.

Male, T., & Palaiologou, I. (2013). Pedagogical leadership in the 21st century: Evidence from the field. Educational Management Administration & Leadership, 43(2), 214-231. https://doi.org/10.1177/1741143213494889

Malpass, R. S. (1977). Theory and method in cross-cultural psychology. American Psychologist, 32(12), 1069-1079. https://doi.org/10.1037/0003-066X.32.12.1069

McCarley, T. A., Peters, M. L., & Decman, J. M. (2016). Transformational leadership related to school climate: A multi-level analysis. Educational Management Administration & Leadership, 44(2), 322-342. https://doi.org/10.1177/1741143214549966

Milfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111-121. https://doi.org/10.21500/20112084.857

Mullen, M. R. (1995). Diagnosing measurement equivalence in cross-national research. Journal of International Business Studies, 26(3), 573-596. https://doi.org/10.1057/palgrave.jibs.8490187

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw Hill.

OECD. (2013). Leadership for 21st century learning, educational research and innovation. Paris: OECD publishing. https://doi.org/10.1787/9789264205406-en

Oterkiil, C., & Ertesvåg, S. K. (2014). Development of a measurement for transformational and transactional leadership in schools taking on a school-based intervention. Educational Management Administration & Leadership, 42(4_suppl), 5-27. https://doi.org/10.1177/1741143214523011

Pérez-García, P., Bolívar, A., García-Garnica, M., & Caracuel, A. (2018). Adaptación española de la escala de liderazgo pedagógico Vanderbilt assessment of leadership in education (val-ed). Universitas Psychologica, 17(1), 1- 13. https://doi.org/10.11144/Javeriana.upsy17-1.aeel

Pont, B., Nusche, D., & Hopkins, D. (2008). Improving school leadership volume 2: Case studies on system leadership. Paris: OECD publishing.

Pont, B., Nusche, D., & Moorman, H. (2008). Improving school leadership volume 1: Policy and practice. Paris: OECD publishing.

Pop, C., & Khampirat, B. (2019). Self-assessment instrument to measure the competencies of Namibian graduates: Testing of validity and reliability. Studies in Educational Evaluation, 60, 130-139. https://doi.org/10.1016/j.stueduc.2018.12.004

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review: DR, 41, 71-90. https://doi.org/10.1016/j.dr.2016.06.004

Rosser, V. J., Johnsrud, L. K., & Heck, R. H. (2003). Academic deans and directors: Assessing their effectiveness from individual and institutional perspectives. The Journal of Higher Education, 74(1), 1-25. https://doi.org/10.1080/00221546.2003.11777185

Schoot, R. v. d., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486-492 https://doi.org/10.1080/17405629.2012.686740

Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-338. https://doi.org/10.3200/JOER.99.6.323-338

Shaked, H., & Schechter, C. (2016). Systems thinking among school middle leaders. Educational Management Administration & Leadership, 45(4), 699-718. https://doi.org/10.1177/1741143215617949

Spillane, J. P. (2005). Distributed Leadership. The Educational Forum, 69(2), 143-150. https://doi.org/10.1080/00131720508984678

Tian, M., Risku, M., & Collin, K. (2015). A meta-analysis of distributed leadership from 2002 to 2013: Theory development, empirical evidence and future research focus. Educational Management Administration & Leadership, 44(1), 146-164. https://doi.org/10.1177/1741143214558576

Tian, M., & Virtanen, T. (2019). Shanghai teachers’ perceptions of distributed leadership: Resources and agency. ECNU Review of Education, 2096531120921051. https://doi.org/10.1177/2096531120921051

Tintoré, M., Cunha, R. S., Cabral, I., & Alves, J. J. M. (2020). A scoping review of problems and challenges faced by school leaders (2003–2019). Educational Management Administration & Leadership, 1741143220942527. https://doi.org/10.1177/1741143220942527

Vaillant, D. (2015). School leadership, trends in policies and practices, and improvement in the quality of education. París: UNESCO.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. https://doi.org/10.1177/109442810031002

Vilkinas, T. (2009). Predictors of leadership effectiveness for Chinese managers. Leadership & Organization Development Journal, 30(6), 577-590. https://doi.org/10.1108/01437730910981944

Vrieze, S. I. (2012). Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological methods, 17(2), 228-243. https://doi.org/10.1037/a0027127

Wang, L. H., Gurr, D., & Drysdale, L. (2016). Successful school leadership: Case studies of four Singapore primary schools. Journal of Educational Administration, 54(3), 270-287. https://doi.org/10.1108/JEA-03-2015-0022

Wenner, J. A., & Campbell, T. (2016). The theoretical and empirical basis of teacher leadership: A review of the literature. Review of Educational Research, 87(1), 134-171. https://doi.org/10.3102/0034654316653478

Yap, S. C. Y., Donnellan, M. B., Schwartz, S. J., Kim, S. Y., Castillo, L. G., Zamboanga, B. L., Weisskirch, R. S., Lee, R. M., Park, I. J. K., Whitbourne, S. K., & Vazsonyi, A. T. (2014). Investigating the structure and measurement invariance of the Multigroup Ethnic Identity Measure in a multiethnic sample of college students. Journal of Counseling Psychology, 61(3), 437-446. https://doi.org/10.1037/a0036253

Contact address: Buratin Khampirat. Suranaree University of Technology, Institute of Social Technology. 111 University Avenue, Suranaree Sub-district, Muang District. Nakhon Ratchasima. 30000. Thailand. E-mail: buratink@sut.ac.th

---