A Bibliometric Analysis on Efficiency in Education

Un análisis bibliométrico sobre la eficiencia en la educación

https://doi.org/10.4438/1988-592X-RE-2024-406-639

Muammer Maral

https://orcid.org/0000-0002-2055-5711

National Defense University

Abstract

Efficiency, a component of performance, refers to how well a decision-making unit utilises its resources in the production process. Education is a complex process with multiple inputs and multiple outputs. In order to make the most of the benefits of education, resources should be used in the best way. This will prevent waste of resources as well as increase the outputs. For these reasons, efficiency in education has become one of the most frequently discussed issues among policy makers, researchers and decision makers in recent years. This study provides a comprehensive review of the growing literature on educational efficiency. This research aims to reveal the structure of the field by addressing the efficiency in education literature from a holistic perspective. The study conducted a bibliometric analysis with 1315 publications covering the period between 1970 and 2022 obtained from the SCOPUS and Web of Science databases. According to the results of the study, the annual growth rate of efficiency in education literature is approximately 9%. The last 10 years of growth cover more than half of the 50 years of literature. This research revealed the contributions to the field in terms of journals, authors, institutions and countries. The most impactful publications of the field were identified. Among 78 countries, China and the United States have made the largest contribution and collaborations. Data envelopment analysis, stochastic frontier analysis, higher education, productivity and performance, technical efficiency, scale efficiency, allocative efficiency, and cost efficiency are the main themes of the field. Research efficiency, ranking, quality, school efficiency, competition, e-learning, and education in Europe are the trending topics in the field.

Keywords: efficiency, education, efficiency in education, review, bibliometric analysis.

Resumen

La eficiencia, un componente del rendimiento, se refiere a lo bien que una unidad de decisión utiliza sus recursos en el proceso de producción. La educación es un proceso complejo con múltiples entradas y múltiples salidas. Para aprovechar al máximo los beneficios de la educación, los recursos deben utilizarse de la mejor manera. Esto evitará el despilfarro de recursos y aumentará los resultados. Por estas razones, la eficacia de la educación se ha convertido en los últimos años en uno de los temas más debatidos entre los responsables políticos, los investigadores y los responsables de la toma de decisiones. Este estudio ofrece una revisión exhaustiva de la creciente bibliografía sobre la eficacia educativa. Esta investigación tiene como objetivo revelar la estructura del campo abordando la eficiencia en la literatura educativa desde una perspectiva holística. El estudio realizó un análisis bibliométrico con 1315 publicaciones que abarcan el período comprendido entre 1970 y 2022 obtenidas de las bases de datos SCOPUS y Web of Science. Según los resultados del estudio, la tasa de crecimiento anual de la eficiencia en la literatura educativa es de aproximadamente un 9%. Los últimos 10 años de crecimiento cubren más de la mitad de los 50 años de literatura. Esta investigación reveló las contribuciones al campo en términos de revistas, autores, instituciones y países. Se identificaron las publicaciones más influyentes del campo. Entre 78 países, China y Estados Unidos han realizado la mayor contribución y colaboración. El análisis envolvente de datos, el análisis de frontera estocástica, la educación superior, la productividad y el rendimiento, la eficiencia técnica, la eficiencia de escala, la eficiencia de asignación y la eficiencia de costos son los temas principales del campo. La eficiencia de la investigación, la clasificación, la calidad, la eficiencia escolar, la competencia, el aprendizaje electrónico y la educación en Europa son los temas de moda en este campo.

Palabras clave: eficiencia, educación, eficiencia en la educación, revisión, análisis bibliométrico.

Introduction

Efficiency is a concept that expresses how well a decision-making unit utilises its resources in the production of goods and services (Kalb, 2010). In the context of education, efficiency, that is, the efficient use of resources, refers to the production of the maximum educational output using the minimum input (Johnes et al., 2017). In education, efficiency can be mentioned if the existing inputs are utilised in the most optimum way possible, that is, if maximum output can be produced. In an inefficient system, there is a possibility of increasing educational outcomes at a certain level of expenditure or decreasing the educational resources used for certain educational outcomes (Bessent & Bessent, 1980).

Efficiency is one of the topics examined by different disciplines in the literature. In the literature, there are studies on efficiency in many fields such as economics (Chen et al., 2018; Khodadadipour et al., 2021; Tan, 2006; Vidoli & Ferrara, 2015), administration (Ang et al., 2021; Holmgren, 2018; Venkadasalam et al., 2020), energy (De Clercq et al., 2019; Moutinho et al., 2020; Quintano et al, 2021), health (Bayley et al., 2022; Gomez-Gallego et al., 2021; Moreno-Enguix et al., 2018), transportation (Chang et al., 2018; Chao et al., 2018; Park et al., 2018), and more.

Kalirajan and Shand (1999) argue that there are mainly three benefits of efficiency measurement. Firstly, efficiency measurement offers the possibility of comparison between similar units. This way, one can compare homogeneous units with each other and determine their relative efficiency levels. Secondly, efficiency measurement can identify the source of efficiency differences between units. Thirdly, efficiency measurement reveals some implications for improving the efficiency of units.

Considering these benefits of effectiveness measurement, one can see that many studies have been conducted on effectiveness in the field of education as in other disciplines. The scope of research on efficiency in education (EE) is quite diverse. Some of the research focuses on the efficiency of a particular level of education in a country (Agasisti & Dal Bianco, 2009; Andersson et al., 2017; Brzezicki, 2020; Dufrechou, 2016). Some studies examine the educational efficiency of more than one country (Agasisti, 2014; Agasisti & Zoido, 2018; Delprato & Antequera, 2021). In addition to these, researchers have also conducted studies examining public educational institutions (Canal et al., 2015; Kantabutra & Tang, 2010; Turkan & Ozel, 2017; Visbal-Cadavid et al., 2017) and private educational institutions (Bayraktar et al., 2013; Shamohammadi & Oh, 2019). Some studies have also examined the efficiency of subsystems such as academic departments (Anastasiou et al., 2007; Kao & Hung, 2008) and libraries (Reichmann, 2004; Tavares et al., 2018). Some studies address the teaching and research missions of higher education institutions (Gralka et al., 2019; Jiang et al., 2020; Maral, 2023; Tran et al., 2020).

The literature shows that the scope of research on EE is diverse and the literature has a rich outlook. Witte and López-Torres (2017) found that the EE literature is growing. Therefore, there is a need to examine these studies conducted at different levels and fields of education with a holistic perspective. There are some studies in the literature that are parallel with this purpose.

Rhaiem (2017) conducted a literature review of research on academic research efficiency. Worthington (2001) listed the papers that used frontier estimation methods for measuring efficiency in education. Johnes (2004) described the techniques used to measure efficiency and explained the uses and disadvantages of applying different methods in an educational context. Lampe and Hilgers (2015) conducted a bibliometric analysis on data envelopment analysis and stochastic frontier analysis, two of the most widely used efficiency measurement methods. Ferro and D’Elia (2020) reviewed the literature on input and output variables used in research on efficiency in higher education. Witte and López-Torres (2017) provide a detailed description of the inputs, outputs, and non-discretionary variables used in research on EE. Additionally, they revealed the efficiency measurement methods used in studies together with their sources. Emrouznejad et al. (2008) examined the state of data envelopment analysis over 30 years with a descriptive point of view. Villano and Tran (2021) conducted a meta-regression analysis to provide a better understanding of the relationship between data envelopment analysis and the quality of efficiency research in higher education. Çetin and Maral (2022) examined the research on efficiency in higher education in terms of content and methodology.

It is seen that each of these studies has made significant contributions to the field of efficiency. However, upon evaluating these studies as a whole, one can see that they focus on a specific area in terms of content or methodology, or they do not specifically examine the whole field of EE. Therefore, there is a need to address the applications of research on EE in the field of education from a holistic perspective with a macro focus. Hence, this study aims to examine the research on efficiency in education from a holistic perspective, explore the intellectual structure of the field, reveal patterns in the literature, and contribute to advancing the field in new and meaningful ways. To achieve this objective, the study seeks to answer the following questions:

1. What is the basic bibliography of research on EE? To reveal the intellectual structure of the resulting research, researchers focused on the following key bibliometrics: (1) Number of publications and citations over time, (2) Most influential publications, (3) Most influential and productive journals, (4) Most influential and productive authors, (5) Institutions with the most contributing, (6) Countries with the most contributing, and (7) Most frequently used keywords.
2. What themes have emerged in the EE literature? A “Thematic Map Analysis” was conducted to reveal the main themes in the studies.
3. What are the trending topics in the EE literature? “Trend Topics” analysis was conducted to identify the trending topics that have emerged in the EE literature in recent years.
4. What is social interaction between countries in the EE literature? A “Co-country analysis” was conducted to reveal the social interaction and connections between countries and their effects on the development of the field in the EE literature.

Method

This study used a bibliometric method to review the EE literature comprehensively and cumulatively, determine the general appearance and trends of publications in the EE literature, reveal social interactions, and reveal the scientific map of the literature. Bibliometric studies attempt to synthesize patterns of knowledge production through the analysis of bibliographic data associated with a body of relevant documents (Van Eck & Waltman, 2014; Zupic & Čater, 2015). This type of systematic review reveals the evolving trends of a field’s literature over time and provides empirical foundations for the path forward (Hallinger, 2021). The volume of scientific research has increased significantly in recent years. This has made it increasingly difficult for researchers to follow the relevant literature in their field. This fact necessitates the use of quantitative bibliometric methods that can handle this vast wealth of data, estimate the impact of important studies, and explore the underlying structure of the literature in a field (Zupic & Čater, 2015).

The Identification of Resources

This study used the Web of Science Core Collection (WOS) and Scopus databases, which cover a significant portion of the global literature and are most frequently used in bibliometric studies. Studies using bibliometric analysis often use only one of these two databases. The researchers decided to use these two databases together to cover the EE literature more comprehensively.

The inclusion criteria for the sources are as follows: (1) The research topic is efficiency, (2) The research is in the field of education, (3) The research falls into one of the categories of an article, paper, literature review, book chapter, and book, (4) The research was published before September 12, 2022. The exclusion criteria are as follows: (1) Research on efficiency in any field other than education, (2) Studies other than articles, papers, literature reviews, books, and book chapters. The reason behind including papers, book chapters, and books, which are called gray literature, in this study is to prevent the exclusion of the main and influential sources of the subject from the analysis. Furthermore, there was no time period filter in this study. The resources obtained cover the period between 1970 and 2022.

The selection of appropriate keywords is very important in bibliometric studies and this directly affects the results (Sweileh, 2018). The final search strategy chosen after many attempts to search for different keyword combinations is presented in the Appendix. The researcher developed a comprehensive search strategy to ensure that none of the relevant sources were overlooked in the formulation of the search strategy. The more specific and narrow the concepts used in databases, the fewer the number of publications that emerge. However, if the efficiency topic does not have this quality and still a person narrows the search strategy, they may miss many relevant resources. There is also a risk of missing important resources on the same topic from external fundamental journals. Therefore, this study developed a comprehensive search strategy. The preliminary review of the literature showed that the research on EE was carried out by many disciplines. Subject-specific filtering of the databases was used to narrow down the search results. However, researchers filtered the fields to exclude subject areas that were the most distant from education (e.g., pharmacy, chemistry, neuroscience). However, although this is very unlikely, it may have resulted in some research on efficiency in education in these areas being overlooked.

In the search strategy, TI stands for searches in the title, AK for searches in the author keywords, and AB for searches in the abstract. The researcher searched for the phrase “efficienc*” in the title and author keywords. The presence of this term in the title and author keywords facilitated the retrieval of research related to the efficiency. However, this concept was not searched in the abstract. The main reason for this is that if the literature is large, many false positives, that is, irrelevant studies, may be included in the data set when searching in the abstract. Although all publications could have been analysed individually with the inclusion and exclusion criteria, this was avoided due to the high number of publications to be analysed. This may have resulted in some relevant studies not being included in the dataset. Since there is research in many disciplines in the literature on the subject of efficiency, concepts related to education (e.g., education, university, higher education, etc.) were also included in the title to distinguish those related to education as much as possible. However, although this brings us closer to the field of EE, it also included many studies other than the concept of economic efficiency in the data. To prevent this, the researchers searched for efficiency-specific concepts (e.g., data envelopment analysis, stochastic frontier analysis, technical efficiency, etc.) in the title, abstract, and author keywords. This search strategy allowed the researcher to get closer to the studies of interest. The researchers decided to use the search strategy in this way because narrowing the search strategy any further might exclude relevant studies.

Data Extraction and Analysis

This review followed the PRISMA guidelines commonly used for reporting systematic reviews of research (Page et al., 2021). Figure I presents the PRISMA flowchart followed in this study. With the determined search strategy, separate searches occurred in WOS and SCOPUS (on September 12, 2022). The WOS database provided 1469 documents and the SCOPUS database provided 2517 documents. Since 1042 of the total 3986 documents obtained were duplicates, the researchers removed them from the data set. Of the remaining 2944 documents, 64 were excluded from the dataset as they did not meet the document type specified in the inclusion criteria. Of the remaining 2880 documents, 1565 were excluded from the dataset as they were irrelevant “false positives”. At the final stage, 1315 documents remained to be included in the analysis.

Bibliometric data associated with the 3986 documents were exported from WOS to a plain text file. The data from SCOPUS were downloaded in two parts, merged, and saved as “.bib” file. The documents obtained from WOS and SCOPUS were converted into a single bibliometric MS Excel data document using the R Studio program. These data documents contain descriptive information such as author, title, publication data, citation data, abstract, etc. The author reviewed the generated MS Excel data document to eliminate irrelevant publications. It was necessary to remove the studies not relevant to the subject after the studies were identified. For example, a publication may contain keywords used when searching. However, this study may not be relevant to the topic of interest. In such cases, these studies can generate “false positives” (Linnenluecke et al., 2020). To eliminate false positives, all publications relevant and irrelevant to EE were identified. The author reviewed this data document and sent it to three independent field experts for further review. Changes were made to the data document in line with the feedback from the field experts. In cases where experts’ opinions contradicted each other, the researchers met with the experts and agreed on the documents that should be included and excluded. Finally, a large body of data from many disciplines related to the topic under study was obtained.

In bibliometric analysis, one of the important points for the accuracy and reliability of the analysis is data cleaning. Problems such as duplicate values, spelling and typographical errors, and lack of important information (year, author name, journal name, etc.) should be corrected in documents retrieved from databases. Because although most bibliometric data is reliable, it can sometimes contain multiple versions of the same study. There may be differences in the spelling of authors and journals (Zupic & Čater, 2015). In this study, data cleaning was a meticulous process. First, duplicate works were eliminated. Second, important missing information was supplemented by accessing databases or, where necessary, the full text of the study. Third, spelling differences were examined (e.g., singular-plural words such as efficiency/efficiencies, hyphenated spellings such as higher education/higher-education, or abbreviations such as DEA/data envelopment analysis, etc.). Fourth, prior to the thematic analysis used to reveal the thematic structure of the EE field, author keywords were removed. At this stage, different words with the same meaning were combined by preparing a “synonym text” text file used in R Bibliometrix software. This way, scattered words with the same meaning were grouped. Another important aspect of thematic analysis is the removal of unnecessary and meaningless words from the analysis to reveal the themes of the field under study. For example, very general expressions such as “education, input, and output” or words that do not make sense for thematic analysis such as “i21, 2009” were excluded from the analysis by preparing a “remove text” text file used in R Bibliometrix software.

The data analysis used the MS Excel program, Bibliometrix, and Biblioshiny software packages in the R tool. The Bibliometrix R package is an open-source software tool that provides a set of tools for quantitative research in scientific methodology. The R software includes a large number of efficient statistical algorithms. High-quality numerical operations and the availability of data visualization tools are among the strong features of R languages compared to other languages in scientific computing (Aria & Cuccurullo, 2017). Biblioshiny is a secondary development of the Bibliometrix-based Shiny package in the R language. Biblioshiny includes the core code of Bibliometrix and offers web-based data analysis. Biblioshiny enables scientific measurements and visual analysis (Xie et al., 2020).

This study fundamentally seeks to answer four questions. The researcher conducted basic bibliometric analyses to answer the first question of the study. As a measurement, the researcher analyzed the authors, documents, journals, institutions, and countries according to the number of publications, total citations, and citations per publication. The ratio between the number of documents and the total number of citations is a good measure of the impact of publications and contributors (Saravanan et al., 2022).

The researcher conducted a thematic analysis to answer the second question. Thematic analysis is a method used to provide visualization and interpretation of the topic and to identify current trends over a given time period (Cobo et al., 2011). This approach creates graphical representations that automatically summarize the main topics of a publication. One can also characterize the emerging themes according to their structure and role in the network. Thematic analysis reveals a strategic diagram. This diagram has two dimensions, intensity and centrality, and allows four different types of topics to be highlighted depending on the quadrant in which they are mapped. High centrality and intensity values identify popular themes relevant to structuring the conceptual framework of the field. High centrality and low-density values define the key subjects that are important for the area. Low centrality and low intensity values identify topics that are not yet developed or marginally interesting for the area. Low centrality and high-density values identify topics that are strongly developed but niche for the area, with a potential for development (Aria et al., 2022).

Trend topics analysis was used to answer the third question. Trend topics analysis is an analysis that reveals the latest trending studies in a field. This determines the direction of the field and guides the authors for future research.

In answering the fourth question, a country co-authorship analysis was conducted. This analysis aims to reveal the social structure of the area and its interactions. Since an article is published by more than one author, institution, or country, a relationship forms between them (Lu & Wolfram, 2012). The co-authorship of scientific publications examines the social networks that scientists, institutions, and countries form by collaborating on scientific publications (Adamides & Karacapilidis, 2006). Collaboration network analysis allows people to learn about a field’s potential research partners for the publication of future research (Xu et al., 2022). Co-authorship is considered a measure of collaboration. This measure reflects social ties more strongly than other measures of collaboration. In addition, bibliometric data also contain information about the institutions and countries of the authors, so one can also examine the collaboration networks between institutions and countries (Zupic & Čater, 2015). This study conducted a country co-authorship analysis to reveal the social interaction network of the field.

Findings

Overview of EE Publications

Table I presents the general view of the publications on EE. Publications on EE span the period from 1970 to 2022 and there were a total of 1315 documents. The annual growth rate of publications is 8.92%. The literature includes contributions from 78 countries and a total of 2349 authors.

Figure II presents the number of publications on EE literature and the average number of citations per year. It is seen that the number of publications on EE has increased significantly over the years. There was not a high number of publications in the period from the 1970s until 2002. Especially since 2002, a period of increase in the number of publications has occurred. In 2015, there was a decrease in the number of publications, but the increase continued in the following years. This trend has peaked in the last 10 years, accounting for 68% of the total publications. The average number of citations per year, on the other hand, is not regular and constantly fluctuates up and down.

The Most Influential Publications

Table II presents the list of the ten most cited publications in the EE literature. The publication with the highest total citations (TC) and total citations per year (TCY) was the research by Emrouznejad et al. (2008). This study examined a 30-year bibliographic history of the data envelopment analysis literature. One of the important results of the research is that the literature on the use of data envelopment analysis has increased significantly during this period. Avkiran (2001) and Abbott and Doucouliagos (2003), who are among the top three according to the total number of citations, examined the technical and scale efficiency of universities in Australia.

The Most Productive and Influential Journals in EE Literature

Table III lists the top ten most prolific and influential journals in the EE literature. The journal with the highest number of publications is “Economics of Education Review”. This journal publishes research on education policies and finance, human capital production, and returns to human capital. The fact that the journal publishes topics related to education policy and finance is an important factor in its being the journal with the highest number of publications in the field of EE. This journal also ranks first in terms of the total number of citations. The second journal that contributes to the field is “Socio-Economic Planning Sciences”. It ranks second in terms of the number of publications and total citations. The journal that stands out in the list is “Journal of The Operational Research Society”. This journal, which publishes studies on decision-making and operational research, has the highest CP value. This shows that the published studies have a high impact and this may increase the journal’s significance in the field of EE in the future.

The Most Prolific and Influential Authors in EE Literature

Table IV presents the list of authors who have made significant contributions to the EE literature. The top ten authors are listed according to the number of publications. Agasisti, T. tops the list with 38 publications. In terms of the number of publications, one can see that this author has published a considerably higher number of publications compared to other authors. In second place is Johnes, J. The striking feature of this author is that his publications are highly cited. They have the highest CP value among all other authors. Other authors in the top four in terms of both NP and CP are Johnes, G. and De, W.K.

Institutions Contributing the Most to EE Literature

Table V presents the list of institutions that have made significant contributions to the EE literature. It is seen that “Lancaster University” has made the most contribution to the EE literature. This is followed by the Polytechnic University of Milan and Maastricht University. Among the top contributors, there are four institutions from Spain. Among the institutions in the top ten, all countries except Australia are in Europe.

Countries with the Highest Contribution to EE Literature

Figure III presents the distribution of the number of publications from 78 countries contributing to the EE literature. The country that contributed the most to the EE literature is China with 323 publications. The USA ranks second with 229 publications. In third place is Spain with 164 publications, in fourth place is Brazil with 117 publications and in fifth place is Italy with 112 publications. These countries are followed by the United Kingdom, Australia, Germany, Czechia, and Russia. One can see that the least contributing countries are those on the African continent.

The Basic Concepts in EE Literature

A “Word Cloud” analysis was conducted to reveal the structure of the concepts in the EE literature. This analysis was based on author keywords. Figure IV presents the results of the analysis. The most frequently repeated words in the EE literature are “efficiency”, “data envelopment analysis (DEA)”, “higher education”, “education economics”, and “performance”. DEA is one of the non-parametric methods used to measure the efficiency of decision-making units. One can consider the fact that DEA is the most used concept as an indicator that it is one of the most used methods in efficiency measurement. Even though studies have been conducted at different levels of education in the EE literature, it is seen that there are more studies related to efficiency measurement in “higher education”. The presence of the term “research efficiency” on the map is one of the indicators of this.

The View of Themes in the EE Literature

Thematic analysis was conducted to answer the second research question. Figure V presents the results of the analysis. In the EE literature, studies on higher education stand out among the popular themes (motor themes) related to structuring the conceptual framework of the field (Abbott & Doucouliagos, 2003). In this theme, it is noteworthy that DEA as a method is an important tool for measuring efficiency (Abbott & Doucouliagos, 2003; Avkiran, 2001; Canal et al., 2015). Moreover, this theme is bordered by the “core themes” that reveal the concepts that underpin the field and relate to the basic building ages of the EE literature. Another theme within the motor themes is efficiency types. This theme includes technical efficiency, scale efficiency, allocative efficiency, and cost efficiency. These four types of activities are among the main types of activities frequently seen in the literature (Andersson et al., 2017; Avkiran, 2001; Turkan & Ozel, 2017). “Stochastic frontier analysis” is a parametric efficiency measurement method that is frequently used in efficiency measurement, although not as much as DEA (Khodadadipour et al., 2021). This method is within the core area of the EE literature. There are many studies on efficiency measurement with panel data (Agasisti & Dal Bianco, 2009; Canal et al., 2015). This type of data includes data covering more than one time period and efficiency measurement occurs with this data. This way, one can analyze the efficiency change of decision-making units in different time periods.

The area where the concept of “PISA” appears on the thematic map refers to a topic that has recently emerged or is gradually losing its influence. Recently, in the EE literature, studies have emerged in which cross-country comparisons of efficiency are made by considering the performance of countries in PISA exams (Agasisti & Zoido, 2018; Delprato & Antequera, 2021). Therefore, PISA-based cross-country comparisons of efficiency have the potential to become the subject of more research over time.

Finally, the emerging and niche themes are “innovation”, “school efficiency”, “technology transfer”, and “human capital”. This theme analyzes the academic achievement-based activities of primary and secondary schools in a specific region (Agasisti & Zoido, 2018; Barra & Zotti, 2017; Delprato & Antequera, 2021). The studies conducted under this theme also included comparisons of the efficiency of schools in regions with different levels of development. Within this theme, some studies that look at the relationship between universities and innovation from different perspectives (Shi, et al., 2020). Technological, scientific, and regional innovation efficiency of universities, the contribution of universities to regional innovation and the relationship between them, and the effect of industry-university cooperation on innovation efficiency are among the topics examined (Agasisti et al., 2019a; Barra & Zotti, 2018; Gong et al., 2022). An important field of efficiency of universities is the transfer and application of the knowledge gained at university to the industry. Therefore, technology-transfer efficiency is also analyzed in the context of different countries (Agasisti et al., 2019a; Anderson et al., 2007). The relationship between universities’ technology transfer efficiency and entrepreneurship is also among the issues examined. In addition to these, studies on human capital in universities are also one of the topics that have been researched in recent years. The evaluation of the output efficiency of universities’ human resources, the evaluation of personnel efficiency, the management of human resources, and the development of the efficiency of this resource come to the fore. Comparing the efficiency of different learning methods in human resource development and the relationship between the efficiency of education and the level of human development are among the niche topics that stand out in the literature.

Trending Topics in the EE Literature

The study conducted a “Trend Topics” analysis to identify trending research topics in the EE literature. Figure VI presents the results of the analysis. When the figure is examined, one can see that there are different topics that have been trending in the EE literature in recent years. People have focused their attention on studies measuring the research effectiveness of universities in recent years. Determining the educational efficiency of countries based on PISA exams (Agasisti & Zoido, 2018; Delprato & Antequera, 2021), innovation efficiency of universities (Shi, et al., 2020), efficiency of schools (Agasisti & Zoido, 2018), technology transfer efficiency of universities (Agasisti et al., 2019a; Anderson et al., 2007), and human capital (Barra & Zotti, 2017) are among the niche topics in the previous thematic analysis. Therefore, these niche topics have also become trending topics in recent years. Among the trending topics, the most studied and literature-dominant topic is the study of the efficiency of higher education institutions. Additionally, the efficiency of publicly funded education, such as the effectiveness of state-owned higher education institutions, universities, national universities, libraries belonging to public universities, and the effectiveness of state expenditures, are also trending topics.

One of the trending topics in the EE literature is quality. Efficiency evaluation of students’ learning quality, quality of entrepreneurship education, quality of teaching and research, and efficiency research on the impact of teacher quality on student achievement are among the topics examined (Arbano et al., 2022; Lee & Johnes, 2022). In addition to these, research on the quality of educational programs in higher education and efficiency research based on quality variables are among the trending topics in the EE literature.

In the EE literature, some studies examine the competition of schools and universities based on efficiency (Liu & Xu, 2017). The impact of competition between universities on efficiency and the impact of competition between schools on efficiency are among the topics examined. Moreover, the relationship between competition and the efficiency of regional higher education institutions, the impact of the financial efficiency of universities on national competitiveness, and the impact of school cooperation on efficiency have recently been the subject of study.

In the EE literature, European countries and the efficiency of the European education system are one of the main interests of the authors (Agasisti et al., 2019b). The examination of the efficiency of the European higher education system, and comparative analysis of the educational efficiency of different countries in Europe are among these topics. Specific areas of investigation include the research and development efficiency of the European education system or the efficiency of European structural funds.

One of the trending topics in the EE literature is e-learning applications (Chang, 2013). In the literature, examining the efficiency of distance education universities, examining the efficiency of e-learning systems, and comparing the efficiency of traditional and e-learning systems are among the topics examined. In addition to these, there are also areas of study such as examining the impact of the use of digital mobile e-learning applications on school efficiency.

Country Collaboration Network in EE Literature

To reveal the collaboration network of the countries contributing to the EE literature, a collaboration network analysis of 78 countries was conducted. The analysis determined that 18 of these countries did not cooperate. The analysis included the remaining 60 countries and Figure VII presents the results. The countries with the highest number of collaborations in the EE literature are China, the USA, Spain, the United Kingdom, and Italy. China dominates the collaboration network, followed by the United States. One can say that countries with low participation in the collaboration network collaboration with countries geographically close to them. For example, the Egypt-UAE collaboration, the Denmark-Switzerland-Finland-Norway collaboration, and the Peru-Cuba collaboration. The Slovakia-Czech Republic collaboration.

Discussion

Efficiency is a research topic studied by many disciplines. Efficiency research has been conducted in many areas such as educational institutions, students, programs, and academic units. The need to map the EE literature has arisen to comprehensively address this literature that has accumulated over time, to review the field of EE, and to provide a perspective for researchers. Based on this need, this research aims to address the research on efficiency in the field of education from a holistic perspective, explore the intellectual structure of the field, reveal patterns in the literature, and contribute to advancing the field in new and meaningful ways. To achieve this goal, the study sought answers to four questions. These are to reveal the basic bibliography of the EE literature, identify the themes in the field, identify the trending topics of the field, and reveal the social interaction of the countries contributing to the field. For this purpose, bibliometric analysis was performed on a total of 1315 publications covering the period between 1970-2022 using the Web of Science and SCOPUS databases.

The EE literature has generally shown continuous growth since 1970. In particular, the progress in the last 10 years covers 68% of the EE literature spanning nearly 50 years. There has been research on various types of efficiencies in many fields from basic education to universities, academic units, and education systems of countries. The study of De Witte and López-Torres (2017) has also clearly demonstrated this. In the EE literature, the most fundamental reason for continuous improvement is the need to make the most effective use of the limited resources available such as education budget, personnel, technology, and infrastructure. Educational efficiency is an important topic of debate among policy makers, teachers and other stakeholders. Apart from the increasing awareness of public efficiency, the rising cost of education is one of the reasons for the growing interest in educational efficiency (De Witte & López-Torres, 2017). It is desirable for all levels of education, from basic education to higher education, to carry out their activities in the most effective way. The way to determine this is to measure the efficiency of the decision-making units in accordance with the purpose. One can consider the significant growth of the EE literature, especially in the last 10 years, as an indication that the literature will grow to achieve the same goals in the future.

Various journals have contributed to this development of the EE literature. The interdisciplinary nature of EE research has a significant contribution to the emergence of this diversity. Although a wide variety of disciplines and journals have contributed to the EE literature, the “Economics of Education Review” has dominated the literature. This journal has been publishing many quality papers in the field of EE since 1981. This journal can be seen as both a crucial resource for researchers who want to publish research in the field of EE and an important alternative for publication opportunities.

Looking at the EE literature from a global perspective, the fact that China and the USA are the two countries that have made significant contributions to the field can be attributed to the higher number of higher education institutions and thus the higher number of researchers in these countries. However, the fact that researchers in some countries specialize and conduct research in the field of EE causes these countries stand out. For example, Agasisti T. (from Italy) turns out to be the most prolific author in the field. This contributed to Italy’s entry into the category of productive countries.

The keyword analysis shows that DEA is quite dominant among efficiency measurement methods. This finding supports De Witte and López-Torres (2017), and Villano and Tran (2021). One of the methods frequently used in EE measurement is stochastic frontier analysis. However, there are some reasons why DEA is used more in efficiency measurement. DEA is an optimization method of mathematical programming. DEA is a method that can measure the efficiency of units with multiple inputs and multiple outputs (Gralka et al., 2019; Mikušová, 2017). The purpose of DEA is to determine the efficiency of resource use in different organizations and the available technologies used and to establish a measure to assess the successes or opportunities for resource conservation for each form of decision-making to which resources are allocated (Charnes et al., 1978). Nonparametric efficiency measurement methods such as DEA have attracted the interest of researchers from many fields of science, especially management scientists. This is because it can be applied to measure the technical efficiency of decision-making units by considering only inputs and outputs without any price information and taking into account multiple inputs and outputs (Villano & Tran, 2021). Educational institutions are not organizations that try to achieve their goals with a single input and a single output. On the contrary, it operates using multiple inputs and outputs. Educational institutions are non-profit organizations without input and output prices (Andersson et al., 2017). Efficiency estimates based on DEA are therefore a theoretically sound approach for educational institutions with multiple inputs and outputs without price information (Villano & Tran, 2021).

Despite the methodological dominance of DEA in the EE literature, higher education has a significant dominance among the education levels that have been analyzed in terms of efficiency. Following the drastic cuts in public expenditures in many developed countries in the 1980s, efficiency in higher education became a particular discussion topic (Peters, 1992). The rapid increase in student numbers, coupled with these cuts, raised concerns about the performance of higher education institutions. During this period, governments tried to meet the growing demand for higher education with the same or lower levels of funding. A negative view related to efficiency developed due to the reduction in resources allocated by public funders (Kupriyanova et al., 2018). However, the value of efficiency was re-established with the adoption of new public management principles in higher education (Broucker et al., 2015). The fact that the demand for higher education continues to increase today, the increase in expenditure amounts in this direction, and the correct realization of the missions and activities expected from higher education caused the subject of efficiency in higher education to stay prevalent.

The topic of the efficiency of higher education research, which dominates the EE literature, has diversified over time. One can consider the efficiency studies on innovation, technology-transfer, and human resources that emerged in this research as an indicator of this. In a knowledge-based economy, higher education institutions play a critical role in innovative economic development as producers and transmitters of knowledge (Etzkowitz & Leydesdorff, 1995; Perkmann et al., 2013). Innovative processes are seen as the driving force of economic development (Dunning, 2000). Since the 1970s, industrialized countries have launched many initiatives to promote closer collaboration between universities and industry, hoping to stimulate innovation and economic growth. In this way, the innovation role of universities has gradually expanded (Anselin et al., 1997; Feller et al., 2002). Countries with higher innovation performance generally achieve higher economic growth, greater competitiveness, and a better standard of living (Acs et al., 2002). The innovation processes of educational institutions, and universities in particular, are critical for ensuring sustainable economic growth (Shi et al., 2020). This important link between universities and innovation has paved the way for the evaluation of innovation efficiency of universities in regional, technological, scientific, and entrepreneurial terms.

In recent years, the interest in efficiency in education has increased in practical and academic terms (Goldstein & Woodhouse, 2000). In parallel with this, efficiency research on human capital has gained importance in the EE literature. One of the reasons for the emergence of research in this area is the increasing significance of educational institutions in the economy as they provide intellectual training, higher quality human capital, and increased labor productivity (Blau, 1996). There are various activities that universities can perform to boost economic development. These include both knowledge creation and those related to regional innovation through research and technology transfer (Barra & Zotti, 2017). Highly qualified and well-educated individuals are one of the main outputs of universities. They are also the driving force of economic development (Florida et al., 2008). Therefore, one can say that the topics of human capital and technology transfer have recently been at the forefront of examining the efficiency of universities.

This rapid growth of the EE literature, especially in the last 10 years, may be an indicator of future growth. Therefore, there may be a need to review the EE literature periodically in the future. A more comprehensive review of the EE literature by expanding the databases used in this study and identifying research trends in the field can be seen as a way forward for future research.

The limitation of this research is the databases used. This study used Web of Science and SCOPUS, which are not used together in many bibliometric studies, together to cover the EE literature as broadly as possible. However, one can find studies on EE literature in other databases other than these two databases. These databases can also be included in future research. Secondly, although a comprehensive search strategy was used to reach all relevant publications, not all relevant publications could be reached in both databases. This may be due to the scope of the search strategy, the lack of titles, abstracts and keywords used by the authors in their publications. Bibliometric analysis studies need bibliometric data. WOS and Scopus are two large databases frequently used for bibliometric analyses. In this study, unlike many other studies, the scope of the research was expanded by using these two large databases together. However, this research does not claim that all studies on efficiency in education are included in the data set. Because there are many databases other than these two large databases. However, most of them do not provide bibliometric data. As a result, although the data set of this research is large, there may be many studies that are not included in the analysis.

When the results of this research are evaluated as a whole, the results show the following. Firstly, there is an increasing interest in efficiency in education. This situation aims to obtain the highest output from the investments made in education and the inputs used. Because, efficient use of educational resources prevents wastage of resources and maximises the expected economic and social returns from education. Secondly, this research has comprehensively analysed the literature on educational efficiency with long-term data. The results of this research provide a general picture of the field for researchers or early career researchers working on educational efficiency. The results of this study provided a holistic view of the field and enabled researchers specialising in educational efficiency to examine the current situation and draw future research directions.

Conclusion

Based on the Web of Science and SCOPUS databases, this article provides an overview of the EE literature from 1970 to 2022 through a total of 1315 documents. It also identified themes and trending topics in the field and addressed the expanding EE literature from a holistic perspective. The findings of this research revealed that the EE literature is growing at about 9% per year. The growth, especially in the last 10 years, covers 68% of the entire EE literature. The key publication in the field is Emrouznejad et al. (2008), which reviews 30 years of DEA literature. The key journal in the field is “Economics of Education Review”. The author who contributed the most to the field is Agasisti, T with 38 publications and an average of 28.2 citations per publication. The institution that contributed the most to the EE literature is Lancaster University in the United Kingdom. Of the 78 countries contributing to the field, China is the largest contributor and collaborator, followed by the USA. The most frequently recurring words are “DEA”, “efficiency”, and “higher education”. DEA, SFA, higher education, productivity, performance, technical efficiency, allocation efficiency, scale efficiency, and cost efficiency are the main themes of the field. Efficiency assessment based on the PISA exam is a new topic emerging in the field. In particular, innovation, technology transfer, and human capital are niche issues related to universities. Additionally, research efficiency, ranking, quality, school effectiveness, competition, e-learning, and European education efficiency research are among the trending topics.

Bibliographic references

Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22(1), 89-97. https://doi.org/10.1016/S0272-7757(01)00068-1

Acs, Z. J., Anselin, L., & Varga, A. (2002). Patents and innovation counts as measures of regional production of new knowledge. Research Policy, 31(7), 1069-1085. https://doi.org/10.1016/S0048-7333(01)00184-6

Adamides, E. D., & Karacapilidis, N. (2006). Information technology support for the knowledge and social processes of innovation management. Technovation, 26(1), 50-59. https://doi.org/10.1016/j.technovation.2004.07.019

Afonso, A., & Aubyn, M. S. (2006). Cross-country efficiency of secondary education provision: A semi-parametric analysis with non-discretionary inputs. Economic Modelling, 23(3), 476-491. https://doi.org/10.1016/j.econmod.2006.02.003

Agasisti, T. (2014). The efficiency of public spending on education: an empirical comparison of EU countries. European Journal of Education, 49(4), 543-557. https://doi.org/10.1111/ejed.12069

Agasisti, T., Barra, C., & Zotti, R. (2019a). Research, knowledge transfer, and innovation: The effect of Italian universities’ efficiency on local economic development 2006-2012. Journal of Regional Science, 59(5), 819-849. https://doi.org/10.1111/jors.12427

Agasisti, T., & Dal Bianco, A. (2009). Reforming the university sector: effects on teaching efficiency-evidence from Italy. Higher Education, 57(4), 477-498. https://doi.org/10.1007/s10734-008-9157-x

Agasisti, T., Munda, G., & Hippe, R. (2019b). Measuring the efficiency of European education systems by combining data envelopment analysis and multiple-criteria evaluation. Journal of Productivity Analysis, 51(2-3), 105-124. https://doi.org/10.1007/s11123-019-00549-6

Agasisti, T., & Zoido, P. (2018). Comparing the efficiency of schools through international benchmarking: Results from an empirical analysis of OECD PISA 2012 data. Educational Researcher, 47(6), 352-362. https://doi.org/10.3102/0013189x18777495

Anastasiou, A., Kounetas, K., Mitropoulos, P., & Mitropoulos, Y. (2007). Evaluating efficiency of university academic departments. Journal of Interdisciplinary Mathematics, 10(3), 409-431. https://doi.org/10.1080/09720502.2007.10700502

Andersson, C., Antelius, J., Mansson, J., & Sund, K. (2017). Technical efficiency and productivity for higher education institutions in Sweden. Scandinavian Journal of Educational Research, 61(2), 205-223. https://doi.org/10.1080/00313831.2015.1120230

Anderson, T. R., Daim, T. U., & Lavoie, F. F. (2007). Measuring the efficiency of university technology transfer. Technovation, 27(5), 306-318. https://doi.org/10.1016/j.technovation.2006.10.003

Ang, S., Zheng, R., Wei, F. Q., & Yang, F. (2021). A modified DEA-based approach for selecting preferred benchmarks in social networks. Journal of the Operational Research Society, 72(2), 342-353. https://doi.org/10.1080/01605682.2019.1671155

Anselin, L., Varga, A., & Acs, Z. (1997). Local geographic spillovers between university research and high technology innovations. Journal of Urban Economics, 42(3), 422-448. https://doi.org/10.1006/juec.1997.2032

Arbona, A., Giménez, V., López-Estrada, S., & Prior, D. (2022). Efficiency and quality in Colombian education: An application of the metafrontier Malmquist-Luenberger productivity index. Socio-Economic Planning Sciences, 79, 101122. https://doi.org/10.1016/j.seps.2021.101122

Aria, M., & Cuccurullo, C. (2017). Bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959-975. https://doi.org/10.1016/j.joi.2017.08.007

Aria, M., Cuccurullo, C., D’Aniello, L., Misuraca, M., & Spano, M. (2022). Thematic analysis as a new culturomic tool: The social media coverage on COVID-19 Pandemic in Italy. Sustainability, 14(6), 36-43. https://doi.org/10.3390/su14063643

Auranen, O., & Nieminen, M. (2010). University research funding and publication performance-An international comparison. Research Policy, 39(6), 822-834 https://doi.org/10.1016/j.respol.2010.03.003

Avkiran, N. K. (2001). Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socio-Economic Planning Sciences, 35(1), 57-80. https://doi.org/10.1016/S0038-0121(00)00010-0

Barra, C., & Zotti, R. (2017). Investigating the human capital development-growth nexus: Does the efficiency of universities matter? International Regional Science Review, 40(6), 638-678. https://doi.org/10.1177/0160017615626215

Barra, C., & Zotti, R. (2018). The contribution of university, private and public sector resources to Italian regional innovation system (in) efficiency. The Journal of Technology Transfer, 43(2), 432-457. https://doi.org/10.1007/s10961-016-9539-7

Bayley, T., Begen, M. A., Rodrigues, F. F., & Barrett, D. (2022). Relative efficiency of radiation treatment centers: an application of data envelopment analysis. Healthcare, 10(6). https://doi.org/10.3390/healthcare10061033

Bayraktar, E., Tatoglu, E., & Zaim, S. (2013). Measuring the relative efficiency of quality management practices in Turkish public and private universities. Journal of the Operational Research Society, 64(12), 1810-1830. https://doi.org/10.1057/jors.2013.2

Beasley, J. E. (1990). Comparing university departments. Omega, 18(2), 171-183 https://doi.org/10.1016/0305-0483(90)90064-G

Beasley, J. E. (1995). Determining teaching and research efficiencies. Journal of the Operational Research Society, 46(4), 441-452 https://doi.org/10.1057/jors.1995.63

Bessent, A. M., & Bessent, E. W. (1980). Determining the comparative efficiency of schools through data envelopment analysis. Educational Administration Quarterly, 16(2), 57-75. https://doi.org/10.1177/0013161X8001600207

Blau, F. D. (1996). Symposium on primary and secondary education. Journal of Economic Perspectives, 10(4), 3-8.

Broucker, B., De Wit, K., & Leisyte, L. (2015). An evaluation of new public management in higher education. Same rationale, different implementation. EAIR 37th Annual Forum, Krems.

Brzezicki, L. (2020). Efficiency of the university libraries in polish higher education. E-Mentor(1), 61-70. https://doi.org/10.15219/em83.1453

Canal, G. Y., Amado, A. P. G., & Hurtado, M. G. (2015). Research efficiency assessment of Colombian public universities 2003-2012: Data envelopment analysis. Inge Cuc, 11(2), 97-108. https://doi.org/10.17981/ingecuc.11.2.2015.10

Chang, T. Y. (2013). Enhancing e-learning management systems to promoting the management efficiency of tourism and hospitality education. The Anthropologist, 16(3), 473-485. https://doi.org/10.1080/09720073.2013.11891373

Chang, Y. T., Park, H., Lee, S., & Kim, E. (2018). Have Emission Control Areas (ECAs) harmed port efficiency in Europe? Transportation Research Part D-Transport and Environment, 58, 39-53. https://doi.org/10.1016/j.trd.2017.10.018

Chao, S. L., Yu, M. M., & Hsieh, W. F. (2018). Evaluating the efficiency of major container shipping companies: A framework of dynamic network DEA with shared inputs. Transportation Research Part a-Policy and Practice, 117, 44-57. https://doi.org/10.1016/j.tra.2018.08.002

Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision-making units. European Journal of Operational Research, 3(4), 339-338.

Chen, Z. F., Matousek, R., & Wanke, P. (2018). Chinese bank efficiency during the global financial crisis: A combined approach using satisficing DEA and Support Vector Machines. North American Journal of Economics and Finance, 43, 71-86. https://doi.org/10.1016/j.najef.2017.10.003

Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). An approach for detecting, quantifying, and visualizing the evolution of a research field: A practical application to the Fuzzy Sets Theory field. Journal of Informetrics, 5(1), 146-166. https://doi.org/10.1016/j.joi.2010.10.002

Çetin, M., & Maral, M. (2022). An analysis of research on the efficiency of higher education in Türkiye. Educational Policy Analysis and Strategic Research, 17(4), 8-44. https://doi.org/10.29329/epasr.2022.478.1

De Clercq, D., Wen, Z. G., & Fei, F. (2019). Determinants of efficiency in anaerobic bio-waste co-digestion facilities: A data envelopment analysis and gradient boosting approach. Applied Energy, 253. https://doi.org/10.1016/j.apenergy.2019.113570

De Witte, K., & López-Torres, L. (2017). Efficiency in education: A review of literature and a way forward. Journal of the Operational Research Society, 68(4), 339-363. https://doi.org/10.1057/jors.2015.92

Delprato, M., & Antequera, G. (2021). School efficiency in low and middle income countries: An analysis based on PISA for development learning survey. International Journal of Educational Development, 80, 1-13. https://doi.org/10.1016/j.ijedudev.2020.102296

Dufrechou, P. A. (2016). The efficiency of public education spending in Latin America: A comparison to high-income countries. International Journal of Educational Development, 49, 188-203. https://doi.org/10.1016/j.ijedudev.2016.03.005

Dunning, J. H. (2000). Regions, globalization, and the knowledge economy: the issues stated. In J. H. Dunning (Ed.), Regions, globalization, and the knowledge-based economy (pp. 7-41). Oxford Academic.

Emrouznejad, A., Parker, B. R., & Tavares, G. (2008). Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. Socio-Economic Planning Sciences, 42(3), 151-157. https://doi.org/10.1016/j.seps.2007.07.002

Etzkowitz, H., & Leydesdorff, L. (1995). The Triple Helix-University-industry-government relations: A laboratory for knowledge based economic development. EASST Review, 14(1), 14-19.

Feller, I., Ailes, C. P., & Roessner, J. D. (2002). Impacts of research universities on technological innovation in industry: evidence from engineering research centers. Research Policy, 31(3), 457-474. https://doi.org/10.1016/S0048-7333(01)00119-6

Ferro, G., & D’Elia, V. (2020). Higher education efficiency frontier analysis: A review of variables to consider. Journal on Efficiency and Responsibility in Education and Science, 13(3), 140-153.

Florida, R., Mellander, C., & Stolarick, K. (2008). Inside the black box of regional development-human capital, the creative class and tolerance. Journal of Economic Geography, 8(5), 615-649. https://doi.org/10.1093/jeg/lbn023

Goldstein, H., & Woodhouse, G. (2000). School effectiveness research and educational policy. Oxford Review of Education, 26(3-4), 353-363. https://doi.org/10.1080/713688547

Gomez-Gallego, J. C., Gomez-Gallego, M., Garcia-Garcia, J. F., & Faura-Martinez, U. (2021). Evaluation of the efficiency of European health systems using fuzzy data envelopment analysis. Healthcare, 9(10). https://doi.org/10.3390/healthcare9101270

Gong, L., Liu, Z. Y., Zhang, S. Q., & Jiang, Z. H. (2022). Does open innovation promote innovation efficiency in Chinese universities? Ieee Transactions on Engineering Management, in press, 1-14. https://doi.org/10.1109/tem.2021.3140116

Gralka, S., Wohlrabe, K., & Bornmann, L. (2019). How to measure research efficiency in higher education? Research grants vs. publication output. Journal of Higher Education Policy and Management, 41(3), 322-341. https://doi.org/10.1080/1360080x.2019.1588492

Hallinger, P. (2021). Tracking the evolution of the knowledge base on problem-based learning: A bibliometric review, 1972-2019. Interdisciplinary Journal of Problem-Based Learning, 15(1). https://doi.org/10.14434/ijpbl.v15i1.28984

Holmgren, J. (2018). The effects of using different output measures in efficiency analysis of public transport operations. Research in Transportation Business and Management, 28, 12-22. https://doi.org/10.1016/j.rtbm.2018.02.006

Jiang, J. L., Lee, S. K., & Rah, M. J. (2020). Assessing the research efficiency of Chinese higher education institutions by data envelopment analysis. Asia Pacific Education Review, 21(3), 423-440. https://doi.org/10.1007/s12564-020-09634-0

Johnes, J. (2004). Efficiency measurement. In G. Johnes & J. Johnes (Eds.), The International Handbook on the Economics of Education (pp. 613–742). Edward Elgar.

Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education, Economics of Education Review, 25(3), 273-288. https://doi.org/0.1016/j.econedurev.2005.02.005

Johnes, J., Portela, M., & Thanassoulis, E. (2017). Efficiency in education. J Oper Res Soc, 68(4), 331-338. https://doi.org/10.1057/s41274-016-0109-z

Kalb, A. (2010). Public sector efficiency: applications to local governments in Germany. Gabler Verlag

Kalirajan, K. P., & Shand, R. T. (1999). Frontier production functions and technical efficiency measures. Journal of Economic Surveys, 13(2), 149-172.

Kantabutra, S., & Tang, J. C. S. (2010). Efficiency analysis of public universities in Thailand. Tertiary Education and Management, 16(1), 15-33. https://doi.org/10.1080/13583881003629798

Kao, C. A., & Hung, H. T. (2008). Efficiency analysis of university departments: An empirical study. Omega-International Journal of Management Science, 36(4), 653-664. https://doi.org/10.1016/j.omega.2006.02.003

Khodadadipour, M., Hadi-Vencheh, A., Behzadi, M. H., & Rostamy-Malkhalifeh, M. (2021). Undesirable factors in stochastic DEA cross-efficiency evaluation: An application to thermal power plant energy efficiency. Economic Analysis and Policy, 69, 613-628. https://doi.org/10.1016/j.eap.2021.01.013

Kupriyanova, V., Estermann, T., & Sabic, N. (2018). Efficiency of universities: Drivers, enablers and limitations. In European higher education area: The impact of past and future policies (pp. 603-618). Springer, Cham.

Lampe, H. W., & Hilgers, D. (2015). Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA. European Journal of Operational Research, 240(1), 1-21. https://doi.org/10.1016/j.ejor.2014.04.041

Lee, B. L., & Johnes, J. (2022). Using network DEA to inform policy: The case of the teaching quality of higher education in England. Higher Education Quarterly, 76(2), 399-421. https://doi.org/10.1111/hequ.12307

Linnenluecke, M. K., Marrone, M., & Singh, A. K. (2020). Conducting systematic literature reviews and bibliometric analyses. Australian Journal of Management, 45(2), 175-194. https://doi.org/10.1177/031289621987767

Liu, F., & Xu, H. (2017). Effects of educational efficiency on national competitiveness based on cross-national data. Education Sciences, 7(4), 81. https://doi.org/10.3390/educsci7040081

Lu, K., & Wolfram, D. (2012). Measuring author research relatedness: A comparison of word-based, topic-based, and author cocitation approaches. Journal of the American Society for Information Science and Technology, 63(10), 1973-1986. https://doi.org/10.1002/asi.22628

Maral, M. (2023). Examination of research efficiency of research universities in Türkiye by data envelopment analysis. International Journal of Educational Management, Advance online publication. https://doi.org/10.1108/IJEM-04-2023-0194

Mikušová, P. (2017). Measuring the efficiency of the Czech public higher education institutions: an application of DEA. Journal on Efficiency and Responsibility in Education and Science, 10(2), 58-63. https://doi.org/10.7160/eriesj.2017.100204

Moreno-Enguix, M. D., Gomez-Gallego, J. C., & Gallego, M. G. (2018). Analysis and determination the efficiency of the European health systems. International Journal of Health Planning and Management, 33(1), 136-154. https://doi.org/10.1002/hpm.2412

Moutinho, V., Madaleno, M., & Macedo, P. (2020). The effect of urban air pollutants in Germany: eco-efficiency analysis through fractional regression models applied after DEA and SFA efficiency predictions. Sustainable Cities and Society, 59, 1-11. https://doi.org/10.1016/j.scs.2020.102204

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., & Brennan, S. E. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews, 10(1), 1-11. https://doi.org/10.1186/s13643-021-01626-4

Park, Y. S., Lim, S. H., Egilmez, G., & Szmerekovsky, J. (2018). Environmental efficiency assessment of US transport sector: A slack-based data envelopment analysis approach. Transportation Research Part D-Transport and Environment, 61, 152-164. https://doi.org/10.1016/j.trd.2016.09.009

Perkmann, M., Tartari, V., McKelvey, M., Autio, E., Broström, A., D’este, P., Fini, R., Geuna, A., Grimaldi, R., & Hughes, A. (2013). Academic engagement and commercialisation: A review of the literature on university–industry relations. Research Policy, 42(2), 423-442. https://doi.org/10.1016/j.respol.2012.09.007

Peters, M. (1992). Performance and accountability in ‘post-industrial society’s: The crisis of British universities. Studies in Higher Education, 17(2), 123-139.

Quintano, C., Mazzocchi, P., & Rocca, A. (2021). Evaluation of the eco-efficiency of territorial districts with seaport economic activities. Utilities Policy, 71. https://doi.org/10.1016/j.jup.2021.101248

Reichmann, G. (2004). Measuring university library efficiency of using data envelopment analysis. Libri, 54(2), 136-146. https://doi.org/10.1515/libr.2004.136

Rhaiem, M. (2017). Measurement and determinants of academic research efficiency: A systematic review of the evidence. Scientometrics, 110(2), 581-615.

Saravanan, P., Rajeswari, S., Kumar, J. A., Rajasimman, M., & Rajamohan, N. (2022). Bibliometric analysis and recent trends on MXene research–A comprehensive review. Chemosphere, 286, 1-14. https://doi.org/10.1016/j.chemosphere.2021.131873

Shamohammadi, M., & Oh, D. H. (2019). Measuring the efficiency changes of private universities of Korea: A two-stage network data envelopment analysis. Technological Forecasting and Social Change, 148, 1-15. https://doi.org/10.1016/j.techfore.2019.119730

Shi, X., Wu, Y., & Fu, D. (2020). Does University-industry collaboration improve innovation efficiency? Evidence from Chinese firms. Economic Modelling, 86, 39-53. https://doi.org/10.1016/j.econmod.2019.05.004

Sweileh, W. M. (2018). Research trends on human trafficking: A bibliometric analysis using Scopus database. Globalization and Health, 14(1), 1-12. https://doi.org/10.1186/s12992-018-0427-9

Tan, G. K. R. (2006). Efficiency estimates for Singapore manufacturing: new evidence from the Malmquist index. Applied Economics Letters, 13(11), 715-721. https://doi.org/10.1080/13504850500401924

Tavares, R. S., Drumond, G. M., Angula Meza, L., & Mexas, M. P. (2018). Efficiency assessment in university libraries. Transinformacao, 30(1), 65-79. https://doi.org/10.1590/2318-08892018000100006

Tran, P. P., Kuo, K. C., Lu, W. M., & Kweh, Q. L. (2020). Benchmarking in Vietnam universities: teaching and research and revenue efficiencies. Asia Pacific Education Review, 21(2), 197-209. https://doi.org/10.1007/s12564-019-09622-z

Turkan, S., & Ozel, G. (2017). Efficiency of state universities in Turkey During the 2014-2015 academic year and determination of factors affecting efficiency. Education and Science, 42(191), 307-322. https://doi.org/10.15390/eb.2017.6980

Thursby, J. G., & Kemp, S. (2002). Growth and productive efficiency of university intellectual property licensing. Research Policy, 31(1), 109-124. https://doi.org/10.1016/S0048-7333(00)00160-8

Van Eck, N. J., & Waltman, L. (2014). Visualizing bibliometric networks. In Y. Ding, R. Rousseau, & D. Wolfram (Eds.), Measuring scholarly impact (pp. 285-320). Springer.

Venkadasalam, S., Mohamad, A., & Sifat, I. M. (2020). Operational efficiency of shipping companies Evidence from Malaysia, Singapore, the Philippines, Thailand and Vietnam. International Journal of Emerging Markets, 15(5), 875-897. https://doi.org/10.1108/ijoem-07-2019-0493

Vidoli, F., & Ferrara, G. (2015). Analyzing Italian citrus sector by semi-nonparametric frontier efficiency models. Empirical Economics, 49(2), 641-658. https://doi.org/10.1007/s00181-014-0879-6

Villano, R. A., & Tran, C. D. T. (2021). Survey on technical efficiency in higher education: A meta-fractional regression analysis. Pacific Economic Review, 26(1), 110-135. https://doi.org/10.1111/1468-0106.12310

Visbal-Cadavid, D., Martinez-Gomez, M., & Guijarro, F. (2017). Assessing the efficiency of public universities through DEA. A Case Study. Sustainability, 9(8), 1-19. https://doi.org/10.3390/su9081416

Witte, K. D., & López-Torres, L. (2017). Efficiency in education: a review of literature and a way forward. Journal of the Operational Research Society, 68(4), 339-363. https://doi.org/10.1057/jors.2015.92

Worthington, A. C. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics, 9(3), 245-268. https://doi.org/10.1080/09645290110086126

Xie, H., Zhang, Y., Wu, Z., & Lv, T. (2020). A bibliometric analysis on land degradation: Current status, development, and future directions. Land, 9(1), 1-37. https://doi.org/10.3390/land9010028

Xu, H. Q., Chung, C.-C., & Yu, C. (2022). Visualizing research trends on culture Neuroscience (2008–2021): A bibliometric analysis. Frontiers in Psychology, 1-14. https://doi.org/10.3389/fpsyg.2022.884929

Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429-472. https://doi.org/10.1177/1094428114562629

Appendix

Web of Science Search Strategy

(TI=(efficienc*) or AK=(efficienc*)) and (TI=(universit* or scholl or college or educat* or tertiary or master or doctor* or bachelor or graduate or phd or academ* or facult* or teach* or student or pupil or department) or AK=(universit* or scholl or college or educat* or tertiary or master or doctor* or bachelor or graduate or phd or academ* or facult* or teach* or student or pupil or department)) AND (AK=(“data envelopment” or “DEA” or “stochastic frontier” or “SFA” or Malmquist or “frontier efficienc*” or tobit or “allocative efficienc*” or “technical efficienc*” or “cost efficienc*” or “scale efficienc*” or “economic efficienc*” or “profit efficienc*” or “constant return” or “CCR” or “BCC” or “variable return” or VRS or “decreasing return” or DRS or “non increasing return” or “input oriented” or “output oriented” or “two stage” or “Cobb Douglas” or “Ordinary Least Squares” or “OLS” or “Corrected Ordinary Least Squares” or COLS or “Modified Ordinary Least Squares” or “MOLS” or “Maximum Likelihood Estimation” or “MLE” or “cross sectional” or “panel data” or “fixed effect” or “random effect” or “deterministic frontier” or “slack* based” or “free disposal hull” or “free disposable hull” or “FDH” or “dynamic ordinary least square” or “DOLS” or “fully modified ordinary least square” or “FMOLS” or “super efficienc*” or Benchmark* or bootstrap or “decision making unit” or “frontier seperation approach” or “multi stage approach” or input or output or productivit* or “technical change” or “total factor productivity” or “TFP” or “directional distance” or “cost direct” or “cost indirect” or “output distance” or tobit or “order m” or “conditional efficienc*” or “output efficienc*” or “meta frontier” or “joint production” or “hybrid return” or “cost frontier” or translog or “discretionary” or “non discretionary” or probabilistic or “relative efficienc*” or “research efficienc*” or “parametric frontier” or “non parametric frontier” or “parametric approach” or “non parametric approach” or “directional distance function” or “technology distance function”) OR AB=(“data envelopment” or “DEA” or “stochastic frontier” or “SFA” or Malmquist or “frontier efficienc*” or tobit or “allocative efficienc*” or “technical efficienc*” or “cost efficienc*” or “scale efficienc*” or “economic efficienc*” or “profit efficienc*” or “constant return” or “CCR” or “BCC” or “variable return” or VRS or “decreasing return” or DRS or “non increasing return” or “input oriented” or “output oriented” or “two stage” or “Cobb Douglas” or “Ordinary Least Squares” or “OLS” or “Corrected Ordinary Least Squares” or COLS or “Modified Ordinary Least Squares” or “MOLS” or “Maximum Likelihood Estimation” or “MLE” or “cross sectional” or “panel data” or “fixed effect” or “random effect” or “deterministic frontier” or “slack* based” or “free disposal hull” or “free disposable hull” or “FDH” or “dynamic ordinary least square” or “DOLS” or “fully modified ordinary least square” or “FMOLS” or “super efficienc*” or Benchmark* or bootstrap or “decision making unit” or “frontier seperation approach” or “multi stage approach” or input or output or productivit* or “technical change” or “total factor productivity” or “TFP” or “directional distance” or “cost direct” or “cost indirect” or “output distance” or tobit or “order m” or “conditional efficienc*” or “output efficienc*” or “meta frontier” or “joint production” or “hybrid return” or “cost frontier” or translog or “discretionary” or “non discretionary” or probabilistic or “relative efficienc*” or “research efficienc*” or “parametric frontier” or “non parametric frontier” or “parametric approach” or “non parametric approach” or “directional distance function” or “technology distance function”) OR TI=(“data envelopment” or “DEA” or “stochastic frontier” or “SFA” or Malmquist or “frontier efficienc*” or tobit or “allocative efficienc*” or “technical efficienc*” or “cost efficienc*” or “scale efficienc*” or “economic efficienc*” or “profit efficienc*” or “constant return” or “CCR” or “BCC” or “variable return” or VRS or “decreasing return” or DRS or “non increasing return” or “input oriented” or “output oriented” or “two stage” or “Cobb Douglas” or “Ordinary Least Squares” or “OLS” or “Corrected Ordinary Least Squares” or COLS or “Modified Ordinary Least Squares” or “MOLS” or “Maximum Likelihood Estimation” or “MLE” or “cross sectional” or “panel data” or “fixed effect” or “random effect” or “deterministic frontier” or “slack* based” or “free disposal hull” or “free disposable hull” or “FDH” or “dynamic ordinary least square” or “DOLS” or “fully modified ordinary least square” or “FMOLS” or “super efficienc*” or Benchmark* or bootstrap or “decision making unit” or “frontier seperation approach” or “multi stage approach” or input or output or productivit* or “technical change” or “total factor productivity” or “TFP” or “directional distance” or “cost direct” or “cost indirect” or “output distance” or tobit or “order m” or “conditional efficienc*” or “output efficienc*” or “meta frontier” or “joint production” or “hybrid return” or “cost frontier” or translog or “discretionary” or “non discretionary” or probabilistic or “relative efficienc*” or “research efficienc*” or “parametric frontier” or “non parametric frontier” or “parametric approach” or “non parametric approach” or “directional distance function” or “technology distance function”))

SCOPUS Search Strategy

(TITLE(efficienc*) OR KEY(efficienc*)) AND (TITLE(universit* or scholl or college or educat* or tertiary or master or doctor* or bachelor or graduate or phd or academ* or facult* or teach* or student or pupil or department) OR KEY(universit* or scholl or college or educat* or tertiary or master or doctor* or bachelor or graduate or phd or academ* or facult* or teach* or student or pupil or department)) AND (TITLE-ABS-KEY(“data envelopment” or “DEA” or “stochastic frontier” or “SFA” or Malmquist or “frontier efficienc*” or tobit or “allocative efficienc*” or “technical efficienc*” or “cost efficienc*” or “scale efficienc*” or “economic efficienc*” or “profit efficienc*” or “constant return” or “CCR” or “BCC” or “variable return” or VRS or “decreasing return” or DRS or “non increasing return” or “input oriented” or “output oriented” or “two stage” or “Cobb Douglas” or “Ordinary Least Squares” or “OLS” or “Corrected Ordinary Least Squares” or COLS or “Modified Ordinary Least Squares” or “MOLS” or “Maximum Likelihood Estimation” or “MLE” or “cross sectional” or “panel data” or “fixed effect” or “random effect” or “deterministic frontier” or “slack* based” or “free disposal hull” or “free disposable hull” or “FDH” or “dynamic ordinary least square” or “DOLS” or “fully modified ordinary least square” or “FMOLS” or “super efficienc*” or Benchmark* or bootstrap or “decision making unit” or “frontier seperation approach” or “multi stage approach” or input or output or productivit* or “technical change” or “total factor productivity” or “TFP” or “directional distance” or “cost direct” or “cost indirect” or “output distance” or tobit or “order m” or “conditional efficienc*” or “output efficienc*” or “meta frontier” or “joint production” or “hybrid return” or “cost frontier” or translog or “discretionary” or “non discretionary” or probabilistic or “relative efficienc*” or “research efficienc*” or “parametric frontier” or “non parametric frontier” or “parametric approach” or “non parametric approach” or “directional distance function” or “technology distance function”)) AND ( EXCLUDE ( SUBJAREA,”MEDI” ) OR EXCLUDE ( SUBJAREA,”PHYS” ) OR EXCLUDE ( SUBJAREA,”ENER” ) OR EXCLUDE ( SUBJAREA,”ENVI” ) OR EXCLUDE ( SUBJAREA,”NURS” ) OR EXCLUDE ( SUBJAREA,”MATE” ) OR EXCLUDE ( SUBJAREA,”HEAL” ) OR EXCLUDE ( SUBJAREA,”BIOC” ) OR EXCLUDE ( SUBJAREA,”AGRI” ) OR EXCLUDE ( SUBJAREA,”DENT” ) OR EXCLUDE ( SUBJAREA,”EART” ) OR EXCLUDE ( SUBJAREA,”NEUR” ) OR EXCLUDE ( SUBJAREA,”CENG” ) OR EXCLUDE ( SUBJAREA,”PHAR” ) OR EXCLUDE ( SUBJAREA,”CHEM” ) OR EXCLUDE ( SUBJAREA,”IMMU” ) OR EXCLUDE ( SUBJAREA,”VETE” ) OR EXCLUDE ( SUBJAREA,”Undefined” ) ) AND ( EXCLUDE ( PUBYEAR,2023) )

Contact address: Konaklar District, National Defense University, Beşiktaş, Istanbul, Turkey. E-mail: muammermaral@hotmail.com