Situational judgment test of basic employability competences development: validation of a psychoeducative instrument for socioeducative intervention

Keywords: employability, basic employability skills, key employability skills, situational test, social pedagogy

Abstract

Basic employability competences, also known as key competences, are essential in accessing, retaining and advancing in employment, and are especially relevant in promoting the employment of disadvantaged young people. This study describes the development of a copyleft psychoeducational instrument based on the model posited by Arnau et al. (2014) that Social Pedagogy professionals can use to evaluate the basic employability competences in adolescents and young people aged 12 to 18: The Situational Test of Basic Employability Competences Development (DCBE). A total of 29 problem situations were designed and included within five stories related to the daily life of any adolescent. A total of 102 adolescents and young people were interviewed and asked how they would react to these problem situations. The responses were subjected to a content analysis in order to select the most frequent responses indicative of different levels of competence development. Ten experts then assigned the problem situations to one of the eight competences proposed by Arnau et al. (2014), before assigning scores of 0 to 5 to the different responses based on the level of development indicated by the responses, with high levels of agreement. Following a pilot study on a group of 47 young people to verify the variability of the responses, the test was administered to 1,011 young people at six training centres. A factor analysis of the responses yielded a unifactorial structure of 23 items, with high values for model fit and reliability. The results obtained support the content, ecological and construct validity of the DCBE test, which was well accepted by both students and teachers for its attractiveness and usefulness. The test makes it possible to begin preparing students for employment in the early stages prior to post-compulsory training. Its copyleft nature means it can be widely used by Social Pedagogy professionals.

Author Biographies

Josefina Sala-Roca, Universitat Autònoma de Barcelona

Profesora del departamento de Teories de l’Educació y Pedagogia Social de la Universidad Autónoma de Barcelona. Coordina el grupo de Investigación de Infancia y Adolescencia en Riesgo Social (IARS). El grupo IARS en colaboración con FEPA, FEDAIA y DGAIA ha desarrollado diversos estudios sobre los factores que inciden en el bienestar y desarrollo positivo de los jóvenes tutelados, así como de las dificultades y apoyos a su transición a la vida independiente. Las investigaciones desarrolladas se han publicado en diversos artículos e informes.

Mercè Jariot-Garcia, Universitat Autònoma de Barcelona

Profesora del departamento de Pedagogía Aplicada de la Universidad Autónoma de Barcelona. Forma parte del grupo de Investigación de Infancia y Adolescencia en Riesgo Social (IARS). Sus líneas de investigación se relacionan con la transición de los jóvenes ex tutelados a la vida adulta, la inserción socio laboral, el cambio de actitudes y la mejora y desarrollo de las competencias de empleabilidad.

Laura Arnau-Sabatés, Universitat Autònoma de Barcelona

Profesora del departamento de Teorías de la Educación y Pedagogía Social de la Universidad Autónoma de Barcelona. Forma parte del grupo de Investigación de Infancia y Adolescencia en Riesgo Social (IARS). Sus líneas de investigación se relacionan con la protección a la infancia tutelada, la transición de los jóvenes ex tutelados a la vida adulta, la inserción socio laboral y la mejora y desarrollo de las competencias de empleabilidad.

Eduardo Doval, Universitat Autònoma de Barcelona

Profesor del departamento de Psicobiología y de Metodología de las Ciencias de la Salud de la Facultad de Psicología de la Universidad Autónoma de Barcelona. Miembro del grupo de investigación en estrés y salud (GIES). Ha desarrollado diversos estudios relacionados con la adaptación de instrumentos de medida y con la evaluación de la calidad métrica de los instrumentos de medida, en especial con la evaluación de patrones atípicos de respuesta. Sus investigaciones se han publicado en diversos artículos, libros e informes técnicos.

References

Abas, M. C., & Imam, O. A. (2016). Graduates' Competence on Employability Skills and Job Performance. International Journal of Evaluation and Research in Education, 5(2), 119-125.

Alonso Bello, E.; Santana Vega, L.E., & Feliciano García, L. (2018). Trayectorias de empleabilidad de los jóvenes que abandonan el sistema de protección. Educación Siglo XXI, 36(3), 485-504. https://doi.org/10.6018/j/350101

Arnau-Sabatés, L., Marzo, M. T., Jariot, M., & Sala-Roca, J. (2014). Learning basic employability competence: a challenge for the active labour insertion of adolescents in residential care in their transition to adulthood. European Journal of Social Work, 17(2), 252-265. https://doi.org/10.1080/13691457.2013.802227

Blalock, L. B., Strieter, L., & Hughes, L. (2006). The SCANS Skills and Competencies Checklist: An assessment tool for youth work readiness programs. Journal of Youth Development, 1(1), 89-99.

Breen, R. (2005). Explaining cross-national variation in youth unemployment market and institutional factors. European sociological review, 21(2), 125-134. https://doi.org/10.1093/esr/jci008

Bridgstock, R. (2009). The graduate attributes we’ve overlooked: Enhancing graduate employability through career management skills. Higher Education Research & Development, 28(1), 31-44. https://doi.org/10.1080/07294360802444347

Cabrera, Lanzo, N., López López, M.C., & Portillo Vidiella, M.C. (2015). La percepción de los empleadores acerca de las competencias de los graduados y su evaluación. Una aproximación a distintos sectores profesionales. En Londoño Orozco, Guillermo and Cano García, Elena: Formación y evaluación por competencias en educación superior. (185-213). https://doi.org/10.19052/9789588844992

Cimatti, B. (2016). Definition, development, assessment of soft skills and their role for the quality of organizations and enterprises. International Journal for quality research, 10(1). https://doi.org.10.18421/IJQR10.01-05

Cinque, M. (2016). “Lost in translation”. Soft skills development in European countries. Tuning Journal for Higher Education, 3(2), 389-427. http://dx.doi.org/10.18543/tjhe-3(2)-2016pp389-427

Clarke, M. (2016). Addressing the soft skills crisis. Strategic HR Review, 15(3), 137-139. https://doi.org/10.1108/SHR-03-2016-0026

Crisp, R., & Powell, R. (2016). Young people and UK labour market policy: A critique of ‘employability’ as a tool for understanding youth unemployment. Urban Studies, 54(8), 1784-1807. https://doi.org/10.1177/0042098016637567

Curtis, D., & McKenzie, P. (2001). Employability skills for Australian industry: Literature review and framework development. Report to: Business Council of Australia; Australian Chamber of Commerce and Industry. Science and Training, Canberra. http://hdl.voced.edu.au/10707/40939.

Dreger, C., López-Bazo, E., Ramos, R., Royuela, V., & Suriñach, J. (2015). Wage and income inequality in the European Union. Policy Department. Retrieved from http://ec.europa.eu/eurostat/cros/system/files/05-2014-wage_and_income_inequality_in_the_eu_0.pdf

Eurostat (2020). Unemployement statistics. http://ec.europa.eu/eurostat/statistics-explained/index.php/Unemployment_statistics

Fleiss, J. L. (1981) Statistical methods for rates and proportions. 2nd ed. (New York: John Wiley) pp. 38–46.

Frías-Navarro, D., & Pascual Soler, M. (2012). Exploratory factor Analysis (EFA) in consumer behavior and marketing research. Suma Psicológica, 19(1), 47-58. Disponible en: http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0121-43812012000100004

Gibb, S. (2014). Soft skills assessment: theory development and the research agenda. International Journal of Lifelong Education, 33(4), 455-471. https://doi.org/10.1080/02601370.2013.867546

Griffiths, D. A., Inman, M., Rojas, H., & Williams, K. (2018). Transitioning student identity and sense of place: future possibilities for assessment and development of student employability skills. Studies in Higher Education, 43(5), 891-913.

Gulikers, J. T., Baartman, L., & Biemans, H. J.(2010). Facilitating evaluations of innovative, competence-based assessments: Creating understanding and involving multiple stakeholders. Evaluation and Program Planning, 33(2), 120-127. https://doi.org/10.1016/j.evalprogplan.2009.07.002

Jariot, M.; Sala, J., Arnau, L. & Marzo, M. (2021). Validación del modelo teórico del Test Situacional de Desarrollo de las Competencias Básicas de Empleabilidad: la perspectiva de los trabajadores. Revista de Pedagogía Social, 37, XXX.

Kvavilashvili, L., & Ellis, J. (2004). Ecological validity and the real-life/laboratory controversy in memory research: A critical and historical review. History & Philosophy of Psychology, 6, 59–80. Disponible en: https://uhra.herts.ac.uk/bitstream/handle/2299/1971/103125.pdf?

Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37(4), 426-441. https://doi.org/10.1108/00483480810877598

Lo Presti, A. (2009). Snakes and ladders: Stressing the role of meta-competencies for post-modern careers. International Journal for Educational and Vocational Guidance, 9(2), 125–134. https://doi.org/10.1007/s10775-009-9157-0

Lo Presti, A. L., & Pluviano, S. (2016). Looking for a route in turbulent waters Employability as a compass for career success. Organizational Psychology Review, 6(2), 192-211. https://doi.org/10.1177/2041386615589398

Lotito, F. (2015). Test Psicológicos y Entrevistas: Usos y Aplicaciones Claves En El Proceso De Selección E Integración De Personas a Las Empresas (Psychological Tests and Interviews: Key Uses and Applications in the Selection and Integration of Companies). RAN-Revista Academia & Negocios, 1(2).

Lorenzo-Seva, U., & Ferrando, P. (2007). FACTOR: A computer program to fit the exploratory factor analysis model. University Rovira y Virgili. Disponible en: https://link.springer.com/content/pdf/10.3758/BF03192753.pdf

Lorenzo-Seva, U., & Ferrando, P. (2011). Manual of the program FACTOR v. 8.02. Disponible en: https://psico.fcep.urv.es/utilitats/factor/Manual-of-the-FactorProgram.pdf

MacCann, C., & Roberts, R. D. (2008). New paradigms for assesing emotional intelligence: theory and data. Emotion, 8(4), 540–551. https://doi.org/10.1037/a0012746

McQuaid, R. W. (2006). Job search success and employability in local labor markets. The Annals of Regional Science, 40(2), 407-421. https://doi.org/10.1007/s00168-006-0065-7

McQuaid, R. W., & Lindsay, C. (2005). The concept of employability. Urban Studies, 42(2), 197–219. https://doi.org/10.1007/s00168-006-0065-7

Moreno Mínguez, A. (2015). La empleabilidad de los jóvenes en España. Explicando el elevado desempleo juvenil durante la recesión económica. Revista Internacional de Investigación en Ciencias Sociales, 11(3), 3-20. https://doi.org/10.18004/riics.2015.julio.3-20

Muhamad, S. (2012). Graduate employability and transferable skills: A review. Advances in Natural and Applied Sciences, 6(6), 882-885. Disponible en: https://www.researchgate.net/profile/Suriyani_Muhamad/publication/287021703_Graduate_employability_and_transferable_skills_A_review/links/5da8703492851caa1babd804/Graduate-employability-and-transferable-skills-A-review.pdf

Nickson, D., Warhurst, C., Commander, J., Hurrell, S. A., & Cullen, A. M. (2012). Soft skills and employability: Evidence from UK retail. Economic and Industrial Democracy, 33(1), 65-84. https://doi.org/10.1177/0143831x11427589

Nisha, S. M., & Rajasekaran, V. (2018). Employability skills: A review. IUP Journal of Soft Skills, 12(1), 29-37. Disponible en. http://search.proquest.com/openview/1f9f5ad8e5798a6b992e4c1bee830cbb/1?pq-origsite=gscholar&cbl=2029989&casa_token=0GFchuXN-2gAAAAA:pkjjLEAu-RD1CNJ10XkoDJTVjnjZ90KmLE4gqWIheZebJKGfq_YJw8j4lZkuzxps_j6kIMXLrJ4

Olea, M. A. (2001). El trabajo como bien escaso. Revista del Ministerio de Trabajo e Inmigración, 33, 17-32.

Olmos-Rueda, P., & Mas-Torelló, O. (2017). Perspectiva de tutores y de empresas sobre el desarrollo de las competencias básicas de empleabilidad en el marco de los programas de formación profesional básica. Educar, 53(2), 261-284. DOI: https://doi.org/10.5565/rev/educar.870

Organización Internacional del Trabajo (2014). Competencias para el empleo. Mejorar la empleabilidad de los jóvenes: La importancia de las competencias clave. Recuperado en http://www.ilo.org/skills/pubs/WCMS_371815/lang--es/index.htm

Riebe, L., & Jackson, D. (2014). The use of rubrics in benchmarking and assessing employability skills. Journal of Management Education, 38(3), 319-344.

Rosenberg, S., Heimler, R., & Morote, E. S. (2012). Basic employability skills: a triangular design approach. Education+ Training. https://doi.org/10.1108/00400911211198869

Rothwell, A., Herbert, I., & Rothwell, F. (2008). Self-perceived employability: Construction and initial validation of a scale for university students. Journal of vocational behavior,73(1), 1-12.

Sala-Roca, J., Arnau, L., Jariot, M., Marzo, T., Urrea, A. & Ortega, D. (2020). Test Situacional de Desarrollo de Competencias Básicas de Empleabilidad (DCBE). Versión revisada de J. Sala-Roca, L. Arnau, M. Jariot, T. Marzo. The Big Eight Test. DDD. https://ddd.uab.cat/record/213070

Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and Reliability of Situational Judgement Test Scores A New Approach Based on Cognitive Diagnosis Models. Organizational Research Methods, 19(3), 506-532. https://doi.org/10.1177/1094428116630065

Webster, E. S., Paton, L. W., ES Crampton, P., & Tiffin, P. A. (2020). Situational judgement test validity for selection: a systematic review and meta‐analysis. Medical Education. https://doi.org/10.1111/medu.14201

Published
2021-01-18