Measuring Social, Emotional and Intercultural/Transcultural Competencies in European Context: Insights from the Policy Experimentation Project HAND IN HAND

By:
Ana Kozina, Educational Research Institute, Ljubljana, Slovenia
Nina Roczen, DIPF | Leibniz Institute for Research and Information in Education, Frankfurt, Germany
Albert Denk, Technical University Munich, Munich, Germany

“HAND in HAND: Social and Emotional Skills for Tolerant and Non-discriminative Societies (A Whole School Approach)” is a European policy experimentation project that involves eight institutions[1] across five countries (Slovenia, Croatia, Germany, Sweden, Denmark) and targets the need detected in Europe and internationally to develop inclusive societies (schools and classrooms) that allow every student to feel accepted and be able to achieve their potential, particularly in response to increasing discriminatory behavior some students are facing. HAND in HAND targeted this need with an innovative support system for school settings that combines the promotion of social, emotional and intercultural/transcultural (SEI) competencies.

The evaluation of the HAND in HAND programme was multifaceted and pursued different objectives. On the one hand, the quality or effectiveness of the programme should be evaluated (summative function) and on the other hand, we wanted to gather information on how our programmes could be optimised in the future (formative function).

In order to test whether the HAND in HAND programmes positively affect the SEI competencies of students and school staff (and an inclusive classroom climate), we assessed the effectiveness of these programmes in a field trial experiment in three EU countries: Slovenia, Croatia and Sweden. We used the same quasi-experimental design with control groups in the three countries.

The starting point for the development of the assessment, was the work on a concrete definition of expected outcomes of the HAND in HAND programmes (“Core Concepts”) to be able to build on a common basis for the development of student and school staff programmes on one hand, and the development of the instruments for the external evaluation of the programmes on the other hand, with a view to achieving the optimal alignment of both (for more information, see Kozina, 2020). The respective Core Concepts are based on CASEL model (Collaborative for Academic, Social and Emotional Learning, 2013), complemented by approaches targeting intercultural/transcultural competences. Even though the social and emotional competencies are often included in the core of intercultural/transcultural competencies (e.g. Stier, 2003), those two areas of competence are usually treated separately within various research traditions. In HAND in HAND, we placed a strong focus on the constructs that are important for both areas (e.g. openness, respect, relationship competence) while also focusing on aspects that are more specific to each (e.g. self-management in the area of social and emotional competences and moving beyond the self–other binary in the area of intercultural/transcultural competences).

In a second step, we researched existing open-access instruments to assess the previously defined Core Concepts and gathered them in an assessment catalogue that in international part consist of contains 168 scientific assessment tools and in national part of 15 existing practices (http://handinhand.si/wp-content/uploads/2019/12/SEI-measures_CATALOGUE.pdf) (Denk et al., 2017). Most existing instruments targeting SEI competencies are based on self-report questionnaire scales. Since several existing scales were available for each Core Concept in HAND in HAND, we decided to test a large number of self-report scales in a set of cognitive laboratories followed by a pilot study.

Next to those self-reports, we selected and developed several other measurement types to realize a broad and multifaceted assessment strategy.

• Other reports: In the student questionnaire, we used one measure to compare different classmates’ perspectives on each students’ cooperative behaviour.
• Vignettes. In addition to questionnaire scales, we also included vignettes that start with a brief description of a scenario, followed by questions asking the participants to assess different aspects of that scenario. We used different vignettes assessing either social perspective-taking or diversity awareness.
• Sociometry. We furthermore asked students with which other students from their class they had most often spent their breaks during school over the previous months and whether there were any students in their class with whom they did not spend any of their breaks during that time. With this information, we could derive the density of the social network in the classroom and the proportion of isolated students.
• Semi-structured focus-group interviews. Interviews were an important component of our evaluation as they consider the perspectives of participants. They encompassed aspects such as why the school had taken part in the HAND in HAND programme, how participants liked the programme and particular exercises, what they had learned from them, and whether they had any suggestions to help improve the programmes.

The HAND in HAND evaluation represents an illustration of the merits of combining different assessment and evaluation strategies – quantitative and qualitative, summative and formative. Thereby, more comprehensive information could be collected as compared to traditional evaluations that focused only on effectiveness: Different stages of effectiveness are considered and regarded from different perspectives, and we have obtained numerous possible leverages to adapt the programmes so that more sustainable effects can be achieved (for a detailed presentation of the evaluation procedure, instruments and results see Kozina, 2020).

We believe that this project is a first step of many in following one of the key messages of the HAND in HAND project stating that each child and young person should have an equal opportunity to access, participate and benefit from high quality and inclusive educational environment and that all learners (both students and school staff) and their various needs should be placed at the centre of education. They should be leaders of their own learning, supported by appropriate policy actions and services coherently organised at the system level.

References:

Collaborative for Academic, Social and Emotional Learning (2013). The 2013 CASEL Guide: Effective social and emo on- al learning programs-preschool and elementary school edition. Chicago, IL: Author.

Denk, A., Muller, F., Lubaway, E., Salzer, C., Kozina, A., Vidmar, M., Rasmunson, M., Marušić, I., Nielsen, L. B., Roczen, N., Ojsteršek, A., & Jurko, L. (2017). Catalogue for the assessment of social, emotional, and intercultural competencies: Hand in Hand. Available at: http://handinhand.si/wp-content/uploads/2019/12/SEI-measures_CATALOGUE.pdf

Kozina, A. (2020). Social, emotional and intercultural competencies for inclusive school environments across Europe. Relationships matter. Hamburg: Verlag Dr. Kovač. Available at: https://www.verlagdrkovac.de/volltexte/978-3-339-11406-8.htm

Stier, J. (2003). Internationalisation, ethnic diversity and the acquisition of intercultural competencies. Intercultural Education, 14(1), 77–91.

Contact information:
ana.kozina@pei.si
www.handinhand.si

[1] Educational Research Institute – project leader (ERI) and the Ministry of Education, Science and Sport (MESS), Slovenia; the Institute for Social Research Zagreb (ISRZ), Croatia; Mid Sweden University (MIUN), Sweden; the Technical University of Munich (TUM) and the Leibniz Institute for Research and Information in Education (DIPF), Germany; VIA University College, Denmark; and the Network of Education Policy Centres (NEPC), network