A Psychometric Perspective on SEL Assessment

By: Michael C. Rodriguez, PhD, Professor, University of Minnesota

The increasing attention to social and emotional learning is consistent with the deepening realization that learning is social, although these concepts have long had presence in the literature on teaching and learning. The Johnson brothers (University of Minnesota) developed the principles of cooperative learning and were among the first to integrate the social and cognitive aspects of learning in the classroom. A new appreciation of the social contexts of education is taking hold in the educational measurement field, promoting the combination of cognitive learning theories and sociocultural theories to promote cognitive development, participation and motivation, and positive identity development. In educational measurement, these concepts offer ways to design assessments that measure the intended disciplinary content, core ideas, and practices, in contexts that are relevant to test takers experiences and ways of knowing/thinking/doing. Such an approach provides a solid grounding for successful SEL assessment, to measure core SEL concepts in ways that are relevant to diverse students’ interests, experiences, and identities. These concepts enable the design of approaches that support growth for all students, as well as the adults in their lives.

Through its efforts to provide guidance on SEL, the Minnesota Department of Education adopted the CASEL five competencies and defined grade-span-level benchmarks for early elementary, late elementary, middle, and high school students. Associated with the competency and benchmark statements, teams of educators, youth development specialist, and higher education faculty with related expertise developed guidance addressing implementation and professional development, and assessment. In part, the assessment guidance was informed by a set of Testing Standards developed and approved by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. High-stakes tests undergo rigorous evaluation under the umbrella of validation, typically by psychometricians. Until recently, few psychometricians have entered the world of SEL measurement

The MN assessment guidance is comprehensive and grounded in the Testing Standards to support appropriate interpretation and use of SEL assessment results. But they also bring to bear a set of principles that will support the broader goals of making SEL instruction and practice imbedded, intentional, and school wide. The assessment guidance opens with a reminder that the primary purpose of assessment in classrooms and schools is to support learning.

Purpose drives all assessment decisions, including what to assess, how to assess, when to assess, who to assess, etc. The purposes for SEL assessment must be consistent with the district’s mission and current strategic plan or goals. As such, it must become a component of the district’s communication strategy. Great benefits come from partnering with neighboring districts. Developing a networked improvement community around SEL assessment and implementation can move this work ahead more productively. Such decision making can and should take place in existing assessment systems and structures. An existing assessment committee should lead this work – or if nonexistent, consider establishing an assessment committee, including students and knowledgeable members of the community. Ideally, the SEL constructs being assessed, the methods of assessment, and the purposes and intended uses of SEL assessment information will align with the existing structures and contexts of districts, schools, and communities.

Context matters. Students exist in multiple spheres of influence, including families, neighborhoods, schools, communities, and larger geo-political areas (cities, counties, etc.), all of which rarely work in concert. Communities worldwide continue to diversify. Because something works in one area/context, does not guarantee it will work in others. An important type of evidence supporting SEL assessment use is measurement invariance. Can the results of an assessment be meaningfully and appropriately interpreted and used in diverse communities? Such evidence tends to be technical, requiring psychometric methods and interpretation. Without such evidence, we fail to have sufficient warrants to defend the use of an assessment in diverse contexts (or with students with characteristics different from those with whom the assessment was developed). SEL assessments, done well, directly raise issues of culture and context and hold promise toward improvement of educational equity, ensuring that all students have access to high-quality instruction and educational resources.

SEL assessment users must ensure that the resulting data meet quality requirements, including evidence of validity (evidence that supports score interpretations and uses) and reliability (score precision and consistency). More high-stakes uses or use with small groups (classrooms) require more rigorous evidence. For this reason, among others, the Minnesota guidance recommends against the use of SEL information at the individual student level. Such uses, and potential interpretations, require a level of psychometric rigor that has not been evident in many existing SEL measures – and, moreover, require a level of training and credentials that make such use impractical. School psychologists are licensed professionals capable of interpreting individual psycho-social measures, and they always use multiple sources of information to support recommendations and decisions. Individual-level use of SEL information is in opposition to the larger goals of employing SEL practices in a way that is imbedded and school wide. Test users have responsibilities and test takers have rights. The standards regarding score interpretation and use and those addressing the rights and responsibilities of test users and test takers are described in the MN assessment guidance.

When selecting an SEL assessment, users must review the existing evidence provided by the assessment developer or publisher. That evidence should be relevant for the intended uses and the user’s community. If new interpretations or uses are proposed, evidence needs to be collected to support them. If results are reported for groups of students with different characteristics, test users are responsible for ensuring sufficient evidence exists to support interpretations for each group and for group comparisons.

A foundation of the Testing Standards is fairness with respect to test takers and test users, a core validity issue. Regarding the psychometric contributions to achieve fairness, measurement bias is a significant threat. Measurement bias occurs when a measure functions differently for individuals with different backgrounds, even though they may have the same skill level. It suggests that the measure is not measuring the same thing or in the same way across communities with different characteristics. In addition, accessibility is an important characteristic, such that all individuals have the opportunity to reveal their level on the measured competency or skill. This can be enhanced through universal design methods that promote clarity of the construct; full representation of the construct; items and tasks that reflect the construct without irrelevant features or language that privileges some communities, race/ethnicity, sex, socioeconomic background, disability, native language, faith tradition, or other characteristics; and scoring and score interpretation that reflects the construct rather than construct-irrelevant characteristics of individuals and their contexts. Any observed differences in levels of the measured SEL competency should reflect real differences, not inaccessibility to the measure itself.

Finally, fairness concerns both the intended and unintended consequences of measurement interpretation and use – the benefits and costs should be equally shared among student communities, with specific attention to negative consequences because of SEL measurement, as they significantly impair our ability to achieve the broader goals of improving educational outcomes. Fairness includes all aspects of SEL assessment, including assessment design, administration, reporting, and use.

Key questions to consider:

1. Purpose: What is the primary reason for doing SEL assessment? What SEL competencies will be assessed? Will SEL assessment include adults and students?
2. Selection: Is there a structure/process in place to identify appropriate and useful SEL assessment methods and tools? Who needs to be involved in selection?
3. Quality: What evidence exists for the selected SEL assessment that supports the intended interpretation and use? Is there evidence that the tool will function well for students/adults in this community?
4. Administration: Are stakeholders prepared and partners engaged in the process? Is administration feasible?
5. Analysis/reporting: Is there sufficient in-house expertise to appropriately analyze and report results? Can SEL results contribute to existing communication strategies and goals?

Contact Information:

Michael C. Rodriguez

University of Minnesota

mcrdz@umn.edu