What’s missing in SEL measurement and assessment? CONTEXT

By Stephanie M. Jones & Sophie P. Barnes, EASEL Lab, Harvard Graduate School of Education

Social and emotional learning (SEL) is on the map. Driven in part by solid evidence linking social competence and self-control in early childhood to life outcomes twenty to thirty years later (e.g., Jones et al., 2015; Moffitt et al., 2011), and by multiple meta-analytic studies showing substantial impacts of SEL interventions on short- and longer-term outcomes (Durlak et al., 2011; Sklad et al., 2012, Taylor et al., 2017), the last five years has seen a veritable explosion in interest and excitement about SEL. Indeed, there are many recent high-profile national, state, and local efforts focused on making the case for integrating SEL into the mission and work of schools and schooling (e.g., the National Commission on Social, Emotional, and Academic Development), providing guidance to the field in the areas of practice, policy, and research (e.g., Navigating SEL from the Inside Out), and building, testing, and deploying measures and assessments that inform practice and effectively link it to research and allow the field to track progress overall (e.g., Assessment Work Group and its Measuring SEL efforts; California CORE Districts; Panorama Education).

Of all these important efforts, those that are directed to measurement and assessment represent the biggest frontier and are probably the most fraught (Duckworth & Yeager, 2015). Much of this conversation has focused on issues of use, as in what is the purpose of measures and assessments, and are they up to the task? Truthfully, much of this conversation is about the measurement and assessment of individual competencies…and this is where it becomes fraught. Is it fair to hold a system – or a teacher — accountable to the social, emotional, and behavioral competencies of an individual child? What can measures and assessments of individuals tell us about a system, or an educator? As it turns out, not much. A critical piece is missing. Decades of theory and research in child development tells us what: the central role of context (e.g., Bronfenbrenner & Morris, 1998; Sameroff, 2000). Focusing on individuals only is misguided because it can perpetuate the idea of “fixing” children, and it misses the fundamentals of human development – that children develop in the context of key relationships and in interaction with their settings. We know intuitively and from our own experience that our capacities, skills, and interactions are shaped by features the environments we are in.

Research in SEL itself tells us the same thing. For example, studies indicate that social, emotional, and behavioral skills are particularly susceptible to experiences of stress and trauma (e.g., Evans & English, 2002; Raver, Blair, & Willoughby, 2013) and we know from developmental psychology and neuroscience that children are “differently sensitive” to their environments (e.g., Pluess et al., 2018). Some studies of SEL interventions underscore this point, indicating that they have their largest effects among students who face the greatest number of risks, including those with lower socioeconomic status or those who enter school behind their peers either academically or behaviorally (e.g., Cadima, Verschuren, Leal, & Guedes, 2016; Jones, Brown, & Aber, 2011). What this implies for measurement and assessment of SEL competencies, is that we may miss the mark if we only focus on individuals – we may think we have a good sense of a child’s functioning, but actually have captured a child’s response to characteristics of their immediate setting (e.g., the children they are with, their relationship with their teacher, the resources and materials available to them, etc.; Milkie & Warner, 2011). Capturing characteristics of the setting may be just as, if not more important, and may lead to more effective approaches to intervention.

Indeed, there is a growing body of evidence that suggests the powerful role of the environment over individual characteristics. Research on compositional effects tells us about the role of peers in classroom settings. Neidell & Waldfogel (2008) find that the saturation of children in Kindergarten classes with preschool experience positively influences individual children’s reading and math achievement through 3rd grade—even for children who did not attend preschool. In addition, children who spend their school year in classrooms with other aggressive children show sharper increases in aggression over time even if they start well below the average (e.g., Jones & Molano, 2016). Finally, the vast majority of interventions in SEL begin with training and support for teachers (i.e., training teachers to implement SEL interventions in their classrooms; Jones, Barnes, Bailey, & Doolittle, 2017), yet measurement most often focuses on the competencies acquired by children and youth.

So, what should we do? In order to address gaps in SEL measurement and assessment, our development efforts should be guided by the following criteria:

1. They should be Relevant. Measurement and assessment in SEL should capture the child or youth in the relevant context. This means that the child deserves to be understood in relation to their ecological system and measurement and assessment should be tied to the distinct demands of a particular setting.
2. They should be Actionable. Measurement and assessment should be tied to actionable, evidence-based practices and strategies that schools and educators can adopt and implement in their setting and these should include actions that are directed to features of the setting itself.
3. They should be Developmentally Salient. Measurement and assessment should align with what is realistic to expect of children and youth at specific ages and should represent features of developmentally salient contexts.
4. They should be Sensitive and Nuanced. Measurement and assessment must be both sensitive and nuanced enough to capture variation that exists within and between children and youth over time as well as meaningful variation that exists within and between contexts, such as different classrooms within the same school.

How would you modify, add to or qualify these four themes?

References

Bronfenbrenner, U., & Morris, P. A. (1998). The ecology of developmental processes. In R. M. Lerner (Ed.), Handbook of child psychology: Theoretical models of human development (5th ed., Vol. 1, pp. 993–1028). New York: Wiley.

Cadima, J., Verschueren, K., Leal, T., & Guedes, C. (2016). Classroom interactions, dyadic teacher–child relationships, and self–regulation in socially disadvantaged young children. Journal of Abnormal Child Psychology, 44(1), 7–17. https://doi.org/10.1007/s10802-015-0060-5

Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. https://doi.org/10.3102/0013189X15584327

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432. https://doi.org/10.1111/j.1467-8624.2010.01564.x

Evans, G. W., & English, K. (2002). The environment of poverty: Multiple stressor exposure, psychophysiological stress, and socioemotional adjustment. Child Development, 73(4), 1238–1248. https://doi.org/10.1111/1467-8624.00469

Jones, S. M., Barnes, S. P., Bailey, R., & Doolittle, E. J. (2017). Promoting social and emotional competencies in elementary school. The Future of Children, 27(1), 49–72.

Jones, S. M., Brown, J. L., & Aber, J. L. (2011). Two-year impacts of a universal school-based social-emotional and literacy intervention: An experiment in translational developmental research. Child Development, 82(2), 533–554. https://doi.org/10.1111/j.1467-8624.2010.01560.x

Jones, D. E., Greenberg, M., & Crowley, M. (2015). Early social-emotional functioning and public health: The relationship between kindergarten social competence and future wellness. American Journal of Public Health, 105(11), 2283–2290. https://doi.org/10.2105/AJPH.2015.302630

Jones, S. M., & Molano, A. (2016). Seasonal and compositional effects of classroom aggression: A test of developmental-contextual models. Journal of Cognitive Education and Psychology, 15(2), 225–247. http://dx.doi.org.ezp-prod1.hul.harvard.edu/10.1891/1945-8959.15.2.225

Milkie, M. A., & Warner, C. H. (2011). Classroom learning environments and the mental health of first grade children. Journal of Health and Social Behavior, 52(1), 4–22. https://doi.org/10.1177/0022146510394952

Moffitt, T. E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R. J., Harrington, H., … Caspi, A. (2011). A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences, 108(7), 2693–2698. https://doi.org/10.1073/pnas.1010076108

Neidell, M., & Waldfogel, J. (2008). Cognitive and non-cognitive peer effects in early education. NBER Working Paper Series, 14277. https://doi.org/10.3386/w14277

Pluess, M., Assary, E., Lionetti, F., Lester, K. J., Krapohl, E., Aron, E. N., & Aron, A. (2018). Environmental sensitivity in children: Development of the Highly Sensitive Child Scale and identification of sensitivity groups. Developmental Psychology, 54(1), 51–70. https://doi.org/10.1037/dev0000406

Raver, C. C., Blair, C., & Willoughby, M. (2013). Poverty as a predictor of 4-year-olds’ executive function: New perspectives on models of differential susceptibility. Developmental Psychology, 49(2), 292–304. https://doi.org/10.1037/a0028343

Sameroff, A. J. (2000). Developmental systems and psychopathology. Development and Psychopathology, 12(3), 297–312.

Sklad, M., Diekstra, R., Ritter, M. D., Ben, J., & Gravesteijn, C. (2012). Effectiveness of school-based universal social, emotional, and behavioral programs: Do they enhance students’ development in the area of skill, behavior, and adjustment? Psychology in the Schools, 49(9), 892–909. https://doi.org/10.1002/pits.21641

Taylor, R. D., Oberle, E., Durlak, J. A., & Weissberg, R. P. (2017). Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects. Child Development, 88(4), 1156–1171. https://doi.org/10.1111/cdev.12864

Disclaimer: The Assessment Work Group is committed to enabling a rich dialogue on key issues in the field and seeking out diverse perspectives. The views and opinions expressed in this blog are those of the authors and do not necessarily reflect the official policy or position of the Assessment Work Group, CASEL or any of the organizations involved with the work group.