Design Challenge Winner: Measuring students’ executive functions in a classroom context with an innovative, tablet-based assessment

By Jelena Obradović

Imagine entering a quiet room where you are greeted by a friendly research assistant. She smiles at you, asks you about your day, maybe makes a funny joke. Next, you are seated together as she explains a game you will be playing. As you practice the rules, she offers enthusiastic praise and encouragement. During the game, she monitors your progress, and when it’s over, she gives you a small toy as an appreciation of your effort.

This highly controlled setting is typical in studies of child development, particularly in assessments of children’s executive functions (EF) — the skills that enable children to control their attention, behaviors, and emotions. Through studies like these, EFs have been linked to many educational outcomes, including school readiness, adaptive classroom behaviors, and academic achievement. However, the assessment setting I’ve just described doesn’t resemble a classroom setting, where they practice and apply EF skills daily.

A further problem with taking children out of the classroom to be assessed, one at a time, in a laboratory-like setting, is that the studies don’t scale well. Individual assessment places a burden on teachers by reducing instructional time and disrupting the students’ day, and it requires a lot of resources — a lot of research assistants putting in a lot of hours — which makes the use of objective direct assessments of EFs prohibitively expensive for large-scale studies.

To address these limitations, my lab developed a new procedure where all students in a classroom are assessed at the same time using developmentally appropriate EF tasks adapted for tablet computers. The lead research assistant, acting as a head teacher, explains the rules with large poster-boards in front of the classroom, runs a group practice session by soliciting verbal responses from the students, and then monitors students’ individual practice on tablet computers. Passcode screens pace everyone’s progress and keep the whole class moving at the same speed. Meanwhile, two more research assistants roam the classroom like teacher’s aides, helping students who need technical assistance.

To examine the reliability and validity of EF data collected via this group assessment, a subset of students completed the same tablet tasks in an individual assessment (e.g., in the school library room), either before or after the classroom assessment (to counterbalance practice effects). Our study, recently published in the Journal of Applied Developmental Psychology, showed that measures of EF skills obtained through the group and individual assessment had analogous internal reliabilities and statistically identical associations with student demographics, classroom behaviors, and academic achievement (Obradović, Sulik, Finch, & Tirado-Strayer, 2017). EF composites from both the group and individual assessments uniquely predicted teacher report of self-regulated classroom behaviors, but only the group assessment EF composite predicted two-year improvement in math and ELA achievement on standardized tests.

This procedure offers a pragmatic, cost-effective, and minimally disruptive way to objectively assess students’ EF skills at scale, in a setting that better approximates real-world conditions. As such, our assessment can be used to evaluate various school-based policies and programs designed to promote student’s self-regulation. To facilitate these efforts, we are sharing the tablet apps, source code, and supporting materials with scientists worldwide at no cost using an open-source license.

Obradović, J., Sulik, M., Finch, J.E., & Tirado-Strayer, N. (2017). Assessing executive functions in the classroom: Validating a scalable group-based procedure. Journal of Applied Developmental Psychology. 1-10. doi: 10.1016/j.appdev.2017.03.003

Disclaimer: The Assessment Work Group is committed to enabling a rich dialogue on key issues in the field and seeking out diverse perspectives. The views and opinions expressed in this blog are those of the authors and do not necessarily reflect the official policy or position of the Assessment Work Group, CASEL or any of the organizations involved with the work group.