Human-Computer Interaction Institute
School of Computer Science, Carnegie Mellon University
Improving Students' Study Practices
A key challenge of the learning sciences is moving research results
into practice. Educators on the front lines perceive little value in the
outputs of education research and demand more "usable knowledge". This work
explores the potential instead of usable artifacts to translate knowledge
into practice. One contribution is the integration of market-oriented
opportunity finding to design these translational systems and tools such
that they can be adopted easily into practice. A second contribution is the
conception of these systems as research probes that both operationalize
theoretical principles and provide instrumentation to build knowledge of
applying theory in practice.
College student study practices are the domain chosen for the development of and reflection upon these methods. Iterative ethnographic field work identified two systems that would be likely to advance both learning in practice and knowledge for applying the employed theories in general. Nudge was designed to improves students' study time management by regularly emailing students with explicit recommended study activities. It reconceptualizes the syllabus into an interactive guide that fits into modern students' attention streams. Examplify was designed to improve how students learn from worked example problems by modularizing them into steps and scaffolding their metacognitive behaviors though problem-solving and self-explanation prompts. It combines these techniques in a way that is exceedingly easy to author, using extant answer keys and students' self-evaluations for correctness.
Nudge and Examplify were evaluated experimentally over a full semester of a lecture-based introductory chemistry course. Nudge messages were found to increase students' sense of achievement and to interact with students' existing time management skills to improve exam grades for poorer students. Among students who could choose whether to receive them, 80% continued to. Students with access to Examplify had higher exam scores (d=0.26), especially on delayed measures of learning (d=0.40). A key design decision in Examplify was not clearly resolvable by existing theory and so was tested experimentally by comparing two variants, one without prompts to solve the steps. The variant without problem-solving was less effective (d=0.77) and was little used, while usage rates of the variant with problem solving increased over time.
These results support the use of the design methods employed and provide specific empirical recommendations for future designs of these and similar systems for implementing theory in practice.