Students use intelligent tutors and other types of interactive learning environments in a considerable variety of ways. In this thesis, I detail my work to understand, automatically detect, and re-design an intelligent tutoring system to adapt to a behavior I term gaming the system . Students who game the system attempt to succeed in the learning environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly.

Within this thesis, I present a set of studies aimed towards understanding what effects gaming has on learning, and why students game, using a combination of quantitative classroom observations and machine learning. In the course of these studies, I determine that gaming the system is replicably associated with low learning. I use data from these studies to develop a profile of students who game, showing that gaming students have a consistent pattern of negative affect towards many aspects of their classroom experience and studies.

Another part of this thesis is the development and training of a detector that reliably detects gaming, in order to drive adaptive support. In this thesis, I validate that this detector transfers effectively between 4 different lessons within the middle school mathematics tutor curriculum without re-training. This detector uses Latent Response Models (Maris 1995), combining labeled and unlabeled data at different-grain sizes, in order to train a model to accurately indicate both which students were gaming, and when they were gaming, and uses Fast Correlation-Based Filtering (Yu and Liu 2003) to efficiently search the space of potential models.

The final part of this thesis is the re-design of an existing intelligent tutoring lesson to respond to gaming. The re-designed lesson incorporates an animated agent ("Scooter the Tutor") who indicates to the student and their teacher whether the student has been gaming recently, and gives students supplemental exercises, in order to offer the student another chance to learn material he/she gamed through. Scooter reduced the frequency of gaming by over half, and Scooter s supplementary exercises were associated with substantially improved learning; Scooter appeared to have little effect on non-gaming students.

123 pages

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu