Keystroke dynamics–technology to distinguish people based on their typing rhythms–could revolutionize insider-threat detection. Insiders accessing backdoors, using shared accounts, or masquerading as other users would be exposed by their unique typing rhythms. In the past thirty years, dozens of classifiers have been proposed for distinguishing people using keystroke dynamics; many have obtained excellent results in evaluation. However, when evaluations are replicated, the results are often wildly different; one classifier's error rate jumped from 1% to 85% upon replication.

Classifier error rates depend on a multitude of factors; until the effects of these factors on error rates are understood, keystroke dynamics cannot realize its promise. To tackle this multitude-of-factors problem, we developed the following methodology: (1) evaluate multiple classifiers under systematically ranging conditions; (2) analyze the results with linear mixed-effects models (LMMs), a technique for inferential statistics well suited to understanding how various factors affect classifier error rates; and (3) validate the models, demonstrating that they accurately predict error rates in subsequent evaluations.

In three investigations using this methodology, we found that while some classifiers had lower error rates than others, the differences were overshadowed by the effects of factors other than the classifier. For the best classifier, error rates vary from 0%to 63%depending on the user. Impostors triple their chance of evading detection if they touch type. On the bright side, the best combination of timing features (hold times and up-down times) reduces error rates by over 50%. Other configuration tweaks, such as increased training and an updating strategy, offer further opportunity to significantly reduce error rates. On a scientific level, this work establishes that understanding these factors is critical to making progress in keystroke dynamics.

By understanding the influential factors, we can deploy the best classifier given the environment, accurately estimate its error rate, and know where to direct future efforts to improve performance. For the first time in keystroke dynamics, we can reliably predict classifier error rates, and our approach is general. Since other computer-security technologies (e.g., intrusion, worm, and malware detection) face analogous multitude-of-factors problems, they would similarly benefit from our methodology.

213 pages

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu