Autonomous robots–including self-driving vehicles, sidewalk delivery robots, and more–must navigate among humans in a safe and socially-compliant manner. Current approaches for building and evaluating such autonomous systems rely on data-driven techniques; however, a generalization gap emerges, as methods trained in these traditional paradigms are unable to cope with unexpected real-world scenarios. Therefore, this thesis aims to develop improved methodologies and evaluation settings to increase and assess robustness in autonomous navigation against these challenges, along two key pillars of enhanced data utilization.

First, we introduce scenario characterization and repartitioning schemes, for robustness against out-of-distribution safety-relevant and corner case scenarios. We create a hierarchical characterization method which leverages counterfactual probes to find hidden safety-relevant scenarios in large datasets. We then address the induced generalization gap by incorporating the characterizations into downstream trajectory prediction models' inductive biases. To promote greater interpretability and generalizability, we factorize scenarios into disentangled contexts, creating compositionally novel test sets. We then use modular architectures and auxiliary signals to implicitly reason over and adapt to these settings.

Second, we design targeted scenario modification approaches, to expose and address failure cases and weaknesses of naive autonomy methods. For robustness against perception errors affecting downstream motion prediction, we construct a framework for converting top-down pedestrian trajectory datasets into a more challenging first-person view perspective. We then develop a correction module to account for the resulting errors, trained end-to-end with trajectory prediction approaches. For robustness against adversarial, safety-critical scenarios, we develop a reactive, skill-based adversary policy which leverages a learned, multi-faceted criticality objective to perturb existing scenarios. We then train ego policies in a closed-loop manner against these generated scenarios, demonstrating improved downstream ego performance. Finally, we process and annotate unlabeled and underutilized data sources, to learn human-like behavior from real-world crash videos. We use these learned behavior models to further increase the realism of adversarially perturbed scenarios, as well as the efficacy of closed-loop ego training.

Overall, we find that enhanced data utilization is a key component in developing robust evaluation settings and policy methodologies in autonomous navigation. Because broader machine learning domains exhibit similar data scarcity and out-of-distribution challenges, generalizing these ideas beyond autonomy is likewise promising.

108 pages

Thesis Committee:
Jean Oh (Chair)
Sebastian Scherer
Reid Simmons
Jonathan Francis (Bosch Center for Artificial Intelligence)

Jignesh Patel, Interim Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science

Creative Commons: CC-BY (Attribution)

Return to: School of Computer Science

This page maintained by reports@cs.cmu.edu