In recent years, we have witnessed greater integration of autonomous and semi-autonomous agents in our world. From self-driving cars to remote sensing drones [24], autonomous agents have developed greater environment awareness due to advancements both at the software level (e.g. AI-based algorithms), as well as at the hardware level (e.g. higher-fidelity sensors). In particular, many autonomous agents are now equipped with sensors that are capable of capturing three-dimensional (3D) information, whether in the form of depth maps with RGB-D cameras, or point cloud data with Light Detection and Ranging (LiDAR) sensors. Due to the widespread integration of LiDAR sensors in modern devices, point clouds became the standard when it comes to collecting 3D data from the environment in real-time.

Yet beyond the hardware, advancements in the fields of computer vision and deep learning have made it practical to utilize 3D data in environment sensing & awareness scenarios. As an example, consider the task of semantic segmentation, which entails assigning distinct labels to semantically equivalent parts of the data – hence segmenting it. Such semantic definitions range from object-level semantics (doors, tables, cars, etc..) to part-level semantics (door handle, table-top, car wheels), and even instance-level semantics (door₁, table₇, car₈ [6]. Like their two-dimensional counterpart (images & vectors), it was quite difficult to achieve acceptable results on such tasks before the dawn of Deep Neural Networks. Unlike images, however, the field of deep learning on 3D data generally, and on point clouds specifically, has considerably lagged behind in the past for reasons that we will detail in section 1.3. Nevertheless, current research trends in the field show promising results and provide a significant room for exploration, as we shall see in this project.

32 pages

Advisor
Gianni Di Caro

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu