Skip to content

Multi-task learning for semantics and depth, 3D Object Detection from Lidar Point Clouds. Graded projects for the ETH course "Deep Learning for Autonomous Driving".

Notifications You must be signed in to change notification settings

nicolaloi/Deep-Learning-for-Autonomous-Driving

Repository files navigation

Deep Learning for Autonomous Driving

Projects regarding the implementation of:

  • sensor fusion of LiDAR, cameras, and IMU data;
  • deep learning networks for semantic segmentation and depth estimation from 2D images;
  • deep learning network for 3D object detection from point clouds.

Only the assignments are present, since the Professors asked to not share the code or the final reports.

Overall rating of the projects: 114% (bonus points thanks to optional assignments).

Project 1: Understanding Multimodal Driving Data

Visualize the outputs of common autonomous driving tasks such as 3D object detection and point cloud semantic segmentation given a LiDAR point cloud, corresponding RGB camera image, ground truth semantic labels and network bounding box predictions.

Moreover, identify each laser ID from the point cloud directly, and deal with the point cloud distortion caused by the vehicle motion with the aid of GPS/IMU data.

Further info: Assignment_Project1.pdf

Project 2: Multi-task learning for semantics and depth

Build Multi-Task Learning (MTL) architectures for dense prediction tasks, i.e. semantic segmentation and monocular depth estimation, exploiting joint architectures, branched architectures, and task distillation.

Finally, improve the network with personal ideas or refer to existing papers to enhance the predictions.

Further info: Assignment_Project2.pdf

Project 3: 3D Object Detection from Lidar Point Clouds

Build a 2-stage 3D object detector to detect vehicles in autonomous driving scenes, i.e. to draw 3D bounding boxes around each vehicle. Unlike Project 2 which was based on 2D images, now irregular 3D point cloud data are exploited to detect vehicles.

The first stage, which is often referred to as the Region Proposal Network (RPN), is used to create coarse detection results from the irregular point cloud data. These initial detections are later refined in the second stage network to generate the final predictions.

Further info: Assignment_Project3.pdf

About

Multi-task learning for semantics and depth, 3D Object Detection from Lidar Point Clouds. Graded projects for the ETH course "Deep Learning for Autonomous Driving".

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published