Autonomous Driving in Unity using ML Agents

• Development Documentation
• Setup Video
• Unity Apps for best trained RL Model:
  • Mac
  • Linux

Docker

Docker docs

Project Overview

A simulation model using Unity and MLAgents to explore a solution for lane keeping in autonomous vehicles. The purpose of this project is to explore deep learning in simulation environment and explore Unity's ML-Agent toolkit. We took a 2 part approach in exploring this problem:

• Part 1: Build a environment using Unity to train a vehicle agent using reinforcement learning.
  • This model will be guided using oracle knowledge of the world.
  • This is intended to save time with the ability to generate driving data.
• Part 2: Using the RL modeled trainined in part 1, training a CNN model for lane keeping using only data internal to the vehicle.
  • This is to simulate a more realistic model

See Detailed Slide Deck for more info

Overview of Reinforcement Learning Results

8x playback

For more details see the following:

• Trained model apps:
  • Mac
  • Linux
• RL Training
• Simulation Environment
• Unity
• Vehicle Agent

Reinforcement Learning I/O

See more details in Vehicle Agent.

Observations

• Wheel Angle, normalized [-1, 1]
• Front Distance to Center, normalized [-1, 1]
• Back Distance to Center, normalized [-1, 1]
• Z axes velocity from the vehicle's rigidbody
• X axes velocity from the vehicle's rigidbody

Actions

• 0: no change in wheel angle
• 1: add .25 degrees (turn right)
• 2: add -.25 degrees (turn left)

Overview of CNN Results

The CNN did not give us our desired results in the given amount of time. We decided not to include the running of our CNN in our docker image because it does not provide satisfactory results. For more details see CNN Training.

Inspiration

• ML Agents repo
• ML Agents paper
• Build a driving environment in Unity, then use the ML Agents toolkit to train our car (agent) to drive within lanes using reinforcement learning

Authors

• Grant Gasser
• Blaine Rothrock