Introduction to Robotics and Perception Draft

Copyright (c) 2022-2024 Frank Dellaert and Seth Hutchinson, All rights Reserved

The code in the notebooks and python .py files is covered by the BSD license, like GTSAM; see the LICENSE file for details.

The remainder (including all markdown cells in the notebooks and other prose) is not licensed for any redistribution or change of format or medium, other than making copies of the notebooks or forking this repo for your own private use. No commercial or broadcast use is allowed. We are making these materials freely available to help you learn robotics, so please respect our copyright and these restrictions.

This repository contains one python notebook per section.

Current Draft TOC (chapters will be viewable as we finish drafts):

Chapter 1: General Introduction

Chapter 2: Discrete: Trash Sorter

2.1 State: plastic|paper|cardboard|metal|glass|unknown
2.2 Actuators: gripper (4 actions or no-op)
2.3 Sensors: Attributes -> colors (5), size (SML), weights (LH)
2.4 Perception: Estimation probabilities of plastic|paper|cardboard|metal|unknown
2.5 Planning: Decision Theory -> sort into three bins (plastics|mixed paper|metals|glass) or do nothing
2.6 Learning: Learning sensor models (discrete)

Chapter 3: Discrete: Vacuum Cleaner

3.1 State: house
3.2 Actuators: noisy graph
3.3 Sensors: noisy light
3.4 Perception: HMM
3.5 Planning: MDP
3.6 Learning: Model-based RL and Policy Iteration

Chapter 4: 2D: Logistics Platform

4.1 State: x, y
4.2 Actuators: Swedish wheels, round robot
4.3 Sensors: 2D LIDAR
4.4 Perception: Localization
4.5 Planning: Graph-based stuff, config space
4.6 Learning: Learning motion models (continuous)

Chapter 5: x, y, theta: Duckiebot

5.1 State: x, y, theta (not fully SE(2) and 3x3 matrices yet)
5.2 Actuators: differential drive
5.3 Sensors: Camera, image formation
5.4 Perception: CNN Place Recognition
5.5 Planning: Dubbins etc, Sampling-based planning
5.6 Deep Learning: Stochastic Gradient descent for CNN

Chapter 6: SE(2): Autonomous Vehicle

6.1 State: SE(2): SO(2) and 3x3 homogeneous
6.2 Actuators: Ackerman steering
6.3 Sensors: 3D LIDAR
6.4 Perception: ICP, Pose-SLAM
6.5 Planning: Motion Primitives
6.6 Learning: Deep Reinforcement Learning (Policy Gradient)

Chapter 7: SE(3): Drone

7.1 State: SE(3): SO(3) and 4x4 homogeneous
7.2 Actuators: Quad-rotor
7.3 Sensors: Camera in 3D, geometry, stereo
7.4 Perception: Visual SLAM
7.5 Planning: Trajectory Optimization
7.6 Learning: Tentative: Safe Flying

Name		Name	Last commit message	Last commit date
Latest commit History 405 Commits
.github/workflows		.github/workflows
Art		Art
Figures0		Figures0
Figures1		Figures1
Figures2		Figures2
Figures3		Figures3
Figures4		Figures4
Figures5		Figures5
Figures6		Figures6
Figures7		Figures7
_static		_static
old_intro		old_intro
plugins		plugins
stonehenge		stonehenge
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
S10_introduction.ipynb		S10_introduction.ipynb
S11_models.ipynb		S11_models.ipynb
S12_reasoning.ipynb		S12_reasoning.ipynb
S13_math.ipynb		S13_math.ipynb
S20_sorter_intro.ipynb		S20_sorter_intro.ipynb
S21_sorter_state.ipynb		S21_sorter_state.ipynb
S22_sorter_actions.ipynb		S22_sorter_actions.ipynb
S23_sorter_sensing.ipynb		S23_sorter_sensing.ipynb
S24_sorter_perception.ipynb		S24_sorter_perception.ipynb
S25_sorter_decision_theory.ipynb		S25_sorter_decision_theory.ipynb
S26_sorter_learning.ipynb		S26_sorter_learning.ipynb
S30_vacuum_intro.ipynb		S30_vacuum_intro.ipynb
S31_vacuum_state.ipynb		S31_vacuum_state.ipynb
S32_vacuum_actions.ipynb		S32_vacuum_actions.ipynb
S33_vacuum_sensing.ipynb		S33_vacuum_sensing.ipynb
S34_vacuum_perception.ipynb		S34_vacuum_perception.ipynb
S35_vacuum_decision.ipynb		S35_vacuum_decision.ipynb
S36_vacuum_RL.ipynb		S36_vacuum_RL.ipynb
S40_logistics_intro.ipynb		S40_logistics_intro.ipynb
S41_logistics_state.ipynb		S41_logistics_state.ipynb
S42_logistics_actions.ipynb		S42_logistics_actions.ipynb
S43_logistics_sensing.ipynb		S43_logistics_sensing.ipynb
S44_logistics_perception.ipynb		S44_logistics_perception.ipynb
S45_logistics_planning.ipynb		S45_logistics_planning.ipynb
S46_logistics_learning.ipynb		S46_logistics_learning.ipynb
S50_diffdrive_intro.ipynb		S50_diffdrive_intro.ipynb
S51_diffdrive_state.ipynb		S51_diffdrive_state.ipynb
S52_diffdrive_actions.ipynb		S52_diffdrive_actions.ipynb
S53_diffdrive_sensing.ipynb		S53_diffdrive_sensing.ipynb
S54_diffdrive_perception.ipynb		S54_diffdrive_perception.ipynb
S55_diffdrive_planning.ipynb		S55_diffdrive_planning.ipynb
S56_diffdrive_learning.ipynb		S56_diffdrive_learning.ipynb
S60_driving_intro.ipynb		S60_driving_intro.ipynb
S61_driving_state.ipynb		S61_driving_state.ipynb
S62_driving_actions.ipynb		S62_driving_actions.ipynb
S63_driving_sensing.ipynb		S63_driving_sensing.ipynb
S64_driving_perception.ipynb		S64_driving_perception.ipynb
S65_driving_planning.ipynb		S65_driving_planning.ipynb
S66_driving_DRL.ipynb		S66_driving_DRL.ipynb
S70_drone_intro.ipynb		S70_drone_intro.ipynb
S71_drone_state.ipynb		S71_drone_state.ipynb
S72_drone_actions.ipynb		S72_drone_actions.ipynb
S73_drone_sensing.ipynb		S73_drone_sensing.ipynb
S74_drone_perception.ipynb		S74_drone_perception.ipynb
S75_drone_planning.ipynb		S75_drone_planning.ipynb
S76_drone_learning.ipynb		S76_drone_learning.ipynb
_config.yml		_config.yml
_toc.yml		_toc.yml
conda-linux-64.lock		conda-linux-64.lock
environment.yml		environment.yml
intro.ipynb		intro.ipynb
logo.png		logo.png
myst.yml		myst.yml
notebooks.ipynb		notebooks.ipynb
references.bib		references.bib
require.js		require.js

License

gtbook/robotics

Folders and files

Latest commit

History