WARNING: This code is designed for use on a specific UR5 robotic arm, if using the physical robot environments, workframes defined in ur5_with_tactip.py will need to be altered before use! (for both cont and disc robot envs)... It is reccomended that you use the robot-jogger tool provided in the required Common Robot Interface to find the neccesary coordinates for your setup, details are given in the guide directory.

Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard

Structure

├── braille_rl
    ├── algos                    
        ├── cont_sac_algo 
            ├── train_cont_model.py
            ├── sac.py 
            ├── core.py
            ├── resume_training.py
            └── evaluate_model.py  
        ├── disc_sac_algo
            └── ...  
        ├── dd_dqn_algo  
            └── ...
        ├── td3_algo
            └── ...
        ├── saved_modles
        ├── image_utils.py
        ├── rl_utils.py
        └── plot_progress.py
    ├── envs   
        ├── robot
            ├── discrete
            └── continuous
        └── sim
            ├── discrete
            └── continuous
├── data  
├── guide
└── CAD

Contents

• guide: Details how to use this repository as a starting point for re-running experiments on a physical robot.

• CAD: This directory contains the STL files needed to 3D print the Cherry MX braille keycaps. SLDPRT Files are also included incase minor adjustments are required.

• data: This directory contains tactile data, collected by exhaustively sampling the braille keyboard with a TacTip sensor. This data is used to create the simulated environments.

• envs: This directory contains the environments used in this project. Simulated environments are approximations to the real physical environments, created using data collected with a physical robot.

• algos: This directory contains the reinforcement learning algorithms used to train agents across the provided environments, these are based off the implementations given in Spinning Up.

  • Each algorithm has a train_model script, this is where experiment hyper-parameters are set.
  • The algorithm logic is provided in the algorithm titled script (e.g. sac.py) and network logic is provided in core.py.
  • When the save_freq parameter is set, trained models will be saved in the saved_models directory (Warning, uncompressed replay_buffers are saved which means file sizes will be large).
  • resume_training.py can be used to continue learning from a previously saved model.
  • evaluate_model.py will evaluate the accuracy and speed of a saved model.
  • Shared utility functions are provided in image_utils.py and rl_utils.py.

Dependencies:

tensorflow 1.15.0
gym
cv2
numpy
cri
vsp

Installation

# requires python>=3.6
# It is reccomended that you use a virtual environment for this set up

# clone and install the repo (this may take a while)
git clone https://github.com/ac-93/braille_rl.git
cd braille_rl
pip install -e .

# install Spinningup from openAI
git clone https://github.com/openai/spinningup.git
cd spinningup
pip install -e .

# install common robot interface
git clone https://github.com/jlloyd237/cri.git
cd cri
python setup.py install

# install video stream processor
git clone https://github.com/jlloyd237/vsp.git
cd vsp
python setup.py install

# install python3-v4l2capture
git clone https://github.com/atareao/python3-v4l2capture.git
cd python3-v4l2capture
python setup.py install

# test the installation by running a training script in simulation, from the base directory run
python algos/dd_dqn_algo/train_discrete_model.py

Setting up Experiments

Check the guide for details on how to setup experiments on a physical robot.