🔭 PiTracker

A deep tracking rover.

Key Features • Usage • Setup • Credits • License

PiTracker leverages TensorFlow Lite and a lightweight neural network - Single Shot Detection - in order to deliver tracking at decent 3-4 FPS despite the RPi's weak CPU and GPU.

Key Features

• Control the rover using your keyboard
• Detect objects using TensorFlow Lite (tflite)
• Track a specified object class (eg. person or bird) using a pan-tilt camera with Proportional Integral Derivative (PID) process control

Usage

usage: main.py

This produces a view from your RPi camera with bounding boxes of objects. Use "WASD" in the terminal to control the rover and the picamera should track any object you specify. Press "E" to exit.

Setup

There are two parts to the set-up: building the rover and setting up your Raspberry Pi.

Build Instructions

You need the following hardware:

• Raspberry Pi 4B (at least 2GB recommended)
• Robot Car Chassis
  • TT Motor + Wheel X 2
  • 4xAA battery holder
  • Acrylic board to hold everything together
• L298N Motor Driver (Used to control the motors)
• 5MP Camera for Raspberry Pi
• FFC Cable longer than 20cm (as you will need to move your camera around)
• SG90 Micro Servo X 2 (to pan/tilt camera)
• Mini breadboard
• Jumper wires

The above schematic is how to connect all our components together. It might seem a little complicated, but don't worry, it's actually not that complex! There are two parts to the diagram: the motors (the batteries, L298N motor driver and the two DC motors) and the servos (the breadboard and two servos)

1. The motors

1. Connect the two DC motors to the L298N motor driver
2. Connect GPIO BOARD pin 13 to the L298N in1 pin (Brown)
3. Connect GPIO BOARD pin 11 to the L298N in2 pin (Orange)
4. Connect GPIO BOARD pin 15 to the L298N en1 pin (Green)
5. Connect GPIO BOARD pin 16 to the L298N in3 pin (Blue)
6. Connect GPIO BOARD pin 18 to the L298N in4 pin (Purple)
7. Connect GPIO BOARD pin 22 to the L298N en2 pin (White)
8. Connect the batteries to the L298N
9. Connect the L298N to a GPIO GND pin

2. The servos

The servos are much simpler, so just follow that part of the schematic diagram

3. Pan-tilt camera

To build the pan-tilt camera, you just need two servos - one for panning and the other for tilting. You can either buy a Pimoroni one, or just tape together two servos like I did below.

Setting up RPi

1. Enable camera on the Raspberry Pi

In the terminal of your rpi, issue:

$ sudo raspi-config

Go -> Interfacing Options -> P1 Camera -> Yes

2. Clone this Github repository

$ git clone https://github.com/jwnicholas99/rpi_rover.git
$ cd rpi_rover/

3. Create a new venv and install packages

First, create and activate a new virtual environment by issuing:

$ python3 -m venv rpi-rover
$ source rpi-rover/bin/activate

Second, install required packages for OpenCV

$ sudo apt-get -y install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
$ sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
$ sudo apt-get -y install libxvidcore-dev libx264-dev
$ sudo apt-get -y install qt4-dev-tools libatlas-base-dev
$ pip3 install opencv-python==3.4.6.27

Third, install Tensorflow Lite. If your Python is version 3.5:

$ pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp35-cp35m-linux_armv7l.whl

If version 3.7:

$ pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl

Fourth, install other python packages:

$ pip3 install -r requirements.txt

4. Update main.py to use your own GPIO pins

It's likely that you will use different GPIO pins, so open up main.py and update the following lines:

# right motor
in1 = 13
in2 = 11
en1 = 15

# left motor
in3 = 16
in4 = 18
en2 = 22

# servos are using pigpio, which uses BCM numbering
# while the motors are using BOARD numbering
pan = 12
tilt = 13

Note that I am using two different modules for controlling GPIO pins: RPi.GPIO (for interfacing with the motor driver and the wheels) and pigpio (for controlling the servos). This is because pigpio is much more accurate than RPi.GPIO in maintaining pulse width modulation, hence preventing the servos from twitching too much.

Note that for the RPi.GPIO, I'm using BOARD numbering, while pigpio only allows me to use BCM numbering.

Credits

• Python - Everything is written in Python
• EdjeElectronic's repo on using Tensorflow Lite for object detection - I modified his TFLite_detection_webcam.py to fit into the PiTracker code
• Adrian Rosebrock's guide on pan-tilt face tracking - his code formed the base for using PID processes for controlling the pan-tilt camera

License

This project is licensed under the MIT License - see the LICENSE.md file for details.