Skip to content

Universal Robot Realtime Facetracking with a webcam or a Rasperry Pi camera

Notifications You must be signed in to change notification settings

robin-gdwl/UR_Facetracking

Repository files navigation

UR_Facetracking

Currently under developement as of 2021-03-14

Code is a bit of a mess Use at your own risk.

Universal Robot Realtime Facetracking with Python3 and OpenCV Uses the UR - RTDE- protocol to send continuous updates to a Robot for smooth continued motion.

Demonstration

https://youtu.be/HHb-5dZoPFQ

Universal Robot Realtime Face Tracking Python

Usage

Run Face_tracking01.py to run the face tracking. Developed for a Universal Robot UR5 CB running Polyscope 3.7

Settings

robot_startposition: The joint angles of the robot at the beginning.
max_x , max_y : The maximum horizontal distance the robot can travel.
hor_rot_max, ver_rot_max: Maximum Rotation of the robot at the edge of the view-window defined by max_x and max_y.
m_per_pixel: Distance (in meters) the robot will move to get to the position of the face in the captured image.

Explanation

https://youtu.be/9XCNE0BmtUg

Universal Robot Realtime Face Tracking Python

The explanation is not comprehensive yet. A better Video is planned.


Requirements

Universal Robot or URsim-software (tested with UR5cb running polyscope 3.7)

python 3.x

Python libraries:
opencv-python 4.1.2.30
numpy 1.18.1
math3d 3.3.5
imutils 0.5.3
six 1.15.0

when used with a raspberry pi and picam:
picamera
...


Notes

When using a linnux distribution or raspberry pi the software will try to use a picamera. If you want to use a webcam instead change it here.

heavily relies on this repository: https://github.com/Mandelbr0t/UniversalRobot-RealtimeControl which builds ontop of: https://bitbucket.org/RopeRobotics/ur-interface/src/master/

License: GPLv3


TODO

  • more testing
  • demonstration video
  • explanation video (not the best explanation so far)
  • License clarification
  • cleanup
  • refactoring
  • documentation
  • comments
  • add links to libraries and other resources

About

Universal Robot Realtime Facetracking with a webcam or a Rasperry Pi camera

Topics

Resources

Stars

Watchers

Forks

Languages