Robotics and computer vision project of Advanced Robotics at SDU.
A dynamic simulated environment using ROS/Gazebo framework is used to facilitate development of a vision-based pick and place pipeline within a workcell consisting of a UR5 manipulator mounted onto a specialized table with designated pick and place areas, equipped with various perception sensors. Read the project report for more information.
The dynamic simulation environment is established with interfaces that enable interaction with the environment and control of the manipulator and gripper. A reachability analysis is used to determine the optimal base mount location of the manipulator. Motion planning is used to facilitate object relocation via trajectory generation; both interpolation-based and collision-free methods are implemented and evaluated. Two vision-based pose-estimation methods are implemented and evaluated for localization of object within the workcell.
This repository is structured as a catkin ROS workspace, consisting of the following packages:
ur5_description- URDF description of the UR5 robot.ur5_dynamics- Dynamics and kinematics library for the UR5 robot.ur5_controllers- ROS Control controllers for the UR5 robot.ur5_moveit_config- Moveit configuration for the UR5 robot.rovi_gazebo- Integration of ROVI workcell into Gazebo with various interface methods.rovi_utils- Various helper methods for the ROVI system.rovi_planner- Motion planning of the UR5 robot using KDL and MoveIt.rovi_pose_estimator- Vision-based pose estimation methods for objects in the ROVI workcell.rovi_system- Integration of and testing ground for ROVI system components.
Description of each package can be found in the corresponding README (TODO).
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
The system is tested on Ubuntu 18.04.3 LTS and Ubuntu 20.04.3 LTS. The project is compiled using GNU C++17, built with catkin (CMake 2.8.3), mainly depending on:
- ROS (noetic) - used as framework for robot operation
- Gazebo - used as robot simulation environment
- KDL - used for dynamics and motion interpolation
- MoveIt - used for motion planning
- OpenCV - used for computer vision
- PCL - used for point clouds
Installation of the system/workspace/package; the following configuration steps are necessary:
- Installing ROS
- Installing external dependecies (OpenCV, PCL via
apt install) - Cloning repository (catkin workspace)
- Installing ROS dependencies and building catkin workspace
Once the catkin directory is cloned, rosdep can be used to install any ROS dependencies; start by installing rosdep. Since noetic is relatively new, some packages must be installed from the ros-testing repository. To do this, open the /etc/apt/sources.list.d/ros-latest.list file as sudo and change /ros/ to /ros-testing/ - then run sudo apt update and sudo apt upgrade.
To install ROS dependecies, run rosdep update from the ws directory, followed by rosdep install --from-paths src --ignore-src -r -y. Once all dependencies are installed, the workspace can be built using catkin_make from the ws directory.
For auto-completion, linting etc., VS Code can be configured as explained here. VS Code should be launched from the ws directory (code rb-rovi/ws), where IntelliSense can then be automatically configured to fetch all necessary headers.
Navigate to ../rb-rovi/ws/ and run source devel/setup.bash to load the necessary environment variables to access the commands to run the packages of this repository. The workspace simulation can then be launched by running roslaunch rovi_gazebo workcell.launch.
For example, the integrated system test can be run by first running source devel/setup.bash. Then, the simulation environemnt is launched by running roslaunch rovi_system planning.moveit.launch (unpause the simulation in Gazebo) and the integration test node is launched by running rosrun rovi_system planning_integrated in a seperate terminal (also after sourcing).
We use SemVer for versioning. For the versions available, see the releases on this repository. Furthermore, this changelog documents the most relevant changes.
No license has been decided yet.
- RoboGnome - configuration of VS Code to run flawlessly with ROS