This repo contains the vision systems and relevant datasets used by the joint team of Multirobot Systems Group of the Czech Technical University (CZ), GRASP at University of Pennsylvania (USA), and Lincoln Centre for Autonomous Systems Research at University of Lincoln (UK) during the Mohammed bin Zayed International Robotics Challenge MBZIRC.
These vision system were used in the MBZIRC Challenge I (automated landing) and MBZIRC Challenge III (cooperative object collection). Both vision systems are available here as ROS nodes with associated config and launch files. Their description is provided in Journal of Field Robotics in a paper by Stepan et al: Vision techniques for on-board detection, following and mapping of moving targets.
Note that the original system was running with Ubuntu version 14.04 and ROS indigo. For the sake of compatibility, the codes provided here were updated for compatibility with ROS kinetic and Ubuntu 16.04. Moreover, we performed an extensive cleanup, removal of ballast and legacy code, and added comments, so that one can easily relate the code blocks to the algorithm description of the aforementioned article. To verify if the aforementioned changes affected the systems' performance, we re-run the experiments described in Section 4 of the article and obtained similar results as in the paper.
The landing pattern detection is done by cross_detector node cross_detector.cpp. The core of vision detection is in the detector.cpp. The codes contains comments with respect to Section 2 of the submitted paper.
To see debug information you can uncomment some definition inside the detector.cpp file.
To quickly test the code, build the node using catkin_make, download one of the rosbags, then run the node using the test.launch:
roslaunch cross_detector test.launch
If everything goes well, you should see the pattern detection output similar to this video:
The datasets collected during the contest are accessible on google drive folder MBZIRC2017: Landing datasets
This vision system is intended for the `Treasure hunt' scenario, where several UAVs search for small objects in the contest arena, pick them up and deliver to a specific location.
The detection and localisation is performed by mbzirc_detector node, which is written in cpp in detector.cpp. The core of the detection is in the CSegmentation class and the core of the 3d localisation is in the CTransformation class. The mapping method runs in a separate package. These codes contain comments numbered in the same way as in Algorithm 3,4,5 of the article, so one can relate the blocks of actual cpp code to the code of the paper.
To quickly test the code, build the nodes using catkin_make, download one of the rosbags, then run the node using the rosbag.launch:
UAV_NAME=uav2 roslaunch object_detection rosbag.launch
Once the node displays three empty windows, play the downloaded rosbag:
rosbag play Grand_Challenge_flight_1_UAV_2.bag
If everything goes well, one of the windows should show the view from the UAV, which should be similar to the video:
This video is related to the Experiment described in Section 4.2 of the paper.
The datasets collected during the contest are accessible on google drive folder MBZIRC2017: Treasure hunt vision datasets.
Please make sure that the following dependencies are installed:
sudo apt-get install ros-kinetic-image-transport-plugins