yolact_ros

The yolact_ros is based on Yolact and is integrated with ROS(Robot Operating System)

Requirement

• ROS(Kinetic)
• GPU supporting CUDA

Installation

• Set up a Python3 environment.
• Install Pytorch 1.0.1 (or higher) and TorchVision.
  • conda install pytorch==1.0.1 torchvision==0.2.2 cudatoolkit=9.0 -c pytorch
• Install some other packages:

  # Cython needs to be installed before pycocotools
  pip install cython
  pip install opencv-python pillow pycocotools matplotlib 
  
  conda install opencv
  pip install pyyaml
  pip install rospkg
  
  

• Clone this repository and enter it:

  git clone https://github.com/dbolya/yolact.git
  cd yolact

• If you'd like to train YOLACT, download the COCO dataset and the 2014/2017 annotations. Note that this script will take a while and dump 21gb of files into ./data/coco.

  sh data/scripts/COCO.sh

• If you'd like to evaluate YOLACT on test-dev, download test-dev with this script.

  sh data/scripts/COCO_test.sh

Weight

Image Size	Backbone	FPS	mAP	Weights
550	Resnet50-FPN	42.5	28.2	yolact_resnet50_54_800000.pth	Mirror
550	Darknet53-FPN	40.0	28.7	yolact_darknet53_54_800000.pth	Mirror
550	Resnet101-FPN	33.0	29.8	yolact_base_54_800000.pth	Mirror
700	Resnet101-FPN	23.6	31.2	yolact_im700_54_800000.pth	Mirror

To evalute the model, put the corresponding weights file in the ./weights directory and run one of the following commands.

In my workspace, After downloaded the weight, copy and paste to the trained_model path.

Yolact ros with Visual grasping

Conda env:

• torch11py36(Custom computer)
• py36_ros(NSCL computer)

Capture image with customized config (In my case, Nobrand)

rosrun yolact_ros yolact_capture_img.py  --trained_model=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/weight/yolact_base_1234_100000.pth  --score_threshold=0.3 --top_k=100 --image=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/image/116.jpg

Launch

./realsense

roscore

(torch11py36) conda env

rosrun yolact_ros yolact_tcp_img.py  --trained_model=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/weight/yolact_base_1234_100000.pth  --score_threshold=0.3 --top_k=100 --image=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/image/116.jpg 

Save image from realsense through ros serveice

rosrun yolact_ros yolact_save_img.py 

rosservice call /save_image "data: false" 

Run ros server for getting call and sending instance information

rosrun yolact_ros yolact_ros_server.py  --trained_model=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/weight/yolact_base_1234_100000.pth  --score_threshold=0.3 --top_k=100