Hardware Specifications:

Updated hardware specification in progress

Project Details

Inspired by Nindamani weed removal robot

Looking to improve on that based on the hardware linked above

Open Weeding Delta,autonomously detects and segment the weeds from crop using artificial intelligence. It's built on ROS2. Open Weeding Delta could be used in any early stage crops for autonomous weeding using mechanical or laser actuators

Parameter	Value
Robotics OS	ROS2.0 Humble
System	Ubuntu 22.04 LTS
Kernel	Realtime kernel
Communication	Wireless, Canbus, UART(internal motor control)
Vision	DepthAI-ROS
AI Framework	Keras
Object instance segmentation	Mask R-CNN
Programming Language	Python3 & C

Install Nindami on 22.04 / Humble
Adapt Nindamani for OakD Use nodelet to publish to Image_transport RPIscript may then work
Add support for TOF sensors/ height adjustment
Publish speed control ROS2 messages to slow down/stop UGV
Improve weed/row recognition algorithms

Parameter	Value
Degrees of freedom	3 DOF
Error	? mm
Payload	0.5 kg
Weight	8 kg
Height	TBC to TBC mm
Width	TBC mm
Arm Reach	TBC sq mm
Processor board	Jetson nano Dev Kit
Microcontroller	TBC
Stepper Motor /BLDC	48V, 6A, Nema 34, 87 kgcm H.Torque
Camera	TBC
Wifi card	Intel 8265
USB-TTL cable	PL2303HX chip
Battery	48V 30ah

Datasets

Latvia 1118 anotated images 7,442 weed images (8x species), 411 crop images (6x crops)

960 unique plants belonging to 12 species at several growth stages

V1 Plant seedings 1.8GB

V2 Plant seedlings 2GB

5000 seedling images

Cropdeep seems useful, but data not available?

CFWD 242 annotated images

1300 images of sesame/weeds

1700 images of weeds native to Autralia

Features:

Fully ROS2 compatible
Battery Operated
Runtime upto 8-10 hours
Robotics Arm based weed removal
Weed detection accuracy upto 85%
Easy to Operate

Packages

In this section we will install all the necessary dependencies in order to be able to launch nindamani robot:

nindamani_agri_robot - integrate all launch node of nindamani robot
rpicam_ai_interface - controlling the rpi camera with AI interface
servo_control - controlling the servo motors with ROS2 interface
stepper_control - controlling the multiple stepper motors with ROS2 interface

Installation on Jetson Nano Dev Kit

1. Ubuntu

Download latest SDK image: https://github.com/Qengineering/Jetson-Nano-Ubuntu-20-image
Completely Format SD card (should not contain any partition). Use Ubuntu default app Disks [Recommeded 64GB SD card]
Copy ZIP(jetpack image) file to SD card: https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit#write

2. ROS2 (Foxy)

Install ROS2 base: https://index.ros.org/doc/ros2/Installation/Foxy/Linux-Install-Debians/
Make sure that you have colcon in your machine if you are installing from Debian packages. sudo apt install python3-colcon-common-extensions
For adding additional packages use: sudo apt install ros-$ROS_DISTRO-<package-name>

3. ROS2-Tensorflow

Install ros2-Tensorflow: https://github.com/alsora/ros2-tensorflow#readme

4. Depthai

Follow these instructions to install DepthAI: https://docs.luxonis.com/projects/api/en/latest/install/#jetson

5. Arduino

Follow this repo to install Arduino on Jetson nano: https://github.com/JetsonHacksNano/installArduinoIDE.git
To get Temporary access to USB: sudo chown <user-name> /dev/tty<usb> and sudo chmod a+rw /dev/tty<usb>
To set Permenantly change USB device permission: http://ask.xmodulo.com/change-usb-device-permission-linux.html
To control arduino from Command line Source:https://github.com/arduino/arduino-cli
Clone this repo: https://github.com/AutoRoboCulture/Arduino-Jetson-nano-interface.git
Place this repo in Arduino Folder

5. Wifi

To setup default wifi connection(Intel 8265 NGW card) while bootup Source: https://desertbot.io/blog/how-to-add-a-dual-wifi-bluetooth-card-to-a-jetson-nano-intel-8265

Create ROS2 Workspace

follow this steps:

  mkdir -p ~/nindamani_ws/src
  cd ~/ros2_mara_ws
  colcon build
  cd src
  git clone https://github.com/AutoRoboCulture/nindamani-the-weed-removal-robot.git

Clone the Mask R-CNN GitHub Repository:

Code: git clone https://github.com/BupyeongHealer/Mask_RCNN_tf_2.x
Copy this cloned repo to rpicam_ai_interface package: cp Mask_RCNN rpicam_ai_interface/.
Run command:
- cd rpicam_ai_interface/Mask_RCNN
- sudo python3 install setup.py
Confirm the Library Was Installed: pip3 show mask-rcnn

Download preTrained Model weights

Link for MASK-RCNN preTrained model

Copy preTrained weights to rpicam_ai_interface package:

mkdir rpicam_ai_interface/preTrained_weights
cp mask_rcnn_trained_weed_model.h5 rpicam_ai_interface/preTrained_weights/.

Follow Folder Structure:

nindamani_ws
├── build
├── install
├── log
└── src
  ├── nindamani_agri_robot
  │   ├── launch
  │   └── scripts
  ├── rpicam_ai_interface
  │   ├── scripts
  │   ├── preTrained_weights
  │   └── Mask-RCNN
  ├── servo_control
  │   ├── config
  │   ├── scripts
  │   └── srv
  └── stepper_control
      ├── config
      ├── scripts
      ├── src
      └── srv

Compile nindamani_ws

Follow steps:
```
cd nindamani_ws
colcon build
```

Dependency

Stepper Motor library implementation on Arduino

Launch nindamani robot

Make sure source setup.bash in bashrc before ROS2 launch command: echo "source /home/<user-name>/nindamani_ws/install/setup.bash" >> ~/.bashrc
ROS2 Launch command: ros2 launch nindamani_agri_robot nindamani_agri_robot.launch.py

Demo video | Proof of Concept

Potential Improvements

We have presented the concept that how weeds can be detected from crops using Artifical Intelligence and through delta arm robot weeds are removed autonomously. It's not perfect of course as you can see in the video link but can be improved. Here are some of our ideas which can improvise this robot in future:

Gripper design enchancement with end tip as arrow shaped.
Delta arm reach can be improved with high torque stepper motor.
With RTK-GPS and 4 wheeled drive + 4 wheel steering implementation on robot, it will make whole robot working autonomously.
Need 3D mapping of land using Lidar, for finding variations in height of crops, weeds and ridge.

References

Mask R-CNN for Object Detection and Segmentation

 @misc{matterport_maskrcnn_2017,
  title={Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow},
  author={Waleed Abdulla},
  year={2017},
  publisher={Github},
  journal={GitHub repository},
  howpublished={\url{https://github.com/matterport/Mask_RCNN}},
}

Developer's Contact Detail

Kevin Patel
Nihar Chaniyara
Email: autoroboculture@gmail.com

# Delta notes

[Inverse Kinematics](https://github.com/giridharanponnuvel/Delta-Robot-Inverse-Kinematics)

[Commercial Igus with 3x linear actuator](https://www.igus.co.uk/product/20433?artNr=DLE-DR-0001)

[Delta X1](https://store.deltaxrobot.com/products/delta-x-basic-kit) 

[TlAlexander planetary gear](https://github.com/tlalexander/brushless_robot_arm#readme)

Name		Name	Last commit message	Last commit date
Latest commit History 120 Commits
demo		demo
hardware		hardware
model		model
nindamani_agri_robot		nindamani_agri_robot
rpicam_ai_interface		rpicam_ai_interface
servo_control		servo_control
stepper_control		stepper_control
utils		utils
LICENSE		LICENSE
README.md		README.md

License

Agroecology-Lab/Open-Weeding-Delta

Folders and files

Latest commit

History

Repository files navigation