Skip to content

ROS 2 implementation of a Teleoperated robot with live video feed using webrtc and SLAM using realsense's stereocameras. Running on a Jetson Nano

License

Notifications You must be signed in to change notification settings

jdgalviss/jetbot-ros2

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Jetbot-ros2

explore_lite.mp4

This is an implementation of a mobile robot in ros2. The software includes the following functionalities:

  • Teleoperation through websockets with live video feed using webrtc (aiortc).

  • Integration of Intel realsense d435 and t265 cameras for depth estimation and localization respectively.

  • Autonomous Navigation using Nav2

  • Autonomous Exploration using m-explore

  • 2D SLAM with slam-toolbox.

  • 3D SLAM using rtabmap.

I used the Xiaor Geek Jetbot as a base platform and modified it to include a wide-angle camera, as well as the Intel Realsense d435 and t265.

rs-viewer

Requirements

Installation

  1. Clone this repo and its submodules.

    git clone --recurse-submodules https://github.com/jdgalviss/jetbot-ros2.git
  2. Clone additional thirdparty packages

    1. SLAM-TOOLBOX

      cd jetbot-ros2/dev_ws/src
      git clone -b eloquent-devel git@github.com:stevemacenski/slam_toolbox.git 
    2. Navigation2

      git clone https://github.com/ros-planning/navigation2.git --branch eloquent-devel
    3. m-explore

      git clone -b eloquent https://github.com/robo-friends/m-explore-ros2.git
    4. BehaviorTree.CPP

      git clone https://github.com/BehaviorTree/BehaviorTree.CPP.git
      cd BehaviorTree.CPP
      git checkout 3.5.1
      cd ../..
  3. Copy our modified files

    cd thirdparty_files
    source copy_files.sh
    cd ../..

Teleoperation support

  1. Install aiortc for webrtc support.

    pip3 install crc32c==2.0
    pip3 install aiortc==0.9.28
    pip3 install aiohttp==3.6.2
  2. Install pyfakewebcam so that camera frames can be modified inside a ROS2 node and then shared through webrtc:

    apt-get install v4l2loopback-utils
    pip3 install pyfakewebcam==0.1.0
  3. Write a service that creates fake webcam devices that can be used to share camera frames.

    gedit /etc/rc.local

    Copy and paste in file:

    #!/bin/sh -e
    modprobe v4l2loopback devices=2 # will create two fake webcam devices
    exit 0

    Save the file and make it executable with this command:

    chmod +x /etc/rc.local

SLAM Support

Install rtabmap and rtabmap_ros following these instructions in the branch ros2.

Build ros 2 workspace

cd dev_ws
rosdep install -y -r -q --from-paths src --ignore-src --rosdistro eloquent
colcon build --symlink-install

Run

Teleoperation

  1. Run local teleoperation server.

    python3 local_server/webcam.py

    In a browser, open the teleoperation interface by going to: <jetson_nano's ip-address>:8080

  2. In a new terminal, run the motion control launchfile to start streaming video and receiving motion commands.

    ros2 launch motion_control jetbot_launch.py

Navigation

  1. In a new terminal, run nav2
    ros2 launch nav2_bringup nav2_navigation_launch.py

SLAM

  1. To Run SLAM.
    • For 2D-SLAM, in another terminal:
      ros2 launch realsense_ros2 realsense_launch.py
      In another terminal:
      ros2 launch slam_toolbox online_async_launch.py
    • For 3D-SLAM, in another terminal:
    ros2 launch realsense_ros2 slam_rtabmap_launch.py
    3D Dense SLAM is too resource consuming for the Jetson Nano, in this case it is recommended to run it on a remote host. For this, simply set the same DOMAIN_ID on both the Jetson Nano and the remote host. (e.g. export DOMAIN_ID=0) and run the cameras in the Jetson Nano:
    ros2 launch realsense_ros2 realsense_launch.py
    Comment the nodes corresponding to the cameras on the host and run the rtabmap launch:
    ros2 launch realsense_ros2 slam_rtabmap_launch.py

Exploration

  1. In a new terminal, run explore_lite
    ros2 run explore_lite explore --ros-args -p costmap_topic:=/map -p visualize:=true -p use_sim_time:=false -p min_frontier_size:=0.4 -p planner_frequency:=0.5

Some Results

Video

jetbot cartographer rtabmap

explore_lite.mp4

About

ROS 2 implementation of a Teleoperated robot with live video feed using webrtc and SLAM using realsense's stereocameras. Running on a Jetson Nano

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published