Skip to content

LIPSedge™ AE400 Industrial 3D Camera brings RealSense™ technology to NVIDIA Isaac Robot platform.

License

Notifications You must be signed in to change notification settings

lips-hci/stereo_ae400

Repository files navigation

LIPSedge™ AE400 Industrial 3D Camera

ae400 product banner LIPSedge™ AE400 is an industrial GigE 3D camera with IP67 protection and powered by Intel® RealSense™ technology and designed for industrial applications, such as robot application, logistic/factory automation, and 3D monitoring/inspection.

Title Video
AE400 360° Product View and Introduction AE400 Industrial 3D Camera
AE400 Point Cloud DEMO (use realsense-viewer) Point cloud demo

PREREQUISITE

  1. Prepare a Ubuntu 18.04 x64 desktop equipped with a NVIDIA video card that supports CUDA.
  • download Isaac SDK to any directory you preferred, e.g. your home directory ~/
  1. Download latest NVIDIA Isaac SDK from website:
  1. Install dependencies of desktop/Bazel/NVIDIA GPU Driver according to the following document:
~/isaac$ engine/build/scripts/install_dependencies.sh

Quick Start

  1. Download latest NVIDIA Isaac SDK
  1. Git clone LIPS stereo_ae400 workspace:
$ cd ~/
$ git clone https://github.com/lips-hci/stereo_ae400.git
  1. Specify path of Isaac SDK in the WORKSPACE file under folder stereo_ae400.
local_repository(
     name = "com_nvidia_isaac",
     path = "/home/nvidia/isaac", # Here to sepcify your Isaac SDK location, change nvidia by your username
 )
  1. Build application under folder stereo_ae400.
$ cd ~/stereo_ae400
$ bazel build //apps/ae400_camera
  1. Run app
$ bazel run //apps/ae400_camera

AE400 Firmware requirements

  1. The branch 'master' now runs AE400 RealSense SDK v1.0.0.2 at host-side, device firmware minimum requirement is V2.1.
  1. 👉 How to check my AE400 firmware version?

  2. The pervious firmware v1.4 works for AE400 SDK 'v0.9.0.7' on ISAAC SDK 2019.x~2020.1

  3. If your device is still firmware 1.1, use tag 'sdk-v0.9.0.5' for development.

Branch/tag information

💡 The branch 'master' now supports latest ISAAC SDK 2020.2 with new features.

  • reading depth/color camera intrinisics
  • reading IMU 6-axis sensor data
  • new app multicam wich can demo both depth and color streaming from 2 (or more) AE400 cameras

AE400 new features for Isaac 2020.1

  • support left/right IR cameras
  • more config options to control streams (Depth/RGB/IR)

From this version, you can select left and right IR channels to view in Isaac Sight webpage.

previous releases support

  • Switch to tag 'isaac-2019.3' if you are using previous Isaac SDK 2019.3.

  • Switch to tag 'isaac-2019.2' for users who want to stay at Isaac release 2019.2.

Learn more

  1. How to switch between Intel and NVIDIA video cards on Ubuntu?
  1. If you want to install CUDA 10.0 manually, you can refer this one-click-install script.

Network configurations

  1. IP address setting - Camera side:

The default setting of AE400 camera is 192.168.0.100, however, you can assign new IP address to it via browser.

Input IP as URL http://192.168.0.100 to open AE400 configuration page, log in, write new IP address and save setting

note: Check your product manual to find login account and password, or contact LIPS to get support

  1. IP address setting - Host side:

Configure the IP address for connecting AE400 by editing the network setting file

 $ vi config/network.json

 {
   "config": {
     "ip": "192.168.0.100"  <= it should match AE400 camera's IP address
   }
 }
  1. Install network setting into your system
 $ sudo mkdir -p /usr/etc/LIPS/lib
 $ sudo cp config/network.json /usr/etc/LIPS/lib/

note: AE400 software looks for setting in system path /usr/etc/LIPS/lib/network.json. If file is missing, default IP address 192.168.0.100 is applied.

Run camera sample application

  1. Launch app by bazel commandline
 $ bazel run //apps/ae400_camera

note: make sure the host, AE400 camera, and the remote robot are at same network domain, so they can connect to each other.

  1. Open a web brower, connect to http://localhost:3000

From left panel, select ae400_camera checkbox to enable depth/color channels for streaming.

Here is a screenshot of Isaac Sight webpage. screenshot of Isaac Sight

Deploy the sample application to remote robot (optional)

You can run the application on remote robot like Jetson Nano or TX2

Deploy the sample application to remote robot

  • To run ae400_camera sample application on the robot first deploy the package with Isaac SDK built-in command.
 # deploy.sh --remote_user <username_on_robot>
              -p //apps/ae400_camera:ae400_camera-pkg
              -d <jetpack44>
              -h <robot_ip>
              --run [optional]

Where <username_on_robot> is your user name on the robot, and <robot_ip> is the IP address of the robot.

Using the --run (or -r) option causes deploy.sh to automatically run the application on the robot after deployment.

  • For example, assume <robot_ip> is 192.168.0.99, the command is
 $ cd ~/stereo_ae400
 ~/stereo_ae400$ ln -s ~/isaac/engine/build/deploy.sh .
 ~/stereo_ae400$ ./deploy.sh --remote_user lips -p //apps/ae400_camera:ae400_camera-pkg -d jetpack44 -h 192.168.0.99

Run the sample application remotely on robot

  • Use below commands on your host side:
  1. make sure you have an SSH key on your desktop machine.
 $ ssh-keygen
 $ ssh-copy-id <username_on_robot>@<robot_ip>

note: use command ssh-copy-id to save your time typing password when loging in to robot by ssh.

  1. install network setting to remote robot
 $ cd ~/isaac
 $ vi config/network.json   //optional, re-assign IP address if you want
 $ ssh <username_on_robot>@<robot_ip> "sudo mkdir -p /usr/etc/LIPS/lib"
 $ scp config/network.json <username_on_robot>@<robot_ip>:/usr/etc/LIPS/lib/
  1. Login to the robot to run the application
 $ ssh <username_on_robot>@<robot_ip>
 $ cd ~/deploy/<username_on_host>/ae400_camera-pkg
 $ ./apps/ae400_camera/ae400_camera
  1. open a web brower, connect to http://<robot_ip>:3000

  2. from left panel, select ae400_camera checkbox to enable depth/color channels for streaming.