Using ouster-sdk python package, a ValueError is fired on pcap vizualization.

[fghoussen]

Describe your question
Using ouster-sdk python package, a ValueError error is fired on pcap vizualization (AFAIU when processing pcap content): is there a workaround or a fix for this problem?

Note that when I open others pcap (like OS1-128_Rev-06_fw23_Urban-Drive_Dual-Returns.pcap provided here https://static.ouster.dev/sensor-docs/#sample-data) I have no problem: the GUI shows up and run as expected (scene OK).

I run ouster-sdk in a virtual environment.

>> python --version
Python 3.10.13

>> ouster-cli --version
ouster-cli, version 0.9.0

Plugins provided:
ouster.cli.plugins.discover
ouster.cli.plugins.io_type
ouster.cli.plugins.source
ouster.cli.plugins.source_mapping
ouster.cli.plugins.source_osf
ouster.cli.plugins.testing

I get the following python stack:

Exception in thread Viz processing:
Traceback (most recent call last):
  File "/home/fhoussen/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/home/fhoussen/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/sdk/viz.py", line 1150, in _process
    self._scan_viz.scan = next(seekable)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/sdk/viz.py", line 871, in __next__
    t = next(self._it)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/core.py", line 418, in __iter__
    packet = next(it)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/pcap/pcap.py", line 124, in __iter__
    validator = LidarPacketValidator(self.metadata)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/data.py", line 144, in __init__
    self._metadata_sn = int(metadata.sn)
ValueError: invalid literal for int() with base 10: ''

As the error message looks like unexpected error thrown on unexpected case, I tried to edit ~/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/data.py to replace the line :

>> sed -n '144,144p' ~/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/data.py 
        self._metadata_sn = int(metadata.sn)

with:

>> sed -n '144,147p' ~/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/data.py 
        if isinstance(metadata.sn, int):
            self._metadata_sn = int(metadata.sn)
        else:
            self._metadata_sn = 0

Then, the code does not trigger any exception, but, I get a viewer with no data in it.

I also tried to clone this repository, but, got a conan error:

>> git clone https://github.com/ouster-lidar/ouster_example
>> python3 -m venv `pwd`/venv
>> source `pwd`/venv/bin/activate
>> pip install conan
>> mkdir build; cd build
>> conan install ..
ERROR: Error loading conanfile at '/home/fhoussen/Workspaces/ouster_example/conanfile.py': Unable to load conanfile in /home/fhoussen/Workspaces/ouster_example/conanfile.py
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/fhoussen/Workspaces/ouster_example/conanfile.py", line 3, in <module>
    from conans import ConanFile, CMake, tools
ImportError: cannot import name 'ConanFile' from 'conans' (/home/fhoussen/Workspaces/ouster_example/venv/lib/python3.11/site-packages/conans/__init__.py)

Not sure if I am supposed to use conan-1.* or conan-2.*: I stopped here...

Questions:

• Is there a workaround in case this is a known problem?
• Or is there a fix what I could test (and PR later if agreed)?

Any help or clue on how to fix or workaround the problem is appreciated! :)

Screenshots
The call stack I get is the following when I try to visualize the pcap:

>> ouster-cli source  /path/to/data.pcap  viz
Reading metadata from: /path/to/live-1024x20.json
GL Renderer: Mesa Intel(R) Graphics (ADL GT2)
GL Version: 4.6 (Core Profile) Mesa 22.3.6 (GLSL: 4.60)
Press '?' while viz window is focused to print key bindings
Exception in thread Viz processing:
Traceback (most recent call last):
  File "/home/fhoussen/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/home/fhoussen/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/sdk/viz.py", line 1150, in _process
    self._scan_viz.scan = next(seekable)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/sdk/viz.py", line 871, in __next__
    t = next(self._it)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/core.py", line 418, in __iter__
    packet = next(it)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/pcap/pcap.py", line 124, in __iter__
    validator = LidarPacketValidator(self.metadata)
  File "/home/fhoussen/.local/share/virtualenvs/mmHNQW6C/lib/python3.10/site-packages/ouster/client/data.py", line 144, in __init__
    self._metadata_sn = int(metadata.sn)
ValueError: invalid literal for int() with base 10: ''

Note that getting informations related to the same pcap does work (no exception triggered):

>> ouster-cli source   "/path/to/2022-10-19-11-08-21_Ouster-OS1_-Data.pcap"  info
Reading pcap:  [####################################]  100%          
File size:     20359.97M
Packets read:  1685534
Encapsulation: ETHERNET
Capture start: 2022-10-19 11:08:28.233297
Capture end:   2022-10-19 11:30:29.278011
Duration:      0:22:01.044714
UDP Streams:
          Src IP                        Dst IP        Src Port        Dst Port        AF        Frag         Size          Count        
          192.168.0.200        255.255.255.255           40931            7502         4          No        12608        1685534        

Platform (please complete the following information):

• Ouster Sensor? OS-1
• Ouster Firmware Version? Not sure, this information has been lost
• Programming Language? Python
  • For Python include the targeted Python version: 3.10.13
• Operating System? Linux (debian bookworm)
• Machine Architecture? x64
• git commit hash (if not the latest). No commit hash, use in python virtual env

[fghoussen]

I just tried to open the pcap in ouster studio.

I get this error message

With following traces:

>> ./OusterStudio-2.0.1.AppImage 
Socket.io listening on port 1041
Checking for update
[699633:0920/163137.231911:ERROR:nss_util.cc(349)] After loading Root Certs, loaded==false: NSS error code: -8018
Update for version 2.0.1 is not available (latest version: 2.0.1, downgrade is disallowed).
[2023-09-20 16:31:47.027] [ouster::sensor] [warning] No imu_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No lidar_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No prod_line found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No prod_sn found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No build_rev found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No build_date found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No image_rev found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No prod_pn found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No status found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No data_format found. Using default legacy data format
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No valid imu_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:47.031] [ouster::sensor] [warning] No valid lidar_to_sensor_transform found. Using default for gen 1
../src/intel/isl/isl.c:2235: FINISHME: ../src/intel/isl/isl.c:isl_surf_supports_ccs: CCS for 3D textures is disabled, but a workaround is available.
[2023-09-20 16:31:48.252] [ouster::sensor] [warning] No imu_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.252] [ouster::sensor] [warning] No lidar_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No prod_line found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No prod_sn found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No build_rev found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No build_date found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No image_rev found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No prod_pn found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No status found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No data_format found. Using default legacy data format
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No valid imu_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.253] [ouster::sensor] [warning] No valid lidar_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.256] [ouster::sensor] [warning] No imu_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No lidar_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No prod_line found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No prod_sn found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No build_rev found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No build_date found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No image_rev found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No prod_pn found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No status found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No data_format found. Using default legacy data format
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No valid imu_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.257] [ouster::sensor] [warning] No valid lidar_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No imu_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No lidar_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No prod_line found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No prod_sn found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No build_rev found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No build_date found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No image_rev found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No prod_pn found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No status found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No data_format found. Using default legacy data format
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No valid imu_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.264] [ouster::sensor] [warning] No valid lidar_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No imu_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No lidar_to_sensor_transform found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No prod_line found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No prod_sn found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No build_rev found in metadata. Will be left blank or filled in with default legacy values
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No build_date found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No image_rev found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No prod_pn found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No status found in metadata. Your metadata may be the result of calling to_string() on the sensor_info object OR you recorded this data with a very old version of Ouster Studio. We advise you to record the metadata directly with get_metadata and to update your Ouster Studio.
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No data_format found. Using default legacy data format
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No valid imu_to_sensor_transform found. Using default for gen 1
[2023-09-20 16:31:48.271] [ouster::sensor] [warning] No valid lidar_to_sensor_transform found. Using default for gen 1
Pcap has 0 frames