Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core.py: Enable correct exporting of char data #95

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

junwoo091400
Copy link

Description

As the data type for saving the char in pyulog was np.int8, the topics that had fields with characters weren't able to get encoded (with error: pyulog struct.error: char format requires a bytes object of length 1).

Therefore, this enables manual conversion of np.int8 object into bytes() with single character, so that struct.pack command works as expected.

Notable example of where this is needed is the "transponder_report" topic (included in PX4 v1.14)'s callsign[] character array. Before this fix, the "write_ulog()" function would fail as the struct.pack couldn't correctly encode the characters stored in int8 type

Testing

Before

Following simple script importing & exporting a ulog file from a PX4 v1.14 vehicle didn't work before (example log downloadable here):

from pathlib import Path
import numpy as np
from pyulog import ULog

ulog_path = '' # Set to the path of the log

TOPIC_NAME = 'transponder_report'
FIELD_NAME = 'callsign[0]'
ulog = ULog(str(ulog_path.absolute()))
dataset = ulog.get_dataset(TOPIC_NAME)
name = dataset.name
field_data = dataset.field_data
timestamp_idx = dataset.timestamp_idx
data = dataset.data
for field in field_data:
    if field == FIELD_NAME:
        print('Field name: {}, type: {}'.format(field.field_name, field.type_str))
field_values = data[FIELD_NAME]
print('Field values:', field_values, 'type:', type(field_values[0]))

output_file_name = ulog_path.name + '_test' + ulog_path.suffix
print('Writing ulog', output_file_name)
ulog.write_ulog(Path.joinpath(ulog_path.parent, output_file_name))

With the following error:

Field values: [0] type: <class 'numpy.int8'>
Writing ulog power_on_without_rc.ulg_test.ulg
Traceback (most recent call last):
  File "scripts\another.py", line 23, in <module>
    ulog.write_ulog(Path.joinpath(ulog_path.parent, output_file_name))
  File "venv\Lib\site-packages\pyulog-1.1.0-py3.11.egg\pyulog\core.py", line 253, in write_ulog
    self._write_data_section(ulog_file)
  File "venv\Lib\site-packages\pyulog-1.1.0-py3.11.egg\pyulog\core.py", line 401, in _write_data_section       
    items = self._make_data_items()
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\pyulog-1.1.0-py3.11.egg\pyulog\core.py", line 429, in _make_data_items
    data.extend(struct.pack('<' + field_encoding, field_data))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: char format requires a bytes object of length 1

After

After the change the error doesn't pop up and exporting the ULog file works as expected.

As the data type for saving the char in pyulog was np.int8, the topics
that had fields with characters weren't able to get encoded (with error:
pyulog struct.error: char format requires a bytes object of length 1).

Therefore, this enables manual conversion of np.int8 object into bytes()
with single character, so that struct.pack command works as expected.

Notable example of where this is needed is the "transponder_report"
topic (included in PX4 v1.14)'s callsign[] character array. Before this
fix, the "write_ulog()" function would fail as the struct.pack couldn't
correctly encode the characters stored in int8 type
Copy link
Member

@bkueng bkueng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, can you fix the CI error? Otherwise looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants