About decodeing the cat30 packets with longer length

[zouyunkaicauc]

Hi,

I‘m trying to decode the cat30 packet from pcap file. When decoding a packet of a shorter length(e.g. length=47), there is no problem. But when the length of the package is longer(e.g. length=504), the program is dead directly.

Here is the code.

f = open(ficher, 'rb')
pcap = dpkt.pcap.Reader(f)
for ts, buf in pcap:
    eth = dpkt.ethernet.Ethernet(buf)
    try:
        data = eth.data.data.data
    except:
        data = eth.data.data
    parsed = asterix.parse(data)
    i = 0
    c = parsed[i]['category']
    if c == 30:
        tp_X = parsed[i]
print(tp_X)

[dsalantic]

Can you send me the minimal data file that causes crash so i can try to reproduce it ?

[zouyunkaicauc]

Hi， Here are two files. Each containing only one cat30 package, and the name with “L” is a longer package. Best

…

------------------ 原始邮件 ------------------ 发件人: "Damir Salantic"<notifications@github.com>; 发送时间: 2019年8月28日(星期三) 中午1:28 收件人: "CroatiaControlLtd/asterix"<asterix@noreply.github.com>; 抄送: "邹云开"<zouyunkaicauc@qq.com>;"Author"<author@noreply.github.com>; 主题: Re: [CroatiaControlLtd/asterix] About decodeing the cat30 packetswith longer length (#92) Can you send me the minimal data file that causes crash so i can try to reproduce it ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[dsalantic]

I don't see files. Can you try to attach them directly on github or send the link where to download it.

[zouyunkaicauc]

1L.zip
1.zip

[dsalantic]

Can you check which version of asterix parser are you using.
import asterix; print(asterix.__version__)

If you are not using latest version (0.5.9) please upgrade and try again.

[zouyunkaicauc]

I'm using the latest version(0.5.9), is there no problem testing on your side?

[dsalantic]

I couldn't reproduce the problem.
Here is the output from your sample:

1.pcap
{'category': 30, 'I340': {'Mod3A': {'val': '0664', 'desc': 'Mode 3/A reply under the form of 4 digits in octal representation'}, 'spare': {'const': 0, 'val': 1, 'desc': 'spare bits set to 0'}, 'L': {'meaning': 'MODE 3/A code as derived from the reply of the transponder,', 'val': 0, 'desc': ''}, 'G': {'meaning': 'Default', 'val': 0, 'desc': ''}, 'V': {'meaning': 'Code validated', 'val': 0, 'desc': ''}}, 'I030': {'FX': {'meaning': 'end of data item', 'val': 0, 'desc': 'Field extension'}, 'C1': {'meaning': 'No', 'val': 0, 'desc': 'Track selected by the Complementary service 1'}, 'spare': {'const': 0, 'val': 10, 'desc': 'spare bits set to 0'}, 'BS': {'meaning': 'No', 'val': 0, 'desc': 'Track selected by the Background service'}}, 'I010': {'SAC': {'val': 8, 'desc': 'System Area Code'}, 'SIC': {'val': 132, 'desc': 'System Identification Code'}}, 'I180': {'Hdg': {'val': 5.06, 'desc': 'Heading'}, 'SPEED': {'val': 202.4, 'desc': 'Speed'}}, 'I035': {'family': {'val': 0, 'desc': 'Family to which the message belongs'}, 'nature': {'meaning': '???', 'val': 4, 'desc': 'Nature of the message (within its family)'}}, 'I140': {'GC': {'meaning': 'garbled information', 'val': 1, 'desc': ''}, 'ModeC': {'max': 127000.0, 'min': -1200.0, 'val': -37800.0, 'desc': 'Mode C'}, 'VAL': {'meaning': 'valid Mode C code', 'val': 0, 'desc': 'Validity indicator of the Flight Level'}}, 'I040': {'sttn': {'val': 1, 'desc': 'Track number indicator'}, 'spare': {'const': 0, 'val': 3, 'desc': 'Spare bits set to zero'}, 'TrackN': {'val': 1602, 'desc': 'Track number'}}, 'I160': {'TFL': {'max': 150000.0, 'min': -1500.0, 'val': -29025.0, 'desc': 'Track Flight Level'}, 'spare': {'const': 0, 'val': 0, 'desc': 'spare bit set to 0'}, 'QNC': {'meaning': 'FL calculated without QNH correction', 'val': 0, 'desc': ''}}, 'ts': 1566977658478, 'len': 39, 'I170': {'PSR': {'max': 63.75, 'val': 3.5, 'desc': 'Age of the last primary plot/local track used to update the track'}, 'AMODE': {'max': 63.75, 'val': 6.5, 'desc': 'Age of the last detection of the mode A contained in the track mode A item I030/060.'}, 'CMODE': {'max': 63.75, 'val': 3.25, 'desc': 'Age of the last valid and credible mode C used to update the track, contained in I030/150.'}, 'SSR': {'max': 63.75, 'val': 3.75, 'desc': 'Age of the last secondary plot/local track used to update the track'}}, 'I060': {'Mod3A': {'val': '0057', 'desc': 'Mode-3/A reply in octal representation'}, 'C': {'meaning': 'Track Mode 3/A has changed', 'val': 1, 'desc': 'Code changed'}, 'spare': {'const': 0, 'val': 1, 'desc': 'Spare bit set to 0'}, 'G': {'meaning': 'Garbled code', 'val': 1, 'desc': 'Garbled code'}, 'V': {'meaning': 'Code not validated', 'val': 1, 'desc': 'Code validated'}}, 'crc': 'F53A2581', 'I181': {'Y': {'max': 7200.0, 'min': -7200.0, 'val': -420.7763671875, 'desc': 'Y-Velocity component'}, 'X': {'max': 7200.0, 'min': -7200.0, 'val': -69.873046875, 'desc': 'X-Velocity component'}}, 'I150': {'spare': {'const': 0, 'val': 0, 'desc': 'spare bits set to 0'}, 'ModeC': {'max': 127000.0, 'min': -1200.0, 'val': -147100.0, 'desc': 'Mode C'}}, 'I070': {'ToLU': {'val': 1726.0078125, 'desc': 'Time Of Last Update'}}, 'I100': {'Y': {'max': 511.984, 'min': -512.0, 'val': 144.21875, 'desc': 'Y-Component'}, 'X': {'max': 511.984, 'min': -512.0, 'val': 492.03125, 'desc': 'X-Component'}}}

1L.pcap
{'category': 30, 'I340': {'Mod3A': {'val': '0222', 'desc': 'Mode 3/A reply under the form of 4 digits in octal representation'}, 'spare': {'const': 0, 'val': 0, 'desc': 'spare bits set to 0'}, 'L': {'meaning': 'MODE 3/A code as derived from the reply of the transponder,', 'val': 0, 'desc': ''}, 'G': {'meaning': 'Default', 'val': 0, 'desc': ''}, 'V': {'meaning': 'Code not validated', 'val': 1, 'desc': ''}}, 'I030': {'FX': {'meaning': 'end of data item', 'val': 0, 'desc': 'Field extension'}, 'C1': {'meaning': 'No', 'val': 0, 'desc': 'Track selected by the Complementary service 1'}, 'spare': {'const': 0, 'val': 10, 'desc': 'spare bits set to 0'}, 'BS': {'meaning': 'No', 'val': 0, 'desc': 'Track selected by the Background service'}}, 'I010': {'SAC': {'val': 8, 'desc': 'System Area Code'}, 'SIC': {'val': 132, 'desc': 'System Identification Code'}}, 'I180': {'Hdg': {'val': 0.5499999999999999, 'desc': 'Heading'}, 'SPEED': {'val': 22.0, 'desc': 'Speed'}}, 'I035': {'family': {'val': 0, 'desc': 'Family to which the message belongs'}, 'nature': {'meaning': '???', 'val': 6, 'desc': 'Nature of the message (within its family)'}}, 'I140': {'GC': {'meaning': 'default value', 'val': 0, 'desc': ''}, 'ModeC': {'max': 127000.0, 'min': -1200.0, 'val': -146600.0, 'desc': 'Mode C'}, 'VAL': {'meaning': 'valid Mode C code', 'val': 0, 'desc': 'Validity indicator of the Flight Level'}}, 'I040': {'sttn': {'val': 1, 'desc': 'Track number indicator'}, 'spare': {'const': 0, 'val': 1, 'desc': 'Spare bits set to zero'}, 'TrackN': {'val': 2626, 'desc': 'Track number'}}, 'I160': {'TFL': {'max': 150000.0, 'min': -1500.0, 'val': 51825.0, 'desc': 'Track Flight Level'}, 'spare': {'const': 0, 'val': 0, 'desc': 'spare bit set to 0'}, 'QNC': {'meaning': 'FL calculated without QNH correction', 'val': 0, 'desc': ''}}, 'ts': 1566980273201, 'len': 41, 'I170': {'PSR': {'max': 63.75, 'val': 3.5, 'desc': 'Age of the last primary plot/local track used to update the track'}, 'AMODE': {'max': 63.75, 'val': 50.0, 'desc': 'Age of the last detection of the mode A contained in the track mode A item I030/060.'}, 'CMODE': {'max': 63.75, 'val': 58.5, 'desc': 'Age of the last valid and credible mode C used to update the track, contained in I030/150.'}, 'SSR': {'max': 63.75, 'val': 1.75, 'desc': 'Age of the last secondary plot/local track used to update the track'}}, 'I060': {'Mod3A': {'val': '0057', 'desc': 'Mode-3/A reply in octal representation'}, 'C': {'meaning': 'Track Mode 3/A has changed', 'val': 1, 'desc': 'Code changed'}, 'spare': {'const': 0, 'val': 1, 'desc': 'Spare bit set to 0'}, 'G': {'meaning': 'Garbled code', 'val': 1, 'desc': 'Garbled code'}, 'V': {'meaning': 'Code not validated', 'val': 1, 'desc': 'Code validated'}}, 'crc': '8751A3C3', 'I181': {'Y': {'max': 7200.0, 'min': -7200.0, 'val': 69.43359375, 'desc': 'Y-Velocity component'}, 'X': {'max': 7200.0, 'min': -7200.0, 'val': 81.5185546875, 'desc': 'X-Velocity component'}}, 'I150': {'spare': {'const': 0, 'val': 0, 'desc': 'spare bits set to 0'}, 'ModeC': {'max': 127000.0, 'min': -1200.0, 'val': 160100.0, 'desc': 'Mode C'}}, 'I070': {'ToLU': {'val': 1468.0703125, 'desc': 'Time Of Last Update'}}, 'I100': {'Y': {'max': 511.984, 'min': -512.0, 'val': 176.21875, 'desc': 'Y-Component'}, 'X': {'max': 511.984, 'min': -512.0, 'val': 35.984375, 'desc': 'X-Component'}}, 'I080': {'COR': {'meaning': 'type 3 correlation', 'val': 3, 'desc': ''}, 'TRM': {'meaning': 'track TERMINATED', 'val': 1, 'desc': ''}, 'FX': {'meaning': 'end of data item', 'val': 0, 'desc': 'Field extension'}, 'ADD': {'meaning': 'Track updated using Aircraft Derived Data', 'val': 1, 'desc': ''}, 'SLR': {'meaning': 'slant range corrected track coordinates using tracked MODE C information.', 'val': 0, 'desc': ''}, 'CNF': {'meaning': 'Confirmed track', 'val': 0, 'desc': ''}, 'LIV': {'meaning': 'Live target', 'val': 0, 'desc': ''}, 'CRE': {'meaning': 'default value', 'val': 0, 'desc': ''}, 'CST': {'meaning': 'coasted track', 'val': 1, 'desc': ''}, 'TYPE': {'meaning': 'combined (P+S) monoradar track', 'val': 4, 'desc': ''}}}

Which OS are you using ?

[zouyunkaicauc]

I'm using the Linux version 4.13.0-21-generic (buildd@lgw01-amd64-037) (gcc version 7.2.0 (Ubuntu 7.2.0-8ubuntu3))
I changed a computer,using the Linux version 4.9.0.7-amd64 (debian-kernel@lists.debian.org)(gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1))
no problem

[dsalantic]

OK, good that you now have version that doesn't crash.
I'll try to reproduce the problem on your version.

Note that the output from your sample is not complete because there are some errors in CAT30 packets. Unfortunately you can't see errors in Python version, they are only visible in C++ executable.
I have in my todo list the implementation of error reporting for Python version (Issue #82).

[zouyunkaicauc]

OK, thank you for your help！

[dsalantic]

Thanks to your sample I managed to reproduce the problem on latest version of python.
It is fixed now and you can use new version of python_decoder v0.5.10
Thank you for your help

[zouyunkaicauc]

Hi, You've said”Currently parsing in python does not return error indication if something is wrong with input data or format definition, but it just ignores the error and returns what is available”in a issue. Does 'something is wrong with input data or format definition' means that the input catxx package itself contains errors, or the module is currently unable to decode all kinds of data item in a catxx package?

…

------------------ 原始邮件 ------------------ 发件人: "Damir Salantic"<notifications@github.com>; 发送时间: 2019年8月29日(星期四) 晚上6:46 收件人: "CroatiaControlLtd/asterix"<asterix@noreply.github.com>; 抄送: "邹云开"<zouyunkaicauc@qq.com>;"Author"<author@noreply.github.com>; 主题: Re: [CroatiaControlLtd/asterix] About decodeing the cat30 packetswith longer length (#92) Thanks to your sample I managed to reproduce the problem on latest version of python. It is fixed now and you can use new version of python_decoder v0.5.10 Thank you for your help — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.