UVSQ-SAT Error while submitting telemetry: HTTP Error 500 via SIDS

[janvgils]

Good evening,

From the beginning of the UVSQ-SAT launch and activation I have tried to forward gr_satellite decoded frames to there own SIDS database, but without any luck. This evening I tried again and with the same outcome.

Every time I use gr_satellites with the submit_tlm=yes in ${HOME}/.gr_satellites/config.ini including the callsign, latitude and longitude I received an error while submitting telemetry: HTTP Error 500

To make sure it isn't a database issue on the UVSQ-SAT server side, I have configured the SIDS forwarder made by DK3WN to forward the data to https://amsat.electrolab.fr/api/V2/SIDS and this is working as expected, no error 500.

So for some reason the SIDS config in the satyaml file or the underlying code is causing an error 500.

Below, more information regarding the commands, files and output that I used or is generated.

gr_satellites UVSQ-SAT --wavfile UVSQ-Sat_9k6_satnogs_3585560_2021-02-03T09-49-28.wav --samp_rate 48e3 --f_offset 12e3 --hexdump

***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 BPSK downlink))
pdu_length = 239
contents = 
0000: 98 82 a8 9a 9e a6 e0 98 82 a8 9a 9e a6 63 03 f0 
0010: 08 01 c0 00 00 d8 20 03 19 00 00 00 02 38 70 48 
0020: b4 00 00 00 0f 03 81 ca 03 ac db 00 02 bf 87 00 
0030: 00 00 70 00 00 00 0b 00 00 00 4c 00 01 bf 24 19 
0040: ce f9 64 3a 8c 3f 22 47 a1 69 14 9a f8 64 77 06 
0050: 44 a4 63 f4 24 9a 72 91 49 af ff ff ff ff ff ff 
0060: ff ff ff ff ff ff ff ff ff 02 54 00 00 07 fe 03 
0070: 72 1f af 06 4c 01 9d 1f bf ff 9a ff e7 00 67 00 
0080: 00 80 00 0b 94 1f af 14 11 07 fe 1f 94 05 77 02 
0090: a9 14 12 00 0a 00 06 14 0d 01 a8 00 42 ff 52 ff 
00a0: fe 00 00 0d 5f 01 ad 00 30 0d 62 00 57 00 08 1a 
00b0: 05 41 01 80 01 00 00 00 0c 79 26 00 00 02 12 00 
00c0: a4 00 06 00 00 00 13 00 00 00 45 00 bc 00 0d 00 
00d0: bd 01 8f 00 09 ff ff dc ad ff ff d1 14 ff ff da 
00e0: c2 ff ff da be 00 00 7c 34 ff ff bb ba cf 8f 
***********************************
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 

name: UVSQ-SAT
norad: 47438
telemetry_servers:
  - SIDS https://amsat.electrolab.fr/api/V2/SIDS
data:
  &tlm Telemetry:
    telemetry: ax25
transmitters:
  1k2 BPSK downlink:
    frequency: 437.020e+6
    modulation: BPSK
    baudrate: 1200
    framing: AX.25 G3RUH
    data:
    - *tlm
  9k6 BPSK downlink:
    frequency: 437.020e+6
    modulation: BPSK
    baudrate: 9600
    framing: AX.25 G3RUH
    data:
    - *tlm

Below a string that is send by the DK3WN forwarder and that is excepted by the UVSQ-SAT database (I removed some private information)

noradID=47438&source=callsign&timestamp=2021-02-04T21:16:06.020Z&frame=9882A89A9EA6E09882A89A9EA66303F00801C00000CB2003190000000238703ACC0000001338703AB001001E959595950064006400C8A53700C9C84D00CA26FC01002441015F079D015F04C8015EEA78015EC65800E4462600CB102C00C26CFD00C81CC900CECAB900E2D8B6003BA60C00BAEC2C00D706A900C0E7C600D8429300C02EF900F1AF14003AB1D700BDF3D300BE477E00C0318900CEEDE800BB37D300BD5DE3003B6C0300E8DF9200C7196300C3440B00BF19A200BF362B00D9846C006A28E601AF0117FE780100FF73009E0024FF12FFC6000F03003AF30002BDD86DE5&locator=longLat&longitude=removed&latitude=removed&version=1.3.9

Based on the earlier WSL issues I have also tested this on a Laptop running gr_satellites and there is a small difference, this is also generating Error while submitting telemetry: HTTP Error 400: Bad Request

Error while submitting telemetry: HTTP Error 400: Bad Request
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 500: 
Error while submitting telemetry: HTTP Error 400: Bad Request
Error while submitting telemetry: HTTP Error 400: Bad Request
Error while submitting telemetry: HTTP Error 400: Bad Request

As far as I know, I have used all the options I could think of to debug this issue, if there are any others please let me know. I even created a cap file but forgot the transport is ssl encrypted 👎

[daniestevez]

I've just verified that the telemetry submitter is created using the correct URL in case this was some silly error when parsing the SatYAML to get the URL. Other than this, I don't really have a clue, since the code is the same that is working well with SatNOGS DB (and also your own test server, I guess).

Would this work over HTTP instead of HTTPS? If so, that would make it much easier to see what is the difference between Mike's request and gr-satellites'. If you feel like debugging further, you could even take a look at the code in submit.py. It's quite simple, and you can easily change some of the request parameters, such as the version, to match Mike's, or print the parameters (params) of the request.

[janvgils]

Thanks for the update, I will have a look.

In meantime I will also reach out to the UVSQ-Sat team, maybe they can also help.

[daniestevez]

I'm taking a look at this right now. I have changed the URL to HTTP in the .yml to be able to see with Wireshark what's happening. The POST request sent by gr_satellites looks completely broken due to some problem with Python types:

POST /api/V2/SIDS?b'noradID=99991&source=EA4GPZ&locator=longLat&longitude=3.0W&latitude=40.0N&version=1.6.6&frame=9882A89A9EA6E09882A89A9EA66303F00801C000008920031900000002387006C4000000151A05A1018007FC037220010228006C2009010900480067000080000BE82001141007FC20000095004F140A001C0005140D01670038FF55000100000D6101BC00300D630044000800DA00680180004D00EF005E017F004C00E90026018000291A05410180010000000C393B0000021200A40006000000130000A52F&timestamp=2021-02-06T21%3A08%3A56.809Z' HTTP/1.1

Note the Python b'' that encodes a bytes() object.

I wonder how this is working with SatNOGS or other servers.

[daniestevez]

The full HTTP request is as follows:

POST /api/V2/SIDS?b'noradID=99991&source=EA4GPZ&locator=longLat&longitude=3.0W&latitude=40.0N&version=1.6.6&frame=9882A89A9EA6E09882A89A9EA66303F00801C000008920031900000002387006C4000000151A05A1018007FC037220010228006C2009010900480067000080000BE82001141007FC20000095004F140A001C0005140D01670038FF55000100000D6101BC00300D630044000800DA00680180004D00EF005E017F004C00E90026018000291A05410180010000000C393B0000021200A40006000000130000A52F&timestamp=2021-02-06T21%3A08%3A56.809Z' HTTP/1.1
Accept-Encoding: identity
Content-Type: application/x-www-form-urlencoded
Content-Length: 453
Host: amsat.electrolab.fr
User-Agent: Python-urllib/3.7
Connection: close

noradID=99991&source=EA4GPZ&locator=longLat&longitude=3.0W&latitude=40.0N&version=1.6.6&frame=9882A89A9EA6E09882A89A9EA66303F00801C000008920031900000002387006C4000000151A05A1018007FC037220010228006C2009010900480067000080000BE82001141007FC20000095004F140A001C0005140D01670038FF55000100000D6101BC00300D630044000800DA00680180004D00EF005E017F004C00E90026018000291A05410180010000000C393B0000021200A40006000000130000A52F&timestamp=2021-02-06T21%3A08%3A56.809Z

So I guess that the UVSQ-SAT server is choking at the malformed URL, while the other servers ignore it and look it directly at the POST content.

[janvgils]

Great find and an example of thinking out of the box, I need to to remember this one.
The server receiving the strings still finds the required fields and just processes the data.

[daniestevez]

I have pushed 081e7d2, which fixes the problem with b'' appearing into the URL. Now the request looks reasonable.

POST /api/V2/SIDS?noradID=99991&source=EA4GPZ&locator=longLat&longitude=3.0W&latitude=40.0N&version=1.6.6&frame=9882A89A9EA6E09882A89A9EA66303F00801C000008920031900000002387006C4000000151A05A1018007FC037220010228006C2009010900480067000080000BE82001141007FC20000095004F140A001C0005140D01670038FF55000100000D6101BC00300D630044000800DA00680180004D00EF005E017F004C00E90026018000291A05410180010000000C393B0000021200A40006000000130000A52F&timestamp=2021-02-06T21%3A19%3A55.905Z HTTP/1.1
Accept-Encoding: identity
Content-Type: application/x-www-form-urlencoded
Content-Length: 453
Host: amsat.electrolab.fr
User-Agent: Python-urllib/3.7
Connection: close

noradID=99991&source=EA4GPZ&locator=longLat&longitude=3.0W&latitude=40.0N&version=1.6.6&frame=9882A89A9EA6E09882A89A9EA66303F00801C000008920031900000002387006C4000000151A05A1018007FC037220010228006C2009010900480067000080000BE82001141007FC20000095004F140A001C0005140D01670038FF55000100000D6101BC00300D630044000800DA00680180004D00EF005E017F004C00E90026018000291A05410180010000000C393B0000021200A40006000000130000A52F&timestamp=2021-02-06T21%3A19%3A55.905Z

However I'm still getting errors:

Error while submitting telemetry: HTTP Error 405:

I can't really debug this using HTTP instead of HTTPS because the server returns a 301 Moved Permanently redirect to the HTTPS URL.

[daniestevez]

I noted two differences in the request from DK3WN that you pasted here. The noradID was different (so I've tested also the NORAD used by DK3WN, and it doesn't fix the problem), and the colons (:) in the timestamp are not escaped (I'm not sure if this could be a reason why the UVSQ-Sat server is choking).

[janvgils]

I will test this with a recording to see if that makes any difference, my system is building as we speak.
After the test I will share the results.

[janvgils]

Nope, just tested it and the same result.

Error while submitting telemetry: HTTP Error 400: Bad Request
Error while submitting telemetry: HTTP Error 500: 

[daniestevez]

Tweaking a bit the code I'm now able to print the body of the HTTP responses sent by the server:

For a 400 Bad Request, I get

Server: nginx
Date: Sat, 06 Feb 2021 21:52:27 GMT
Content-Length: 0
Connection: close
Vary: Accept, Cookie
Allow: GET, POST, HEAD, OPTIONS
Content-Security-Policy: img-src 'self' https://*.gravatar.com https://*.mapbox.com https://*.google-analytics.com data: blob: https://db-satnogs.freetls.fastly.net; worker-src 'self' blob:; default-src 'self' https://*.mapbox.com 'unsafe-inline' https://db-satnogs.freetls.fastly.net; child-src blob:; script-src 'self' https://*.google-analytics.com 'unsafe-eval' https://db-satnogs.freetls.fastly.net; frame-src blob:
X-Frame-Options: DENY

For a 500 I get

Server: nginx/1.15.9
Date: Sat, 06 Feb 2021 21:52:27 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: close

I'm inclined to think that the UVSQ-Sat server is broken. After fixing the problem with the b'''s, the HTTP request sent by gr_satellites looks completely reasonable as far as I can tell.

[janvgils]

I had a better look at the cap files that I created and found the following difference:

When using the UVSQsatDecoder the time field is different:

    Form item: "timestamp" = "2021-02-06T22:17:46Z"
        Key: timestamp
        Value: 2021-02-06T22:17:46Z

This structure isn't recognised as a valid field value and therefor the test SIDS database is rejecting it.

Comparing this to GetKISS+ I have the following:

    Form item: "timestamp" = "2021-02-06T22:24:24.090Z"
        Key: timestamp
        Value: 2021-02-06T22:24:24.090Z

It could be possible that the UVSQ-SAT sids database is using a different timestamp field check and therefor rejects the frames.

I guess we need some help from the UVSQ-SAT team.

[janvgils]

I tried to reproduce this error with other SIDS database servers but I am unable to.
So it seems to be specific to the UVSQ-Sat SIDS configuration and I can't check anything further without a clear understanding of there SIDS server/database setup.

I am out of options.

[daniestevez]

I'm also out of options, other than rewriting the gr-satellites submit block using requests instead of urllib, which might fix things. I can't see anything wrong with the HTTP POST requests sent by gr-satellites, but for some reason the UVSQ-Sat server doesn't like them.

This is something to consider for further developments of the SIDS protocol. Maybe a reference implementation would help prevent these sorts of problems?

I'll keep the issue open in case someone from the UVSQ-Sat team wants to weigh in.