Hamming code correction method (Glonass)

[koloboxer]

I have added Hamming code correction method for Glonass signal and google tests for this method. I have also replaced the call to CRC_test by the call to this new method, so that errors in messages are automatically corrected if possible.

The code works as described in Glonass ICD 5.1, with only a single difference: the document says that if

C_1 = C_2 = ... = C_7 = 0
C_Sigma = 1

then the error is not recoverable. My experiments show that this case just means that all information bits are valid and there is a single error in C_Sigma. Therefore setting C_Sigma to 0 completely corrects the string.

Please, let me know if I am wrong, i.e. you can create a case

C_1 = C_2 = ... = C_7 = 0
C_Sigma = 1

by introducing 2 errors in a valid message.

[vladisslav2011]

Have you seen my commit, that does the same thing 533e107 and discussion here #655 ?

[koloboxer]

Have you seen my commit, that does the same thing 533e107 and discussion here #655 ?

I have not seen it, thanks for the link. I will take a look and write back.

[koloboxer]

My experiments show that this case just means that all information bits are valid and there is a single error in C_Sigma

So the words "multiple errors" is a mistake? Anyway, mistake in checksum is not correctable, because that just means that there may be another mistake in some other bit of checksum and also mistakes in data and here you go, right checksum for wrong data. If you cannot get the checksum bits reliably, there is no point in trying to recover. That is a consensus for all error resilence AFAIK.

Did you test with real satellites or just math?

It is just math, which is reflected in google tests. I take a valid string and generate all possible errors in this string. First all possible single bit errors, then all possible two bit errors.

Can you tell me, please, any reference to this consensus? Because I am not sure I understand the reasoning... If I get a message, which has all sums valid, it does not mean anything, because these correction codes are just probability based and I can change 3 bits in a valid message, keeping all the check sums valid. So I guess there should be some research which says "OK, the strategy without recovering errors gives correct results in X % of cases which is larger than Y % for a strategy with error correction", or vice versa. Right?

Straightforward estimate seems to be in favor of correction in case of low error rate: If probability of error in a single bit is p, then for a message of n bits the probability of a valid message is

p_0 = (1-p)^n

Probability of a message with a single error is

p_1 = (1-p)^{n-1} p n

And the rest is

p_{rest} = 1 - p_0 - p_1

For small enough p we can have p_rest < p_1 which means that it is better to correct a message.

[vladisslav2011]

In real world, if you get a message, that requires error correction, chances, that next message will be uncorrectable, are very high. And chances, that you'll have correct navigation message decoded when you receive all required strings are very low.
I've made an IQ recording from a vehicle-mounted antenna (~26Gb of data) and did some tests. According to test results, enabling ephemeris validation and rejection gives noticeable performance improvement, enabling ECC does not give noticeable performance improvement, compared to 'main' branch.

[vladisslav2011]

Almost full recording was made in a "bad environment". The skyview was blocked by trees and buildings most of time. Only ~5 minutes of clean skyview (not continous). And, while driving through a city, there were some places with strong interference.
https://www.youtube.com/watch?v=QvpAruQmgy8

[koloboxer]

Everywhere when parity bit does not pass you just remove the string.

As far as I understand the parity bit is needed for correction only. Nothing more. If there is no intent to correct the message, the parity is not needed. If I have invalid parity, there are two options: single bit error error in parity or two or more bit error somewhere else. If the probability of single error is higher, I win by correcting the parity. Right?

[koloboxer]

In real world, if you get a message, that requires error correction, chances, that next message will be uncorrectable, are very high.

I am sorry, I did not understand the results. Was it better with correction in your experiments or without correction?

[vladisslav2011]

Was it better with correction in your experiments or without correction?

I have not seen any significant improvement with correction enabled.

[vladisslav2011]

I had several runs with debug message enabled, but I have not implemented error counters. OK. I'll add counters (error-free strings, 1-bit errors and uncorrectable) and re-test.

[vladisslav2011]

Some statistics:

Total GNSS-SDR run time: 243.291980 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:words_decoded=3547
Glonass_Gnav_Navigation_Message:words_corrected=121 (3.411%)
Glonass_Gnav_Navigation_Message:words_dropped=444 (12.518%)


Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 172.699069 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:words_decoded=3564
Glonass_Gnav_Navigation_Message:words_corrected=113 (3.171%)
Glonass_Gnav_Navigation_Message:words_dropped=464 (13.019%)


Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 172.801854 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:words_decoded=3541
Glonass_Gnav_Navigation_Message:words_corrected=115 (3.248%)
Glonass_Gnav_Navigation_Message:words_dropped=425 (12.002%)

It looks, like percentage is incorrect... It is calculated against successful decodes instead of total words processed.
I think, I should count successful decodes of navigation messages, containing words with 1-bit errors.

[vladisslav2011]

words should be strings here.
Another 3 runs with more stats:

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 170.387228 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3613
Glonass_Gnav_Navigation_Message:strings_corrected=105 (2.529%)
Glonass_Gnav_Navigation_Message:strings_dropped=434 (10.453%)
Glonass_Gnav_Navigation_Message:ephemeris_total=204
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=22 (10.784)%

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 171.052081 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3563
Glonass_Gnav_Navigation_Message:strings_corrected=101 (2.462%)
Glonass_Gnav_Navigation_Message:strings_dropped=439 (10.699%)
Glonass_Gnav_Navigation_Message:ephemeris_total=203
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=20 (9.852)%

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 172.540794 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3592
Glonass_Gnav_Navigation_Message:strings_corrected=104 (2.531%)
Glonass_Gnav_Navigation_Message:strings_dropped=413 (10.051%)
Glonass_Gnav_Navigation_Message:ephemeris_total=203
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=18 (8.867)%

C_1 = C_2 = ... = C_7 = 0
C_Sigma = 1

I have not tested for that specific case .

• It looks, like I should count ECC correction events in strings 1,2,3,4 only. So percentage of ephemeris with ECC will be less.

[vladisslav2011]

I think, I have obtained more or less correct stats:

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 171.495172 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3589
Glonass_Gnav_Navigation_Message:strings_corrected=102 (2.470%)
Glonass_Gnav_Navigation_Message:strings_dropped=439 (10.630%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=1 (0.024%)
Glonass_Gnav_Navigation_Message:ephemeris_total=203
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=9 (4.433)%

It may be more interesting to test, how many ephemeris, decoded with ECC are dropped by the validator later.

[vladisslav2011]

3 more runs:

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 173.250314 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3533
Glonass_Gnav_Navigation_Message:strings_corrected=97 (2.415%)
Glonass_Gnav_Navigation_Message:strings_dropped=387 (9.634%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=0 (0.000%)
Glonass_Gnav_Navigation_Message:ephemeris_total=200
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=9 (4.500)%

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 174.797389 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3572
Glonass_Gnav_Navigation_Message:strings_corrected=113 (2.737%)
Glonass_Gnav_Navigation_Message:strings_dropped=444 (10.753%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=1 (0.024%)
Glonass_Gnav_Navigation_Message:ephemeris_total=204
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=8 (3.922)%

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 173.863523 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3520
Glonass_Gnav_Navigation_Message:strings_corrected=116 (2.869%)
Glonass_Gnav_Navigation_Message:strings_dropped=407 (10.067%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=0 (0.000%)
Glonass_Gnav_Navigation_Message:ephemeris_total=201
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=10 (4.975)%

[vladisslav2011]

More, than 50% of ephemeris with 1-bit errors corrected are passing validation. Not as bad as I thought.

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 171.794032 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3558
Glonass_Gnav_Navigation_Message:strings_corrected=104 (2.557%)
Glonass_Gnav_Navigation_Message:strings_dropped=405 (9.958%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=0 (0.000%)
Glonass_Gnav_Navigation_Message:ephemeris_total=200
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=9 (4.500%)
Glonass_Gnav_Navigation_Message:ephemeris_dropped=11 (5.500%)
Glonass_Gnav_Navigation_Message:ephemeris_dropped_ecc=4 (2.000%)

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 174.710891 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3580
Glonass_Gnav_Navigation_Message:strings_corrected=106 (2.583%)
Glonass_Gnav_Navigation_Message:strings_dropped=418 (10.185%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=0 (0.000%)
Glonass_Gnav_Navigation_Message:ephemeris_total=202
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=10 (4.950%)
Glonass_Gnav_Navigation_Message:ephemeris_dropped=11 (5.446%)
Glonass_Gnav_Navigation_Message:ephemeris_dropped_ecc=5 (2.475%)

Current receiver time: 22 min 49 s
Stopping GNSS-SDR, please wait!
Total GNSS-SDR run time: 173.525739 [seconds]
GNSS-SDR program ended.
Glonass_Gnav_Navigation_Message:strings_decoded=3583
Glonass_Gnav_Navigation_Message:strings_corrected=111 (2.708%)
Glonass_Gnav_Navigation_Message:strings_dropped=405 (9.880%)
Glonass_Gnav_Navigation_Message:strings_dropped_csigma=0 (0.000%)
Glonass_Gnav_Navigation_Message:ephemeris_total=199
Glonass_Gnav_Navigation_Message:ephemeris_with_ecc=8 (4.020%)
Glonass_Gnav_Navigation_Message:ephemeris_dropped=11 (5.528%)
Glonass_Gnav_Navigation_Message:ephemeris_dropped_ecc=3 (1.508%)

Tests are here: https://github.com/vladisslav2011/gnss-sdr/tree/glonass_ephemeris_validation_test commit 3cbfc6b.

[vladisslav2011]

I have not tested Glonass in assisted mode, but I don't think, that published ephemeris are much different from broadcast ephemeris.
Getting ephemeris from Google SUPL may help to decrease time to first fix in case of GPS/Galileo?/BeiDou, but not Glonass. You'll always have to wait until the first ephemeris set will be received from minimum 4 satellites. On the other hand time to first fix is almost always less, than in case of unassisted GPG/Galileo/BeiDou.

For RTK it is always better to setup a base station in a place with known coordinates not far from the place, where actual measurements are to be done. You'll get corrected pseudoranges and phase measurements either in real-time or as a RINEX file to do post-processing.

[vladisslav2011]

APPS is different thing. It is GPS-only. And, as I can see, they are offering post-processing only.
There is WAAS/SBAS telemetry decoder in GNSS-SDR, but there are no acquisition/tracking adapters (pcps_acquisition and dll_pll_veml_tracking blocks should work). So I have not tested it too and can't confirm that it works with Glonass (at least ionospheric delay compensation should work).

Are you sure you are not getting this cosmetic issue

No. Read the code of this method: https://github.com/gnss-sdr/gnss-sdr/blob/main/src/core/system_parameters/glonass_gnav_navigation_message.cc#L234 especially this line https://github.com/gnss-sdr/gnss-sdr/blob/main/src/core/system_parameters/glonass_gnav_navigation_message.cc#L363 and it's outer blocks.

[vladisslav2011]

Does this string pass the validator if you correct the last bit (parity bit)?

This would be hard to test as this is a very rare case. One error per 4..5 runs.

Does it mean noise/garbage is passing validation?

Noise/garbage may pass the validation only when validator_accept_first is enabled and only once per satellite. Chances, that two ephemeris sets with exactly the same set of flipped bits will be received without at least one ephemeris set with different set of flipped bits inbetween are very, very low.

Can we implement some other AGPS service so that it would not be needed, or is it some kind of data that cannot be duplicated on the Internet?

No. It should receive time from each satellite before trying to solve for the PVT. Glonass satellites transmit time information once per 30 seconds (L1OF, L2OF signals). GPS satellites transmit time information once per 6 seconds.

[vladisslav2011]

that means the original data was noise

The original data is not noise as it has correct preamble. This means, that there are more, than 2 bits flipped.

In the last case only 3 satelities are needed in fact to get 3D fix.

Knowledge of absolute time does not help because delay from antenna to tracking/acquisition block is unknown and unstable.
3 satellites are enough to get 3D fix using single observation if approximate altitude is known and the constellation is good. 2 satellites are enough to get 3D fix using multiple observations if the receiver is fixed and a stable clock (ex. rubidium) is available.