Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for D-Star DV Fast Data #470

Open
timclassic opened this issue Nov 7, 2018 · 105 comments
Open

Support for D-Star DV Fast Data #470

timclassic opened this issue Nov 7, 2018 · 105 comments

Comments

@timclassic
Copy link
Contributor

I've been testing D-Star DV "Fast Data" as implemented on the Kenwood TH-D74. I tried sending data via a ZUMspot only to find that it gets corrupted on its way through (juribeparada/MMDVM_HS#71 has an example of the corruption). MMDVMHost also reports a BER of ~12.6%.

I discovered that if I comment out all three calls to m_fec.regenerateDStar() in DStarControl.cpp, the data is able to pass through without corruption. It seems that all 72 bits of each voice frame are used for data and there is no FEC applied, which also probably explains the ~12.6% BER report.

Some things I don't know:

  1. Is there a way to tell when a voice frame contains "fast data" instead of voice+FEC?
  2. Do Icom and Kenwood implement the same fast data spec? (I hope so)
  3. Why is the FEC recalculated and reapplied in MMDVMHost? Is it so that the BER reports are effectively per-hop? Is there something important to the protocol going on?

To answer question 1, I'm planning to look for a difference in the packet that indicates whether voice or data is contained within the voice frames. I think there must be a way to tell the difference, because the demos I've seen of Icom radios doing fast data allow the user seamlessly transmit audio while a data transfer is ongoing and the radio degrades to slow data while sending voice frames. (The Kenwood does not seem to have this slow data capability)

I'm going to email Kenwood and Icom in an effort to answer question 2 (and maybe question 1?) and also ask if they have any specs to share.

I'm hoping that someone reading this can answer question 3 for me.

I'm interested in all this because I think that fast data + hotspots and the reflector system could be a powerful combination.

-TimS, KG4BXH

@g4klx
Copy link
Owner

g4klx commented Nov 7, 2018 via email

@timclassic
Copy link
Contributor Author

Jonathan,

Now it totally makes sense why you would want to regenerate the FEC at each hop. Even though "error correction" is in the name, I was thinking of it as a mere checksum. I suppose it is lucky that the currently-installed Icom repeaters don't recalculate FEC, at least from the perspective of using fast data through them. I certainly don't have the experience to know how much it would improve repeater use if the repeaters also did the recalculation.

I will grab some traces and post the results here. Thank you for the quick guide. If I can find the flag, I should be able to put together a patch that disables the recalc only in the case that fast data is in use.

@juribeparada
Copy link
Collaborator

Yes, I know, I follow with attention

@timclassic
Copy link
Contributor Author

Here are my findings so far.

In fast data mode, I've found that the TH-D74 sends the following bytes in the first 21 voice frames before starting to send the actual data:

EE C2 A1 C8 42 6E 52 51 C3

After the 21st frame, the contents of the voice frames begin to vary, presumably with the actual data I tried to send. Indeed, even when I send a single A followed by a LF, the same 21 frames appear.

Here is a trace from the log that was generated by a D-RATS transmission as it started up and advertised that I was online:

D: 2018-11-09 05:06:49.213 RX D-Star Header
D: 2018-11-09 05:06:49.213 0000:  E0 2E 10 40 00 00 4B 47 34 42 58 48 20 47 4B 47    *...@..KG4BXH GKG*
D: 2018-11-09 05:06:49.214 0010:  34 42 58 48 20 42 43 51 43 51 43 51 20 20 4B 47    *4BXH BCQCQCQ  KG*
D: 2018-11-09 05:06:49.214 0020:  34 42 58 48 20 20 44 37 34 20 19 8A 00 2F          *4BXH  D74 .../*
D: 2018-11-09 05:06:49.214 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
M: 2018-11-09 05:06:49.215 D-Star, received RF header from KG4BXH  /D74  to CQCQCQ
M: 2018-11-09 05:06:49.221 Debug: Mode set to D-Star
D: 2018-11-09 05:06:49.232 RX D-Star Data
D: 2018-11-09 05:06:49.232 0000:  E0 11 11 EE C2 A1 C8 42 6E 52 51 C3 55 2D 16 00    *.......BnRQ.U-..*
D: 2018-11-09 05:06:49.232 0010:  2F                                                 */*
D: 2018-11-09 05:06:49.232 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
D: 2018-11-09 05:06:49.249 RX D-Star Data
D: 2018-11-09 05:06:49.249 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 30 1B FA       *.......BnRQ.0..*
D: 2018-11-09 05:06:49.271 RX D-Star Data
D: 2018-11-09 05:06:49.271 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 1D 6F FA       *.......BnRQ..o.*
D: 2018-11-09 05:06:49.288 RX D-Star Data
D: 2018-11-09 05:06:49.289 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 31 21 B3       *.......BnRQ.1!.*
D: 2018-11-09 05:06:49.312 RX D-Star Data
D: 2018-11-09 05:06:49.313 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 3D 2E FD       *.......BnRQ.=..*
D: 2018-11-09 05:06:49.329 RX D-Star Data
D: 2018-11-09 05:06:49.329 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 32 27 F2       *.......BnRQ.2'.*
D: 2018-11-09 05:06:49.351 RX D-Star Data
D: 2018-11-09 05:06:49.351 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 04 3B F2       *.......BnRQ..;.*
D: 2018-11-09 05:06:49.373 RX D-Star Data
D: 2018-11-09 05:06:49.374 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 33 21 B3       *.......BnRQ.3!.*
D: 2018-11-09 05:06:49.391 RX D-Star Data
D: 2018-11-09 05:06:49.393 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 50 6F B3       *.......BnRQ.Po.*
D: 2018-11-09 05:06:49.411 RX D-Star Data
D: 2018-11-09 05:06:49.413 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.432 RX D-Star Data
D: 2018-11-09 05:06:49.433 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.451 RX D-Star Data
D: 2018-11-09 05:06:49.453 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.470 RX D-Star Data
D: 2018-11-09 05:06:49.472 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.489 RX D-Star Data
D: 2018-11-09 05:06:49.491 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.509 RX D-Star Data
D: 2018-11-09 05:06:49.510 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.529 RX D-Star Data
D: 2018-11-09 05:06:49.530 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.549 RX D-Star Data
D: 2018-11-09 05:06:49.550 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.573 RX D-Star Data
D: 2018-11-09 05:06:49.574 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.591 RX D-Star Data
D: 2018-11-09 05:06:49.593 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.610 RX D-Star Data
D: 2018-11-09 05:06:49.612 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.630 RX D-Star Data
D: 2018-11-09 05:06:49.631 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5       *.......BnRQ..).*
D: 2018-11-09 05:06:49.650 RX D-Star Data
D: 2018-11-09 05:06:49.652 0000:  E0 11 11 0E 31 ED 3E 66 0A 13 4E 0A 55 2D 16 00    *....1.>f..N.U-..*
D: 2018-11-09 05:06:49.654 0010:  2F                                                 */*
D: 2018-11-09 05:06:49.656 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
D: 2018-11-09 05:06:49.669 RX D-Star Data
D: 2018-11-09 05:06:49.670 0000:  E0 0F 11 2D 6D AE 00 66 49 2D 0D 6B EC 14 C0       *...-m..fI-.k...*
D: 2018-11-09 05:06:49.689 RX D-Star Data
D: 2018-11-09 05:06:49.690 0000:  E0 0F 11 4D 71 A4 88 66 49 2D 38 0A EC 00 D1       *...Mq..fI-8....*
D: 2018-11-09 05:06:49.709 RX D-Star Data
D: 2018-11-09 05:06:49.710 0000:  E0 0F 11 0E 31 ED 41 66 75 6C 31 2A E4 31 ED       *....1.Aful1*.1.*
D: 2018-11-09 05:06:49.729 RX D-Star Data
D: 2018-11-09 05:06:49.730 0000:  E0 0F 11 71 4E 92 1B 66 31 22 72 76 E4 31 ED       *...qN..f1"rv.1.*
D: 2018-11-09 05:06:49.753 RX D-Star Data
D: 2018-11-09 05:06:49.754 0000:  E0 0F 11 2D 92 AE 00 66 49 2D 31 2E E4 14 C0       *...-...fI-1....*
D: 2018-11-09 05:06:49.772 RX D-Star Data
D: 2018-11-09 05:06:49.774 0000:  E0 0F 11 D0 0F AE 00 66 6C 26 77 1F E4 00 D1       *.......fl&w....*
D: 2018-11-09 05:06:49.792 RX D-Star Data
D: 2018-11-09 05:06:49.794 0000:  E0 0F 11 0E 0C C2 03 66 25 2E 61 55 E4 0D CB       *.......f%.aU...*
D: 2018-11-09 05:06:49.812 RX D-Star Data
D: 2018-11-09 05:06:49.813 0000:  E0 0F 11 0E 37 49 73 66 80 A2 FB E2 E4 07 ED       *....7Isf.......*
D: 2018-11-09 05:06:49.832 RX D-Star Data
D: 2018-11-09 05:06:49.833 0000:  E0 0F 11 00 9E 9E 32 66 78 64 E6 2F E4 83 D8       *......2fxd./...*
D: 2018-11-09 05:06:49.851 RX D-Star Data
D: 2018-11-09 05:06:49.853 0000:  E0 0F 11 4D 0F B9 CC 66 70 DE 6B 6E E4 1A 43       *...M...fp.kn..C*
D: 2018-11-09 05:06:49.870 RX D-Star Data
D: 2018-11-09 05:06:49.871 0000:  E0 0F 11 16 29 F5 26 66 12 0B 56 4D F3 00 D1       *....).&f..VM...*
D: 2018-11-09 05:06:49.889 RX D-Star Data
D: 2018-11-09 05:06:49.890 0000:  E0 0F 11 16 29 F5 26 66 12 0B 56 4D F3 12 F5       *....).&f..VM...*
D: 2018-11-09 05:06:49.909 RX D-Star Data
D: 2018-11-09 05:06:49.910 0000:  E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 55 55 55       *.......BnRQ.UUU*
M: 2018-11-09 05:06:49.916 Debug: DStarRX: Found end sync in Data
D: 2018-11-09 05:06:49.923 RX D-Star EOT
D: 2018-11-09 05:06:49.925 0000:  E0 03 13                                           *...*
M: 2018-11-09 05:06:49.926 D-Star, received RF end of transmission, 0.7 seconds, BER: 0.0%, RSSI: -47/-47/-47 dBm

So, my theory is that there is something special about EE C2 A1 C8 42 6E 52 51 C3, such that it won't be generated by AMBE+FEC, or perhaps it's just extremely unlikely to occur 21 times in a row. Further, I imagine that 9-byte sequence itself validates against FEC (just guessing, I don't yet know how to check this).

Anyway, if my theory is correct, we should be able to support fast data through hotspots (and repeaters?) by turning off FEC recalculation in the case that we see 21 of these in a row.

I wonder if there's a "re-enable voice" sequence that indicates we should re-enable FEC, supporting Icom's apparent ability to move between the two modes in a single transmission. (I say "apparent" because I think this is possible based on the Icom video that demos text and image sending while also being able to send voice during the transfer). Maybe that's what 16 29 F5 26 66 12 0B 56 4D is for?

@timclassic
Copy link
Contributor Author

I should note that this dump was created by MMDVMHost with the FEC recalculation disabled, which is why you see a 0% BER.

Also, I said above that "I imagine that 9-byte sequence itself validates against FEC." If memory serves, these bytes contributed to the 12.6% BER in my original tests, so I think my statement is wrong. I'll double-check and confirm, though I don't think it affects my theory much overall.

@g4klx
Copy link
Owner

g4klx commented Nov 11, 2018

That is very very interesting. What you say makes sense. I think I can move forwards with what you have told me. If you could investigate the Icom mode switch, and also if Icom use the same system, I would appreciate it. I have some ideas and I'll try and implement them next week. Unfortunately the gateway will also need modifying which is not ideal, but necessary.

@juribeparada
Copy link
Collaborator

Hi, I have done some tests with the same file (sent.log) and my ID51 with Fast DV data. Unfortunately it looks different from Kenwood format:

D: 2018-11-11 23:54:54.411 RX D-Star Header
D: 2018-11-11 23:54:54.411 0000:  E0 2E 10 40 00 00 43 41 36 4A 41 55 20 47 43 41    *...@..CA6JAU GCA*
D: 2018-11-11 23:54:54.411 0010:  36 4A 41 55 20 43 20 20 20 20 20 20 20 45 43 41    *6JAU C       ECA*
D: 2018-11-11 23:54:54.411 0020:  36 4A 41 55 20 20 20 20 20 20 56 C0 00 04          *6JAU      V...*
M: 2018-11-11 23:54:54.411 D-Star, received RF header from CA6JAU  /     to        E
D: 2018-11-11 23:54:54.430 RX D-Star Data
D: 2018-11-11 23:54:54.430 0000:  E0 11 11 B2 4D 22 48 C0 16 28 26 C8 55 2D 16 00    *....M"H..(&.U-..*
D: 2018-11-11 23:54:54.430 0010:  04                                                 *.*
D: 2018-11-11 23:54:54.449 RX D-Star Data
D: 2018-11-11 23:54:54.449 0000:  E0 0F 11 AE CC 2A 78 E1 13 3C 67 C0 30 0E FD       *.....*x..<g.0..*
D: 2018-11-11 23:54:54.468 RX D-Star Data
D: 2018-11-11 23:54:54.468 0000:  E0 0F 11 CE 8E 2E 39 66 12 78 31 B0 14 36 B3       *......9f.x1..6.*
D: 2018-11-11 23:54:54.492 RX D-Star Data
D: 2018-11-11 23:54:54.492 0000:  E0 0F 11 AE CC 2A 78 E1 91 34 67 C0 31 1A E1       *.....*x..4g.1..*
D: 2018-11-11 23:54:54.511 RX D-Star Data
D: 2018-11-11 23:54:54.511 0000:  E0 0F 11 CA 8E 26 19 26 1E 4C A1 F0 19 2D F6       *.....&.&.L...-.*
D: 2018-11-11 23:54:54.529 RX D-Star Data
D: 2018-11-11 23:54:54.529 0000:  E0 0F 11 AE CC 2A 78 E1 13 3C 67 C0 32 6F BC       *.....*x..<g.2o.*
D: 2018-11-11 23:54:54.547 RX D-Star Data
D: 2018-11-11 23:54:54.547 0000:  E0 0F 11 BE 6E 22 48 23 1B 0C 77 80 50 06 D7       *....n"H#..w.P..*
D: 2018-11-11 23:54:54.572 RX D-Star Data
D: 2018-11-11 23:54:54.572 0000:  E0 0F 11 CE 8E 22 09 E4 1E 4C 20 F4 33 7A A2       *....."...L .3z.*
D: 2018-11-11 23:54:54.590 RX D-Star Data
D: 2018-11-11 23:54:54.590 0000:  E0 0F 11 AA CC 2A 78 E1 91 3C 67 C0 35 6F B3       *.....*x..<g.5o.*
D: 2018-11-11 23:54:54.612 RX D-Star Data
D: 2018-11-11 23:54:54.612 0000:  E0 0F 11 CE CE AC 31 47 1E 58 E0 F0 16 29 F5       *......1G.X...).*
D: 2018-11-11 23:54:54.629 RX D-Star Data
D: 2018-11-11 23:54:54.629 0000:  E0 0F 11 AE 8C 2E 68 22 1F 1C 37 84 16 29 F5       *......h"..7..).*
D: 2018-11-11 23:54:54.651 RX D-Star Data
D: 2018-11-11 23:54:54.651 0000:  E0 0F 11 50 26 E0 60 66 15 4D 44 4E E4 1B FB       *...P&.`f.MDN...*
D: 2018-11-11 23:54:54.668 RX D-Star Data
D: 2018-11-11 23:54:54.668 0000:  E0 0F 11 03 3B B3 2F 66 12 4D 44 43 E4 26 E0       *....;./f.MDC.&.*
D: 2018-11-11 23:54:54.691 RX D-Star Data
D: 2018-11-11 23:54:54.691 0000:  E0 0F 11 15 3D F4 25 66 1A 0E 49 0B E4 2A B3       *....=.%f..I..*.*
D: 2018-11-11 23:54:54.708 RX D-Star Data
D: 2018-11-11 23:54:54.708 0000:  E0 0F 11 12 3D FC 21 66 10 0E 51 58 E4 2A FE       *....=.!f..QX.*.*
D: 2018-11-11 23:54:54.732 RX D-Star Data
D: 2018-11-11 23:54:54.732 0000:  E0 0F 11 03 3B F6 2D 66 5A 4D 10 7F E4 3B B3       *....;.-fZM...;.*
D: 2018-11-11 23:54:54.751 RX D-Star Data
D: 2018-11-11 23:54:54.751 0000:  E0 0F 11 18 26 E0 60 66 1D 1E 10 44 E4 3C EA       *....&.`f...D.<.*
D: 2018-11-11 23:54:54.768 RX D-Star Data
D: 2018-11-11 23:54:54.768 0000:  E0 0F 11 11 6F E7 25 66 07 19 1E 26 FD 21 FF       *....o.%f...&.!.*
D: 2018-11-11 23:54:54.793 RX D-Star Data
D: 2018-11-11 23:54:54.793 0000:  E0 0F 11 7A 4F 93 40 66 74 6D 30 2B FD 36 B3       *...zO.@ftm0+.6.*
D: 2018-11-11 23:54:54.811 RX D-Star Data
D: 2018-11-11 23:54:54.811 0000:  E0 0F 11 CE 8E A0 01 C4 1E 4C 20 F4 16 29 F5       *.........L ..).*
D: 2018-11-11 23:54:54.829 RX D-Star Data
D: 2018-11-11 23:54:54.829 0000:  E0 0F 11 AA CC 2E 68 23 91 3C E6 C4 16 29 F5       *......h#.<...).*
D: 2018-11-11 23:54:54.852 RX D-Star Data
D: 2018-11-11 23:54:54.852 0000:  E0 0F 11 CE 8E 2E 39 66 12 78 31 B0 55 55 55       *......9f.x1.UUU*
D: 2018-11-11 23:54:54.852 RX D-Star EOT
D: 2018-11-11 23:54:54.852 0000:  E0 03 13                                           *...*
M: 2018-11-11 23:54:54.858 D-Star, received RF end of transmission, 0.5 seconds, BER: 0.0%

@juribeparada
Copy link
Collaborator

I've realized that not always starts with the same data sequence, after repeating the same transmission with a ID51. For example, if I transmit a single "A" (I "cleaned" the log for better understanding):

70 4F 93 40 64 74 6D 30 2B 55 2D 16       *...pO.@dtm0+U-.*
70 4F 93 40 64 74 6D 30 2B F2 0E 99       *...pO.@dtm0+...*
70 4F 93 40 64 74 6D 30 2B F2 4F 93       *...pO.@dtm0+.O.*
CE 8E 2E 39 66 12 78 31 B0 30 0E FD       *......9f.x1.0..*
AE CC 2A 78 E1 91 34 67 C0 14 36 B3       *.....*x..4g..6.*
CA 8E 26 19 26 1E 4C A1 F0 31 1A E1       *.....&.&.L..1..*
AE CC 2A 78 E1 13 3C 67 C0 19 2D F6       *.....*x..<g..-.*
BE 6E 22 48 23 1B 0C 77 80 32 6F BC       *....n"H#..w.2o.*
CE 8E 22 09 E4 1E 4C 20 F4 50 06 D7       *....."...L .P..*
AA CC 2A 78 E1 91 3C 67 C0 33 7A A2       *.....*x..<g.3z.*
CE CE AC 31 47 1E 58 E0 F0 35 6F B3       *......1G.X..5o.*
AE 8C 2E 68 22 1F 1C 37 84 16 29 F5       *......h"..7..).*
CE CE 2A 29 A5 1E 58 61 F4 16 29 F5       *.....*)..Xa..).*
AA CC 2A 78 E1 13 3C 67 C0 55 55 55       *.....*x..<g.UUU*

And if I repeat the same "A":

B2 4D 22 48 C0 16 28 26 C8 55 2D 16       *....M"H..(&.U-.*
AE CC 2A 78 E1 13 3C 67 C0 30 0E FD       *.....*x..<g.0..*
CE 8E 2E 39 66 12 78 31 B0 14 36 B3       *......9f.x1..6.*
AE CC 2A 78 E1 91 34 67 C0 31 1A E1       *.....*x..4g.1..*
CA 8E 26 19 26 1E 4C A1 F0 19 2D F6       *.....&.&.L...-.*
AE CC 2A 78 E1 13 3C 67 C0 32 6F BC       *.....*x..<g.2o.*
BE 6E 22 48 23 1B 0C 77 80 50 06 D7       *....n"H#..w.P..*
CE 8E 22 09 E4 1E 4C 20 F4 33 7A A2       *....."...L .3z.*
AA CC 2A 78 E1 91 3C 67 C0 35 6F B3       *.....*x..<g.5o.*
CE CE AC 31 47 1E 58 E0 F0 16 29 F5       *......1G.X...).*
AE 8C 2E 68 22 1F 1C 37 84 16 29 F5       *......h"..7..).*
70 4F 93 40 64 74 6D 30 2B F2 0E 99       *...pO.@dtm0+...*
70 4F 93 40 64 74 6D 30 2B F2 4F 93       *...pO.@dtm0+.O.*
CE CE 26 19 27 12 6C 70 B0 16 29 F5       *.....&.'.lp..).*
AE EC 2A 78 E1 13 3C 67 C0 16 29 F5       *.....*x..<g..).*
CE CE 2E 39 67 1E 58 E0 F0 55 55 55       *......9g.X..UUU*

@juribeparada
Copy link
Collaborator

I did more experiments, trying to decode the frames. Now I transmitted: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"

I got:
fastdv_icom

My guess:
A: a kind of data start?. It seems to me always the same, but for some transmissions is not there!
B: sync flag
C: data header?, it looks always the same and it is always there. It doesn't change if I modify D-Star fields, for example.
D: Slow data, here has my TX message "Andy Uribe / ID51E "
E: data payload

The data payload seems to be scrambled with: 70 4F 93 40 64 74 6D 30 2B E2 4F 93 (not sure about 64 and E2). If I de-scramble the payload:

45 46 47 48 02 49 4A 4B 4C 06 41 42                *EFGH.IJKL.AB*
4D 4E 4F 50 02 51 52 53 54 06 43 44                *MNOP.QRST.CD*
59 5A 61 62 02 63 64 65 66 06 55 56                *YZab.cdef.UV*
67 68 69 6A 02 6B 6C 6D 6E 06 57 58                *ghij.klmn.WX*
73 74 75 76 02 77 78 79 7A 06 6F 70                *stuv.wxyz.op*
30 31 32 33 02 34 35 36 37 06 71 72                *0123.4567.qr*
00 00 00 00 02 00 00 00 00 10 38 39                *..........89*
00 00 00 00 02 00 00 00 00 10 00 00                *............*

@g4klx
Copy link
Owner

g4klx commented Nov 12, 2018

It looks like the Kenwood data is the easiest to support. I think the data marker is transmitted multiple times to make sure that you receive it. You'll find that 21 frames occurs a lot in D-Star and is the distance between sync frames, so it sort of makes sense to use that as the basic unit of data transmissions. Once I have reliably detected the data mode, I'd just switch off the FEC code and let it continue. I'd probably also regenerate the data marker also for good measure. I'm still looking at the Icom version. In theory switching off the FEC will be fine if the system switches back to voice, you just won't get the benefit of the FEC for the rest of that transmission. I do wish that Kenwood and Icom had used a bit in the header instead or as well as, to make my life easier.

@juribeparada
Copy link
Collaborator

I am going to repeat the same tests with a ID-4100A and see if I get the same results. Something that I forget to mention: after all my tests with ID-51A, the first frame could be one of the following two possibilities:

  1. The sequence: B2 4D 22 48 C0 16 28 26 C8
  2. Directly an "almost" empty payload: 70 4F 93 40 64 74 6D 30 2B, where the following slow data will contain the first 4 bytes of the message.

Anyway, it seems to me that the large structure "C" is always in any Icom fast DV transmission.

@timclassic
Copy link
Contributor Author

timclassic commented Nov 12, 2018

Unfortunately, I don't have any Icom hardware to test with :( I'm hoping to get an Icom ID-4100A by the end of the year, though.

I have received an initial response from my email to Kenwood:

  • The TH-D74A supports fast DV data
  • The TH-D74A does not support slow DV data (to/from a connected computer; obviously it's using slow data for the display messages)
  • The TH-D74A's fast data is compatible with Icom's fast data

The last point could mean the D74A implements a simpler subset, I suppose, since it doesn't support switching back and forth between fast and slow.

I haven't gotten a negative response from either Icom or Kenwood support about getting tech specs, so I'll update here when I have any answers there.

Maybe there is some shared property of the bytes I saw (EE C2 A1 C8 42 6E 52 51 C3) and block C of @juribeparada's dump, e.g. some checksum they both validate against, and Kenwood just implemented something simpler. But I'm just shooting in the dark here :)

Anyway, I plan to poke around a bit more with the TH-D7A this week and compare to the other posts here, in case I can discover anything.

Thanks for looking at this!

@g4klx
Copy link
Owner

g4klx commented Nov 12, 2018

I've added some code to the D-Star support in the host. It should detect the use of fast data mode, and log it. It should work on RF and network although the ircDDB Gateway needs to be modified before we can send fast data over any network. For now, if you can test/modify the code that I've added for the fast data mode.

@juribeparada
Copy link
Collaborator

I will test at night here

@juribeparada
Copy link
Collaborator

I had little time for testing today, but I could check:

  1. ID-4100A sends the same frames as ID-51 in fast DV mode, with the two "identifiers".
  2. The current MMDVMHost detects both Icom identifiers well, and works well with ID-51 and ID-4100A in fast DV data.

@g4klx
Copy link
Owner

g4klx commented Nov 13, 2018

I've added data marker regeneration on RF and network traffic. I've checked the ircDDB Gateway and it does no AMBE FEC regeneration so fast data should pass through it without problems. The only potential issue is the DTMF decoder which could trigger and blank the data. This will need testing to make sure what goes in, comes out unchanged. I'm hoping that we can get this finished soon.

@g4klx
Copy link
Owner

g4klx commented Nov 13, 2018

BTW in your data, I wonder if element C is your RF header, scrambled. The only way to be sure would be for someone else to generate some Icom fast data with a dissimilar set of callsigns and do a comparison.

@timclassic
Copy link
Contributor Author

I will test the changes tonight.

@juribeparada
Copy link
Collaborator

It seems to me the block "C" doesn't change: I used different D-Star fields YOUR, RPT1, etc (except the callsign, I've used always the same), even I transmitted accidentally in simplex with the 4100, and I captured the same "C" structure, something curious. I will be out of the town for 10 days, then I will be not able to do more tests until come back.

@timclassic
Copy link
Contributor Author

The changes look good to me so far. I was able to send and receive the same text using an echo test, and I get the Kenwood message in the logs.

Now I'm hanging out on REF030D in D-RATS if either of you are around and want to test.

@timclassic
Copy link
Contributor Author

I've shared a file via D-RATS in case one of you decide to test via REF030D while I am away. I realize this may not work yet, but I thought it might be fun to try.

Goodnight!

-TimS, KG4BXH

@g4klx
Copy link
Owner

g4klx commented Nov 14, 2018

That's very pleasing. I am still worried about the DTMF processing in the gateway which could potentially corrupt the data if triggered by random data. I will look to switch it off in data mode. I also need to think about the older D-Star Repeater software, whether to upgrade it or not. It will be interesting if there is a switch in the Icom data mode to go back to audio, as things stand it should work anyway, but any following audio won't have the extra processing that audio gets now. Sorry I wasn't on D-RATS but I live in the UK and the time difference killed that, as well as other things.

@timclassic
Copy link
Contributor Author

I suspected you were in the UK and it was a long shot :) Also, my pi-star decided to restart all services at 3:30am EST anyway, so that killed the test after a while... need to turn that off. So, to be clear, my D-RATS test is now over, heh.

Anyway, I'm still waiting on potential docs from Kenwood and Icom. That would be really nice if one of them could just tell us how it works. I'll see if I can find someone willing to test with me, given that I have only one radio.

@phl0
Copy link
Collaborator

phl0 commented Nov 14, 2018 via email

@g4klx
Copy link
Owner

g4klx commented Nov 14, 2018

I'm in the same boat Flo. I have a TH-D74 and an ID-51 and haven't used fast data on either of them. That's why it's never been added to my work before. I've updated the ircDDB Gateway to bypass all DTMF handling when fast data is detected, but it's on a branch because I can't build it currently. It'll be rolled into master when it's been compiled cleanly.

@phl0
Copy link
Collaborator

phl0 commented Nov 14, 2018 via email

@juribeparada
Copy link
Collaborator

In my case, I enabled Fast DV data from ID-51 menu (DV Set->DV Fast Data->Fast Data->ON), and then simply I send the data using the serial port of the radio at 38400 bps:

echo "Test DV fast data" > /dev/ttyXXX
cat file.txt > /dev/ttyXXX

If fast data is OFF, all data will be sent as slow data as usual.

@phl0
Copy link
Collaborator

phl0 commented Nov 16, 2018

I finally have my fast data equipment up and running. Sorry for the delay. Anyone up for a test (across reflectors)?

@ravny
Copy link

ravny commented Nov 17, 2018

Hi, yes we can do the test just suggest which ref and time.

@timclassic
Copy link
Contributor Author

@phl0 I hope you're asleep right now! If not, I hope to be on 30C in about 30 minutes.

@phl0
Copy link
Collaborator

phl0 commented Jan 31, 2019

@timclassic Are you still on there? If so I am going to start my machines.

@phl0
Copy link
Collaborator

phl0 commented Jan 31, 2019

@timclassic I am on REF030 D now. Seems like I am the only one :) U still there?

@phl0
Copy link
Collaborator

phl0 commented Jan 31, 2019

Is this sufficient for a test result?

bildschirmfoto vom 2019-01-31 06-33-50

@timclassic
Copy link
Contributor Author

I've only tested so far with my TH-D74A. Here are my results with the latest code from master. I did an echo test.

MMDVMHost logs:

D: 2019-01-31 05:35:32.376 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
M: 2019-01-31 05:35:32.376 D-Star, received RF header from KG4BXH  /D74  to        E
D: 2019-01-31 05:35:32.393 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
D: 2019-01-31 05:35:32.394 D-Star, audio sequence no. 0, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.411 D-Star, audio sequence no. 1, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.433 D-Star, audio sequence no. 2, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.455 D-Star, audio sequence no. 3, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.472 D-Star, audio sequence no. 4, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.494 D-Star, audio sequence no. 5, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.512 D-Star, audio sequence no. 6, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.531 D-Star, audio sequence no. 7, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.551 D-Star, audio sequence no. 8, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.574 D-Star, audio sequence no. 9, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.594 D-Star, audio sequence no. 10, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.612 D-Star, audio sequence no. 11, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.631 D-Star, audio sequence no. 12, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.655 D-Star, audio sequence no. 13, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.672 D-Star, audio sequence no. 14, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.692 D-Star, audio sequence no. 15, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.715 D-Star, audio sequence no. 16, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.733 D-Star, audio sequence no. 17, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.751 D-Star, audio sequence no. 18, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.771 D-Star, audio sequence no. 19, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.791 D-Star, audio sequence no. 20, errs: 0/48 (0.0%)
D: 2019-01-31 05:35:32.814 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
D: 2019-01-31 05:35:32.816 D-Star, audio sequence no. 0, errs: 7/48 (14.6%)
M: 2019-01-31 05:35:32.831 D-Star, switching to fast data mode
D: 2019-01-31 05:35:33.232 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
M: 2019-01-31 05:35:33.337 D-Star, received RF end of transmission, 1.0 seconds, BER: 0.0%, RSSI: -47/-47/-47 dBm
M: 2019-01-31 05:35:35.339 D-Star, received network header from KG4BXH  /ECHO to CQCQCQ  
M: 2019-01-31 05:35:35.801 D-Star, switching to fast data mode
M: 2019-01-31 05:35:36.319 D-Star, received network end of transmission, 1.0 seconds, 0% packet loss, BER: 0.0%

The logs show 21 voice frames corresponding with 10 blocks, which I believe is empty voice data combined with the slow data for the screen message. Then the data starts and you can see "switching to fast data mode." However, just before the switch to fast data, we receive a single voice frame with a BER of 14.6%, when the block sequence is starting again, and I believe this is voice frame 1 in block 1.

vbindiff output of PuTTY logs showing what I sent and what was echoed back to me:

dstar-sent.log                                                                  
0000 0000: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0010: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0020: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0030: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0040: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0050: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0060: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0070: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0080: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0090: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00A0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00B0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00C0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00D0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00E0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00F0: 0D 0A                                             ..                 
                                                                                
dstar-received.log                                                              
0000 0000: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0010: 41 41 41 41 40 43 61 41  45 41 C1 45 41 41 41 41  AAAA@CaA EA.EAAAA  
0000 0020: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0030: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0040: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0050: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0060: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0070: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0080: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 0090: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00A0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00B0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00C0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00D0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00E0: 41 41 41 41 41 41 41 41  41 41 41 41 41 41 41 41  AAAAAAAA AAAAAAAA  
0000 00F0: 0D 0A                                             ..                 

All looks well with this test except for the corruption at bytes 21-28. These bytes correspond exactly with "1st voice" in block 1 if you look at section 7.3 of the translated PDF.

In #470 (comment) above, I worried:

The only potential complication I can see is handling Block 1, where we need to hold off recalculating the FEC on voice frame 1 until we receive data frame 2 and check its mini header to determine whether voice frame 1 contains voice or data.

I think this is what we are seeing here.

I don't understand the overall control flow well, but from what I've read this particular requirement may not fit well with the current implementation. @g4klx, what do you think? Would it be straightforward to set aside voice frame 1 (always voice+sync) for one round and then handle it once the next voice+data frame is received and the mini header is available?

Regarding @phl0's log, I don't know how to explain the three 14.46% BER voice frames (sequence numbers 0-2). @phl0, is it possible that your RX started late? What kind of radio was it?

Also, I still plan to try with my ID-4100 but I expect I'll see the same issue there. For what it's worth, I think we're very close to the correct solution here.

@phl0
Copy link
Collaborator

phl0 commented Jan 31, 2019

@timclassic Are you still online/on air?

@phl0
Copy link
Collaborator

phl0 commented Jan 31, 2019

bildschirmfoto vom 2019-01-31 08-13-20

Guess it was a one time issue. Did some DStar Queries to " E".

@timclassic
Copy link
Contributor Author

(@phl0 I'm sorry we didn't connect--I only saw your post right as I was shutting down for the night. I really hope you didn't trade any sleep to be ignored by me!)

@phl0
Copy link
Collaborator

phl0 commented Jan 31, 2019

@timclassic Never mind. Sleep is overvalued :) Want to make a new sked? My equipment is still on the table :)

@phl0
Copy link
Collaborator

phl0 commented Feb 11, 2019

I did test again with @timclassic
Results:

  1. It works (switching to fast data mode)
  2. We had several errors in single transmissions. Obviously caused by missing error correction on application level

@hipmark
Copy link

hipmark commented Nov 13, 2020

Mentioned this issue in an article I have written on D-Star Picture Mode, which will appear in the December 2020 issue of RSGB RadCom Magazine. The article talks about transfer of images using DV Fast Data mode, and unfortunately this issue prevents users of low-cost MMDVM hotspots (Pi-Star & others) from transmitting images in Fast Data mode.

Would be nice to see D-Star DV Fast Data mode fully implemented and supported in MMDVMHost; anything I/we (@timclassic @phl0) can do to help progress of this issue towards a fix?

Thanks,
Mark, M5BOP

@timclassic
Copy link
Contributor Author

@hipmark, I'm planning to revisit this issue over the next week or two. It has come up again in my own projects since I'm trying to build a digital radio using a Kenwood TM-V7 and the MMDVM_PCM code + MMDVMHost modifications[1] described at the DUDE-Star Radio Project. As a part of this I would love to get DV Fast Data working.

Conceptually I think I know what needs to be done--I still believe that #470 (comment) captures the final piece of the puzzle (famous last words).

I hope to send @g4klx a PR with a solution in the next week or two. Maybe @phl0 will be up for another test?

[1] I think the MMDVMHost modifications are represented by #647. Maybe there are additional changes? I haven't diffed the two repos to check.

@g4klx
Copy link
Owner

g4klx commented Nov 14, 2020 via email

@hipmark
Copy link

hipmark commented Nov 15, 2020

If you do find a bug then I would be most interested.

Hi Jonathan, thanks for your rapid reply. Please see comments from @timclassic above - he believes that #470 (comment) captures the final piece of the puzzle needed to fully implement DV Fast Data mode (famous last words). Tim is going to take another look at the code based on his findings and send you a PR if he can get to the bottom of this.

I think it’s the first time that the MMDVM has been mentioned in the magazine. I keep meaning to do a write up of the project, but never get around to it.

I believe MMDVM is a fantastic piece of work that has really been under-sung - many people know about Pi-Star, but really don't understand the excellent engineering under the covers that has made low-cost multimode digital hotspots possible. From comments I've read, I'm sure the creators of Pi-Star agree entirely - and for this reason I think you really should write an article about your journey in developing MMDVM. I'd never written for a radio mag before, but with the excellent editing skills of Giles G1MFG at RSGB, I'm really pleased with how my humble efforts turned out!

73 & stay safe!
Mark, M5BOP

@timclassic
Copy link
Contributor Author

@hipmark and @g4klx, just a quick update to say that I’m testing a patch that so far has greatly improved the reliability of sending and receiving Fast Data via MMDVMHost. I have a bit more testing to do, but I plan to submit a PR for review this upcoming weekend.

Stay tuned!

@timclassic
Copy link
Contributor Author

@phl0 and @hipmark, let me know if either of you would like to do some fast data testing now that #667 has been merged. I'm happy to make a sked, and weekends are generally good for me.

@g4klx, thanks in general for all the work on MMDVM!

@phl0
Copy link
Collaborator

phl0 commented Dec 15, 2020

Tnx @timclassic. I'll get my equipment ready. Need to set D-Rats up on my new computer. Which reflector are you on usually?

@hipmark
Copy link

hipmark commented Dec 29, 2020 via email

@hipmark
Copy link

hipmark commented Dec 29, 2020

Tnx @timclassic. Which reflector are you on usually?

D-Star Picture Net is held weekly on REF072D, so that is where I usually hang out for picture TX/RX. Mark, M5BOP

@s-s-s
Copy link
Contributor

s-s-s commented Dec 30, 2020

"Conclusion

The ID-51 E3 release somehow switches to Fast Data Mode as soon as it transmits GPS Data. Once it does this it does not send the End of Transmission frame.

This behavior confuses both MMDVM gear and Icom gear. Both will then wait for a timeout and then decide the transmission is gone. Hence the 3s delay on the IC-E92D before end of transmission beep and the garbled R2D2 tail signal on the MMDVM.

The workaround is to switch GPS data off.

Downgrading the ID-51E Anniversary Edition to firmware release pre-November 5th solved the issue."

https://www.f4fxl.org/buggy-icom-id-51-and-id-5100-firmware/

@timclassic
Copy link
Contributor Author

@hipmark I'll be on the D-Star Picture Net tonight as well. I plan to send images from both my IC-9700 and my TH-D74 (by way of RS-MS1A).

@hipmark and @phl0, I'll look up your contact info on QRZ and we can work out further testing that way, so we don't have to pollute this Issue with scheduling details. I suspect we can close this issue once we do some a bit more testing with images and D-RATS.

@g4klx
Copy link
Owner

g4klx commented Jan 1, 2021 via email

@timclassic
Copy link
Contributor Author

@g4klx I've been using the Fast Data changes on both of my hotspots since submitting #667 and voice traffic has been working well.

I also created #675 today so I could watch logs in a more useful way while listening to REF030C for a few hours. All looks well.

Maybe some others (@hipmark and/or @phl0, perhaps?) would be willing to run the latest MMDVMHost for a while and report back with their results, so we could become more confident in the changes?

(Oh, and I did attend the D-Star Picture Net and it went well! The image I sent had good reports from others and the images I received were as good as or better than the quality reported by others on the net. @hipmark sent a nice picture of the moon.)

@hipmark
Copy link

hipmark commented Jan 8, 2021 via email

@timclassic
Copy link
Contributor Author

timclassic commented Jan 10, 2021

@hipmark, I have tested this on a ZUMspot by building both MMDVM firmware + MMDVMHost on the Pi itself. I've gone through my notes and stepped through the build process again, writing down everything I did. These instructions were validated using a ZUMspot RPi running Pi-Star v4.1.2.

Build and Flash MMDVM_HS Firmware

First, I recommend building a new MMDVM_HS firmware and flashing it to the ZUMspot board so you can take advantage of the fix in juribeparada/MMDVM_HS#135. Without this change, there are some data patterns that the MMDVM firware will interpret as an End Of Transmission sequence.

Here are the steps I followed on my ZUMspot, adapted from the instructions at https://www.hamradio.com/images_managed/misc/H0-017019_ZUMspot_Duplex_Quick_Start_Guide_1_2020.pdf in the section called "Building firmware on Pi-Star." Hopefully your hotspot also uses the MMDVM_HS firmware. If so, do the following, replacing config/ZUMspot_RPi.h with an appropriate config file if you use a different hotspot (look for the line with the comment # replace with appropriate config):

# Log in to your pi-star as the pi-star user, then make the disk writable
rpi-rw

# Stop services that would interfere with installation
sudo systemctl stop pistar-watchdog.timer
sudo systemctl stop pistar-watchdog.service
sudo systemctl stop mmdvmhost.timer
sudo systemctl stop mmdvmhost.service

# Install required packages
sudo apt-get install gcc-arm-none-eabi gdb-arm-none-eabi libstdc++-arm-none-eabi-newlib libnewlib-arm-none-eabi

# Build and install the latest stm32flash utility (not sure if this is required, but I did it)
cd ~
git clone https://git.code.sf.net/p/stm32flash/code stm32flash
cd stm32flash
make
sudo make install

# Build the MMDVM_HS firmware
cd ~
git clone https://github.com/juribeparada/MMDVM_HS.git
cd MMDVM_HS
git submodule init
git submodule update
cp configs/ZUMspot_RPi.h Config.h   # replace with appropriate config
make

# Flash the ZUMspot with new firmware
sudo /usr/local/bin/stm32flash -v -w bin/mmdvm_f1.bin -g 0x0 -R -i 20,-21,21:-20,21 /dev/ttyAMA0

Build and Install MMDVMHost

The following steps assume the disk is writable and services are still stopped after following the MMDVM_HS instructions above. Now, build and install MMDVMHost:

cd ~
git clone https://github.com/g4klx/MMDVMHost.git
cd MMDVMHost
make
sudo make install

# This build of MMDVMHost can't open the modem when started via systemd for some reason.  I haven't
# figured out the root cause yet, but forcing "dialout" into the service's supplementary groups resolves
# the problem for me:
sudo mkdir /etc/systemd/system/mmdvmhost.service.d
echo "[Service]" | sudo tee /etc/systemd/system/mmdvmhost.service.d/modem-access.conf >/dev/null
echo "SupplementaryGroups=dialout" | sudo tee -a /etc/systemd/system/mmdvmhost.service.d/modem-access.conf >/dev/null
sudo systemctl daemon-reload

# Reboot as an easy way to bring everything back online
sudo shutdown -r now

Testing

At this point I think you can use the hotspot as you normally would. Be careful running an update, however--it may overwrite the updated version of MMDVMHost you've just installed.

The debug logging for D-Star frames has also been enhanced as of #675, so you can see more details in /var/log/pi-star/MMDVM-*.log by setting FileLevel=1 in the [Log] section in /etc/mmdvmhost. Be careful, though; I've filled my /var partition more than once because I left this enabled for too long.

@hipmark
Copy link

hipmark commented Jan 15, 2021

Hi @timclassic

I've gone through my notes and stepped through the build process again, writing down everything I did. These instructions were validated using a ZUMspot RPi running Pi-Star v4.1.2.

Thanks so much for your detailed notes, much appreciated! I have used them to upgrade my Pi-Star hotspot (constructed from a Pi-Zero with an AliBaba-sourced MMDVM_HS radio hat) and can report that I have a working upgraded hotspot and both audio and picture transfer works well under the new build.

The only problem I have had is that the OLED display on my hotspot is no longer operational/updating, but I’m pretty sure that will be something to do with configuration parameters in the STM32 flash build - any tips on how to resolve much appreciated, but even without the display the modem side of things works a treat and is much improved for picture transmission in Fast Data mode, as well as retaining full functionality for audio RX and TX.

As a test, I used the upgraded hotspot to take part in the busy D-Star Picture Net this week (as well as my day-to-day REF001C operating) with my Icom-9700 as the client. I recorded all incoming audio from the net onto the 9700’s SD card, as well as listening and had clean audio - no unexpected issues. Also had positive reports re. my transmitted audio! Best of all, I had very good reports for my Pictures, sent in Fast Data mode - not something I have achieved before using this low-cost hotspot - in fact Jef ON8NT sent me via e-mail a perfect copy of the picture that I had transmitted!

So a clean bill-of-health as far as I am concerned @g4klx and a gold star to @timclassic for his contributions - thanks Tim. Will, of course, keep on testing! What is your callsign Tim?

Thanks, Mark M5BOP

@hipmark
Copy link

hipmark commented Jan 15, 2021

Here’s a copy of my Fast Data transmitted picture from this week’s Picture Net, as received and sent to me via e-mail by Jef ON8NT. In case you’re wondering, it’s a picture of our church video streaming set-up!

Pic20210114_M1BOP_01

Cheers, Mark M5BOP
(See previous post above for hotspot test details)

@hipmark
Copy link

hipmark commented Jan 17, 2021

Hi @timclassic, @g4klx

The only problem I have had is that the OLED display on my hotspot is no longer operational/updating, but I’m pretty sure that will be something to do with configuration parameters in the STM32 flash build - any tips on how to resolve much appreciated

Managed to solve the OLED problem by doing that old important thing when you have such a problem - reading the manual! or in this case OLED.md on Jonathan's Github! Re-ran the MMDVMHost make using: make -f Makefile.Pi.OLED
having first checked that I had the required OLED library built and installed at /usr/local/lib/libArduiPi_OLED.so.1.0

Now have a much-improved MMDVM hotspot with a working OLED status display - I am a very happy man!

Thanks!
Mark, M5BOP

@timclassic
Copy link
Contributor Author

@hipmark Sorry for missing your questions earlier--my callsign is KG4BXH. Also, I'm glad you figured out the OLED configuration. I don't have an OLED screen on my ZUMspot so I didn't have any slam-dunk solutions for you.

I wasn't on the picture net last Wednesday, but I'm pleased that it went well! I plan to attend the next one, and hopefully we'll run into each other.

I've been doing a lot of listening and some talking using my patched hotspot, and I still haven't run into any problems with D-Star voice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants