further Bluetooth Low Energy support

[UweRenschler]

We are currently releasing our Solario+ Variometer, which is a solar powered Bluetooth Low Energy based variometer, GPS, IAS...
Are there plans to support f.e. a VSP (virtual serial port) on Bluetooth Low Energy ? Currently our plan is to write an Android app, which translates from VSP (not SPP on Bluetooth 4.0) to a TCP port. For USB connection this is a good working solution, but generally we want to have BLE (no wires). Makes it more sense, if we contribute here into a driver or is it a better idea to do our own VSP to TCP/IP port android translator ? It would be great to get some feedback ( uwe@renschler.de ) from the developers.

[lordfolken]

Please contribute to a driver. That would help.

[UweRenschler]

Hallo Philipp, Vielen Dank für die Info. Gibt es zu XCSoar Infos wie man einen Driver in das Ganze einzubinden hat ODER Ist der Code Dokumentation genug um Driver zu schreiben ? Ich würde mich über ein paar Worte sehr freuen. Grüßle Uwe P.S.: Ein aktueller Bluetooth Gatt Server von Android läuft timing mässig sauber mit einem Service mit einer characteristic. In dem Moment wo man 4 characteristics mit unterschiedlichen flexiblen Latenzzeiten bedient rennt Android in diverse Betriebssystem Timing Probleme. Das sieht man auch an den Android Apps (FlyMe) die kein Notify können. Was vernünftiges baut auf die Nordic Lib auf, die auch sauber 4 Characteristics mit Notify kann, aber hier fragt es sich, wie man die Nodric Lib in das Gesamtkonzept von XCSoar einbindet. P.P.S.: Ich dachte das ganze (XCSoar) ist Tod, da sich auf github kein Mensch meldet. (bei LK8000 ist das anders) Von: Philipp Wollschlegel [mailto:notifications@github.com] Gesendet: Montag, 4. Mai 2020 23:40 An: XCSoar/XCSoar Cc: UweRenschler; Author Betreff: Re: [XCSoar/XCSoar] further Bluetooth Low Energy support (#357) Please contribute to a driver. That would help. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#357 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMGMBTIKA2OFYJAHQTCWH6TRP4YZLANCNFSM4LATHTNQ> . <https://github.com/notifications/beacon/AMGMBTJU34654RDAFLHVRPDRP4YZLA5CNFSM4LATHTN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEUWT5EY.gif>

[lordfolken]

Our Development Manual: (these are nightly builds, you may have to browse in the nightly_build paths)
https://download.xcsoar.org/nightly_builds/7.0_preview15~git%239d776759/manual/XCSoar-developer-manual.pdf

For a development environment you can use either the docker container or the vagrant box if you are not running debian10 as your main environment:
https://github.com/XCSoar/XCSoar/tree/master/ide

The procedure is also explained in the latest development manual.

Beware that your solution should work on a lot of platforms which are not android. (WINPC, Kobo, Plain Linux... ) So code platform neutral as much as possible

For drivers I'd recommend to look at the current drivers,:
https://github.com/XCSoar/XCSoar/tree/master/src/Device/Driver

Writing code that runs is quickly done. Writing maintainable code is 3 times as hard. That is why the quality standards are high and the code must fit into the rest of the platform. Also you should be willing to help maintain the code long term.

[UweRenschler]

Hello Phillipp, Thanks a lot. We mainly want to do an Android version and not more. So I think ower own translator from BLE to a supported socket using the Nordic lib (for Android) is best. Cheers Uwe Von: Philipp Wollschlegel [mailto:notifications@github.com] Gesendet: Dienstag, 5. Mai 2020 22:18 An: XCSoar/XCSoar Cc: UweRenschler; Author Betreff: Re: [XCSoar/XCSoar] further Bluetooth Low Energy support (#357) Our Development Manual: (these are nightly builds, you may have to browse in the nightly_build paths) https://download.xcsoar.org/nightly_builds/7.0_preview15~git%239d776759/manual/XCSoar-developer-manual.pdf For a development environment you can use either the docker container or the vagrant box if you are not running debian10 as your main environment: https://github.com/XCSoar/XCSoar/tree/master/ide The procedure is also explained in the latest development manual. Beware that your solution should work on a lot of platforms which are not android. (WINPC, Kobo, Plain Linux... ) So code platform neutral as much as possible For drivers I'd recommend to look at the current drivers,: https://github.com/XCSoar/XCSoar/tree/master/src/Device/Driver Writing code that runs is quickly done. Writing maintainable code is 3 times as hard. That is why the quality standards are high and the code must fit into the rest of the platform. Also you should be willing to help maintain the code long term. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#357 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMGMBTI3WWNMQXNLLCAAGADRQBX6PANCNFSM4LATHTNQ> . <https://github.com/notifications/beacon/AMGMBTMHQMVX5C2K4T6E6KLRQBX6PA5CNFSM4LATHTN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEU245TI.gif>

[kedder]

Isn't BLE already supported? I used to connect to Chineese JDY-08 BLE couple of years ago on android and have seen the NMEA stream in XCSoar device. Don't remember doing anything special to support that.

[UweRenschler]

No. Only BLE with SPP (serial port protocol). We use the same protocol like the SENSBOX from Flytec/Bräuniger and this is not supported. Have you ever connected a SensBox to XCSoar ? (No NMEA stream, but a BLE Gatt Server with 4 Services and several characteristics.) Von: Andrey Lebedev [mailto:notifications@github.com] Gesendet: Mittwoch, 6. Mai 2020 10:55 An: XCSoar/XCSoar Cc: UweRenschler; Author Betreff: Re: [XCSoar/XCSoar] further Bluetooth Low Energy support (#357) Isn't BLE already supported? I used to connect to Chineese JDY-08 BLE couple of years ago on android and have seen the NMEA stream in XCSoar device. Don't remember doing anything special to support that. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#357 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMGMBTMQEXFEEVPGJFQMIC3RQEQXNANCNFSM4LATHTNQ> . <https://github.com/notifications/beacon/AMGMBTL3RTNBWX7F5N4HQGDRQEQXNA5CNFSM4LATHTN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEU4XYGI.gif>

[kedder]

We use the same protocol like the SENSBOX from Flytec/Bräuniger and this is not supported.

I see, makes sense.

Have you ever connected a SensBox to XCSoar ?

No, I don't know much about SensBox and have never seen it.

[MaxKellermann]

This feature request does make sense. XCSoar's BLE support is currently limited, and adding support for other BLE products is useful, so I'm going to reopen this.
But the problem is, as always: who wants to spend time on writing the code? Ideally, the vendor writes code to support his own product and submits it to XCSoar for code review and merging. If the code is well done, it will be merged. But the path to good code is long and difficult (for everybody of course, not just for vendors).
There is no documentation for writing a completely new way of attaching devices. Writing such extensive documentation would not be worthwile, because the time would better be spent on writing code. So anybody who wants to hack XCSoar needs to read existing code, and/or talk to people who know the code already.
Code using Android APIs needs to be written in Java, but XCSoar is C++, so there needs to be a bridge between Java and C++. That's not easy (nothing is).
Here's the Java part for talking to currently supported BLE devices: https://github.com/XCSoar/XCSoar/blob/master/android/src/BluetoothGattClientPort.java (written by @felixhaedicke)
This code implements the interface AndroidPort, and this class already has C++ bindings because it is shared with IOIO and "classic" Bluetooth.
If you implement something which doesn't work like AndroidPort, it needs a new (Java) interface with new Java/C++ bindings.

[UweRenschler]

Thanks a lot for the link. I will have a closer look at it and check if I see an easy way to use the Nordic Lib to interface to an AndroidPort. I think I first will finalise our own App with BLE and the lib and see all timing related stuff. With this knowledge and the links below I will check the effort. I also can program C and C++ (mainly on Cortex-M Cpus for realtime software). Hints how Bluetooth is adopted in C++ on Android are welcome. Von: Max Kellermann [mailto:notifications@github.com] Gesendet: Mittwoch, 6. Mai 2020 12:40 An: XCSoar/XCSoar Cc: UweRenschler; Author Betreff: Re: [XCSoar/XCSoar] further Bluetooth Low Energy support (#357) This feature request does make sense. XCSoar's BLE support is currently limited, and adding support for other BLE products is useful, so I'm going to reopen this. But the problem is, as always: who wants to spend time on writing the code? Ideally, the vendor writes code to support his own product and submits it to XCSoar for code review and merging. If the code is well done, it will be merged. But the path to good code is long and difficult (for everybody of course, not just for vendors). There is no documentation for writing a completely new way of attaching devices. Writing such extensive documentation would not be worthwile, because the time would better be spent on writing code. So anybody who wants to hack XCSoar needs to read existing code, and/or talk to people who know the code already. Code using Android APIs needs to be written in Java, but XCSoar is C++, so there needs to be a bridge between Java and C++. That's not easy (nothing is). Here's the Java part for talking to currently supported BLE devices: https://github.com/XCSoar/XCSoar/blob/master/android/src/BluetoothGattClientPort.java (written by @felixhaedicke <https://github.com/felixhaedicke> ) This code implements the interface AndroidPort, and this class already has C++ bindings because it is shared with IOIO and "classic" Bluetooth. If you implement something which doesn't work like AndroidPort, it needs a new (Java) interface with new Java/C++ bindings. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#357 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMGMBTLUGH4555EH7JVY5HLRQE46XANCNFSM4LATHTNQ> . <https://github.com/notifications/beacon/AMGMBTI47HXUP34P2MR6WF3RQE46XA5CNFSM4LATHTN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEU5DSKY.gif>

[MaxKellermann]

Support for Bluetooth LE sensors has been added last year in version 7.11; this release contains a framework which makes writing new BLE drivers easy. I suppose this feature request is complete.
To support more sensors, a small device-specific piece of code needs to be written, but that's out of scope here. If anybody needs help with writing such a driver, contact me.

[iltis42]

I have added BLE advertising meanwhile in XCVario, follwing this setup here: https://openlabpro.com/guide/ble-uart-on-esp32-controller/ .
In principle in a terminal this works, i can see the data coming in as expected (1), hence XCS won't accept the device, configures a "BLE Sensor" with no option to configure a Driver protocol and stops talking to the device after a couple of seconds telling the device is unsupported.
Apparently to me this looks like my device cannot be configured properly in XCS, would the BLE experts give me a hint what we needed to do in order to get the device properly in when advertising BLE ?

2)

[lordfolken]

XCSoar recognizes the sensor from the MAC address. So you need to add XCVario's bluetooth mac in order for XCSoar to recognize it.

[lordfolken]

See android/src/BluetoothUuids.java

[lordfolken]

(this is an educated guess. i'm not familar with the code there.)

[lordfolken]

There is also the getFeatures function in BluetoothHelper.java that should help set the characteristics on the device.

[MaxKellermann]

This is not a BLE sensor, this is NMEA tunneled over some obscure BLE protocol. This is a different thing from the one being discussed here previously.

[iltis42]

Its a serial port service providing NMEA data, and today discovered as a BLE sensor port.
Do we agree that in "BluetoothHelper.java", this code that constructs the Bluetooth ports should be the key to get the BLE port similar to the usual classic Bluetooth (RFCOM) as a "serial device" (what seems to be the naming in BLE) in?

My idea is somehow this 2 steps below, would be kind to get some comment about this (these are the one from EPS32 library examples):

Step 1) Bring in the public UART service ID's for BLE UART in here ( https://docs.pybricks.com/projects/pybricksdev/en/latest/api/ble/nus.html )
diff --git a/android/src/BluetoothUuids.java b/android/src/BluetoothUuids.java
@@ -68,4 +68,8 @@ public interface BluetoothUuids {
+

• UUID UART_SERVICE_UUID = UUID.fromString("6E400001-B5A3-F393-E0A9-E50E24DCCA9E"); // UART service UUID
• UUID UART_CHARACTERISTIC_UUID_RX = UUID.fromString("6E400002-B5A3-F393-E0A9-E50E24DCCA9E");
• UUID UART_CHARACTERISTIC_UUID_TX = UUID.fromString("6E400003-B5A3-F393-E0A9-E50E24DCCA9E");
  }
  Step 2) Add new functionality in the Bluetooth classes to link the BluetoothClientPort class with this new UUIDs (tbd, any helpfuly advice welcome).

**

• An #AndroidPort implementation that initiates a Bluetooth RFCOMM
• connection.
  */
  class BluetoothClientPort extends ProxyAndroidPort implements Runnable {
  private static final String TAG = "XCSoar";

[ygaillard]

@iltis42 I wanted to warn you about a new issue you will encounter implementing your routine. Currently android limits BLE messages by 20 bytes. I didn't find out how to solve it. As you want to stream a full NMEA sentence which is pretty long, this may cause problems. Some workarounds I found so far is to chunk the message in little piece. But there you have to reprogram your source side.

My longest characteristic is 23 bytes long. I don't know if a will chunk it or create another characteristic.

[iltis42]

@iltis42 I wanted to warn you about a new issue you will encounter implementing your routine. Currently android limits BLE messages by 20 bytes. I didn't find out how to solve it. As you want to stream a full NMEA sentence which is pretty long, this may cause problems. Some workarounds I found so far is to chunk the message in little piece. But there you have to reprogram your source side.

If longer than 20 bytes, i add the next chunk with a new BLE notification. SeeYou Mobile can already receive this (see below), as also the terminal does (have posted this earlier), so this seems not to be a restriction to me as the NMEA engines in XCS detect well opening and closing symbol of those sentences.

[ygaillard]

Ok, great! I found out how to enlarge the size. You can appenrently doing it by the command gatt.configureMTU(int i). Currently I have the bug that onChangeCharacteristic does not fire anymore.

I'm really interested how you implemented your code. It seems that you doing it completely differently than me. Did you open a pull request yet?

[iltis42]

The code on the sender side is on an embedded system with ESP32. You will find this part here:
https://github.com/iltis42/XCVario/blob/master/main/BLESender.cpp
For XCSoar i did not have a great progress until now as i am not expert in java and esp. this part how its implemented in XCSoar needed more understanding from my side. The above is the SeeYou Mobile App from Naviter out of the box from playstore. I can see there that in principle my sending part should work, hence even so, SeeYou Mobile can't benefit from as the public version in PlayStore, unlike the App on an Oudie, does not support to configure a device e.g. Cambridge 302, so the protocol does not get used/decoded in there. Hence we are talking to them :)

[ygaillard]

If perhaps you want to increase the MTU I found a solution

   boolean mtuConfirmed=false;
    int mtuRequestCounter=0;
    while (!mtuConfirmed) {
      mtuConfirmed=gatt.requestMtu(50);
      mtuRequestCounter++;
      try {
        Thread.sleep(1000);
      } catch (InterruptedException e) {
        e.printStackTrace();
      }
    }
    Log.d(TAG, "MTU change reply received after " + mtuRequestCounter + " attempts");

Actually configureMtu is correct however it have to be request until the other device acknowledge it. Any else there are some mismatching.

[iltis42]

@lordfolken, may the idea above go into the right direction?

[ygaillard]

I'm developing a BLE Device. We can have a chat.

[lordfolken]

yes please do so. if possible here :)

[ryanwoodie]

I have a ESP32 board that I am using as a "dongle" to send data (vario, gps, etc) to XCsoar (which is working fine), and also, hopefully, send outbound data (ie. enviroment, MC settings, ballast, etc) from XCsoar to the ESP32 device. The problem I run in to is that the BLE only seems to get the first character "$" of each sentence. Very rarely it will get the whole sentence, which likely the odd time the sentence happens to be less than 23 bytes.

LK8000 sends the data OK, but it has also increased the mtu size to 85: (edit: I just tried an older version of LK8000 and still worked without the MTU size increase) LK8000/LK8000@08d8dde

I've tried a number of different ways of handling the rxValue (as as full string, etc), but always get the same result. Here is the code for the callback on the ESP32. I am processing it wrong, or is it just a inherent limitation of XCsoar? I am a hack, not an experienced developer, any help is appreciated!

The XCsoar logs have a "PutMcReady() filed: Port write failed" message whenever an MC button is pushed.

` class MyCharacteristicCallback : public BLECharacteristicCallbacks {
String receivedData = "";

void onWrite(BLECharacteristic *pCharacteristic) {
std::string rxValue = pCharacteristic->getValue();

if (rxValue.length() > 0) {
  for (int i = 0; i < rxValue.length(); i++) {
    char receivedChar = rxValue[i];
    if (receivedChar == '\n') {
      receivedData.trim();
      ble_external = receivedData;
      receivedData = "";
    } else {
      receivedData += receivedChar;
    }
  }
}

}
};

`

[MaxKellermann]

Tunneling NMEA over BLE is a bad kludge. If you design new hardware, don't do that.

[ryanwoodie]

Thanks for the response Max. Forgive my ignorance here... Are you saying in general to not use BLE, or is there a different way of using BLE that doesn't involve "tunnelling"? Any examples off-hand?

[MaxKellermann]

The standard way of BLE is to transmit sensor data as binary, like the Flytec does, for example the Flytec speed/track values here:

XCSoar/android/src/BluetoothSensor.java

Lines 180 to 185 in 6bdc477

	} else if (BluetoothUuids.FLYTEC_SENSBOX_MOVEMENT_SENSOR_CHARACTERISTIC.equals(c.getUuid())) {
	flytecGroundSpeed = c.getIntValue(c.FORMAT_SINT16, 6) / 10.;
	flytecTrack = c.getIntValue(c.FORMAT_SINT16, 8) / 10.;
	listener.onVarioSensor(c.getIntValue(c.FORMAT_SINT16, 4) / 100.f);
	listener.onAccelerationSensor1(c.getIntValue(c.FORMAT_UINT16, 16) / 10.);

[ryanwoodie]

Thanks Max, that makes sense. My device is just a dongle to take serial nmea input(s) (vario, SoftRF, gps, etc) and route it to each other if needed and send it to XCsoar/phone over Wi-Fi, serial, BT and BLE. A couple of the esp32c3 boards I bought, unknowingly, only have BLE not classic. It sounds like, to do the BLE properly, I'd need to break the nmea data into binary before sending to XCsoar. Only making a handful, so likely better just to get a different board with BT but I've been enjoying the challenge. Out of curiosity, does XCsoar also send outbound data this same binary way as an option, or does it only "tunnel"? That is part that isn't working anyway, but does in LK8000.

…

On Mon., Apr. 17, 2023, 12:23 p.m. Max Kellermann, ***@***.***> wrote: The standard way of BLE is to transmit sensor data as binary, like the Flytec does, for example the Flytec speed/track values here: https://github.com/XCSoar/XCSoar/blob/6bdc477ee40e88657b918b637539c0bc88b787ab/android/src/BluetoothSensor.java#L180-L185 — Reply to this email directly, view it on GitHub <#1120 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVQCWB74JY3P6OGKASFFYTXBWDAXANCNFSM6AAAAAAVA3RL3E> . You are receiving this because you commented.Message ID: ***@***.***>

[iltis42]

SeeYou Navigator is a different application, afaik XCSoar does not support BLE yet.

[hperrot]

I understood @MaxKellermann's comment (see #1120 (reply in thread) ) so, that he meant tunneling NMEA sentences as uart via BLE should be possible with XCSoar for a long time already.
Or was this meant differently?

[ryanwoodie]

What are you trying to connect to XCsoar via BLE? The issue, from my
understanding, is not so much that XCsoar can't process the BLE data (it
can), but that only a limited selection of devices are configured to be
accepted by XCsoar. Thus the device needs a UUID that is in the XCsoar code
ie:
https://github.com/XCSoar/XCSoar/blob/master/android/src/BluetoothUuids.java

I've also had some luck as per an earlier post tricking XCsoar to accept a
device by manually editing the PRF file.

[hperrot]

I am using an esp32 to bridge a Flarm UART to BLE.
For now I used the Nordic UART profile (see the UUIDs here: https://github.com/avinabmalla/ESP32_BleSerial/blob/9250e6724ab7edb6838460183820ebc47d0cf3a0/src/BleSerial.h#L75-L77 ).

[ryanwoodie]

You can see how I implemented BLE here on ESP32 and the UUID used: https://github.com/ryanwoodie/ESP32-Glider-Dongle

[iltis42]

In XCVario i have added BLE support some time ago beside BT classic (RFCOM) as an option with the EPS32 module. And with TopHat actually this already worked sending with the standard MTU size of 20 bytes the longer NMEA data in slices of up to 20 bytes using the notify() method.
https://github.com/iltis42/XCVario/blob/master/main/BLESender.cpp
In my opinion classic BT support in the chips on the market will shrink, so i have done some tests regarding performance e.g. distance that could be covered from the ESP32 module by classic BT and BLE in comparison, and i found for this bandwith we need for our purpose (max 5 sentences of ~80 byte per second) it worked same good that classic BT. I could even go a bit further with my mobile phone running the TopHat Monitor before the data stopped what surprised me as BLE would use less power than classic BT.

[ryanwoodie]

@iltis42 did you have any success sending enviroment variables (MC, ballast, settings) out from XCsoar over BLE? Or only with TopHat?

LK8000 and any BLE tools can send strings just fine. It's only XCsoar that seems to have the issue which is interesting as I think the code is similar to LK8000

[MaxKellermann]

XCSoar has an issue? I can't seem to find one. What issue number is that?

[ryanwoodie]

Hi Rowan, Yes. What type of device UUID are you using? You need to use the same as HM10 for XCsoar to accept it as a port. Or, you can trick it by editing the prf file to change device type to "hm10" Even with the HM10 UUID it would sometimes get stuck as BLE sensor. Usually forgetting the bt device in android settings would help fix this.

…

On Mon., May 15, 2023, 2:13 p.m. r0w4n, ***@***.***> wrote: @ryanwoodie <https://github.com/ryanwoodie> did you get nmea over uart to work? I'm doing a similar thing to you and sending legacy nmea over bluetooth ble. XCS just sees my ble uart as a ble sensor and only provides me with an engine list in the configuration. — Reply to this email directly, view it on GitHub <#1120 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVQCWDR6MCVRDFU27KEKNTXGKE5HANCNFSM6AAAAAAVA3RL3E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[r0w4n]

I'll give that a play