Missing Datatype TEXT

[hakha4]

Hi!
I have tried a number of 'mesh' like system but mostly fails due to short distance,instability ets. Stumbled over FDRS and it looks awesome! I have set up a basic scenario with one gateway and two ESPnow node's sofar and worked more or less out of the box. I'm working on a node using MCP23017 for easy managing up to 16 relays with one cmd. It would be nice if you could send a text string. In my case I want to send a string like "1110011001100101" showing status of all relays for further processing (Home Assistant). It could also be nice with a text string that could send alerts etc. My working but not finished code looks like this ;

/*
MCP23017 uses inverted logic. Output from 0 to 15
To set relay no 12 to ON for example send:
[{"id":31,"type":0,"data":120}] //set to on
[{"id":31,"type":0,"data":121}]//set to off

*/
#define FDRS_DEBUG
#define READING_ID 31
#define GTWY_MAC 0x01
#define USE_ESPNOW
#define COIL_PIN LED_BUILTIN

#include <fdrs_node.h>
#include <Adafruit_MCP23X17.h>

Adafruit_MCP23X17 mcp;

// relay var's
int tmpdata = 0;
int relayno = 0;
int relay_status = 0;

//########## Non blocking delay #
unsigned long previousMillis = 0;
const long interval = 2000;
int previousValue = 0;
int currentValue = 0;
//##############################
void fdrs_recv_cb(DataReading theData) {
tmpdata = (int)theData.d;

}
void setup() {

beginFDRS();
if (addFDRS(1000, fdrs_recv_cb))
subscribeFDRS(READING_ID);

mcp.begin_I2C();
//set all pins to output
for (int i = 0; i <= 255; i++) {
mcp.pinMode(i, OUTPUT);

}

}
void loop() {
unsigned long currentMillis = millis();
currentValue = tmpdata;
//split cmd in relay no and status
relayno = tmpdata / 10;
relay_status = tmpdata % 10;

loopFDRS();
if (currentMillis - previousMillis >= interval) {
// save the last time you did something
previousMillis = currentMillis;
//if (currentValue != previousValue) {
//write new value
mcp.digitalWrite(relayno, relay_status);
// read relaystatus
int port_status_A= mcp.readGPIO(0);
int port_status_B= mcp.readGPIO(1);
// ToDo: convert portstatus to '1010010111001011'

//send data
loadFDRS(relayno,STATUS_T);
loadFDRS(relay_status,STATUS_T);
// loadFDRS(port_status_A,STATUS_T);
// loadFDRS(port_status_B,STATUS_T);

// DBG(sendFDRS());
if(sendFDRS()){
DBG("Relaynumber ");
DBG(relayno);
DBG("Relay status ");
DBG(relay_status);

} else {
  DBG("ERROR");
}

//}
previousValue = currentValue;
// sleepFDRS(10); //Sleep time in seconds
}
}

Is it possible to add TEXT datatype?
Best Regards Håkan

[timmbogner]

Thanks a lot, Håkan! It's great to see you're using controllers.

What you'd like is possible, as long as you convert your 16 bit binary number into an 16 bit unsigned integer. This is actually getting in to an area of FDRS that needs improvement:
In FRDS's DataReading structure, the data portion is defined as a 32-bit floating-point number (type float), not a 32-bit unsigned integer (type uint32_t). I think it would technically be considered better to transport all of FDRS's data as binary representations, and convert that to floats as needed. However, when I was starting the project it was a lot easier for me to use floats, since that was how all of my sensor data was represented. Changing it now will be a moderate undertaking, but it's definitely a goal and is on the big "to-do" list.

So, what this means in this case is that a binary number like 1110011001100101 will be converted to 16-bit integer uint16_t 58982, which will be converted to floating-point decimal float 58982.0, which will be sent via FDRS. This is fine, but once you get to 16777217.0, whole numbers start being rounded up and down due to the way floats work.

So all of that is to say that yes, you can represent binary with a float just fine, but be careful if going above 16 bits. I think adding a datatype for 16-bit binary data would be okay, but fixing the system to allow up to the full 32 bits would be better. I'm going to leave #187 open as a reminder, and I'll use it as a planning point for how to implement the fix. My current favorite idea is to split the 256 datatypes into two, and make the first 128 floats, and the second half integers. So, for example, type 129 would indicate a binary temperature (where type 1 is float temperature).

[hakha4]

Hi!
Thank's for answering. I'm not able to test right now due to work but I get back and report when tested. I'm far from a 'proffesional' programmer, just tinkering as a hobbyist so when looking at your code I'm not always sure how it works. So understand me right when proposing this dirty way : Could a second sendFDRS() ('sendFDRS2) that handles integers directly or will that mess up the whole concept?

Best Regards Håkan

[timmbogner]

Don't worry, I'm just a hobbyist too! I had to learn every concept I've used here from scratch, and it took a lot of time and thought.

Part of the reason I've stuck with it for so long is because I enjoy the puzzles that arise in the effort to build such a minimalist protocol. In this case the conundrum is that we could easily use a different function to format the data as an integer instead of a float, but how would the receiving device know whether it is meant to be a float or a uint32_t? There needs to be some kind of flag within the packet that indicates which type to expect.

One reason I like the idea of splitting the byte used for the type value into two halves is that it would make the highest bit of that byte into a flag indicating integer or float.

[hakha4]

Works fine using a 16 bit unsigned integer!. Need to tidy up code when finishing though. It's perfectly ok for now but in future it would be nice to be able to send both float's and integers as you mentioned. And the possibility to send plain text would also be on my wishlist.

`/*
MCP23017 uses inverted logic. Output from 0 to 15
To set relay ON or OFF convert short integer to 'binary' and set each output according
To set relay 1,3 and 4 to ON (=0) and the rest to OFF (=1)
convert 'binary representation 1111111111110100 = 65524
SEND:
[{"id":31,"type":0,"data":65524}]

*/
#define FDRS_DEBUG
#define READING_ID 31
#define GTWY_MAC 0x01
#define USE_ESPNOW
#define status_PIN 2

#include <fdrs_node.h>
#include <Adafruit_MCP23X17.h>

Adafruit_MCP23X17 mcp;

// for debug only
String str_binary = "";
/*### incoming data variable ###
make sure all relays are set to OFF if node restarts for safety
1111111111111111 = 65535
*/
int tmpdata = 65535;
//########## Non blocking delay var's #################
unsigned long previousMillis = 0;
const long interval = 2000;

//only send if value have changed not to stress system, causes TIMEOUT !!!!
//int previousValue = 0;
//int currentValue = 0;

//##############################
void fdrs_recv_cb(DataReading theData) {
digitalWrite(status_PIN,LOW);//for visual debug in field
tmpdata = (unsigned short)theData.d;
str_binary = "";
String tmpbit;
//#### write new values to MCP23017 #####
// read bitvalue at each position and write value to MCP23017
for(int i = 15; i >= 0; i--) {
// check bits, LSB first
int bitValue = bitRead(tmpdata, i); // read the bit at position i
tmpbit = String(bitValue);//save result for debug
mcp.digitalWrite(i, bitValue);//write value to MCP23017
str_binary = str_binary + tmpbit;//add in debugstring
}

}
void setup() {
pinMode(status_PIN, OUTPUT);//visualizing node ok

beginFDRS();
if (addFDRS(1000, fdrs_recv_cb))
subscribeFDRS(READING_ID);
Wire.begin(); // I/O expander

mcp.begin_I2C();
//set all pins to output
for (int i = 0; i <= 15; i++) {
mcp.pinMode(i, OUTPUT);

}

}
void loop() {
unsigned long currentMillis = millis();
//currentValue = tmpdata;

loopFDRS();
//avoid stressing system, only send when new value arrives
//if (currentValue != previousValue) {

if (currentMillis - previousMillis >= interval) {
// save the last time when code was running
previousMillis = currentMillis;

//send data
loadFDRS(tmpdata,STATUS_T);

// DBG(sendFDRS());
if(sendFDRS()){
//LED blinking: alive,sending OK
digitalWrite(status_PIN,HIGH);
Serial.println(str_binary);
DBG("TMPDATA ");
DBG(tmpdata);

} else {
  //LED NOT blinking: ERROR of some kind
  digitalWrite(status_PIN,LOW);
  DBG("ERROR SENDING DATA");
}

}//end if currentmillis...
//}//end if current value
//previousValue = currentValue;
//sleepFDRS(10); //Sleep time in seconds
}//end void loop

I have one small issue,not a critical one but when I'm in code only want the node to send only if a value has changed (to minimize unessasary traffic) it times out and will not recover unless rebooted. Is it mandatory to use the sleepFDRS function? I have commented out this since this node will never sleep

BTW the <> (code insert) doesn't seem to work or I make som stupid mistake that's why it looks so messy.
Best Regards Håkan

[timmbogner]

To add a block of code put three accent symbols above and below it: " ``` "

void setup() {
pinMode(status_PIN, OUTPUT);//visualizing node ok

beginFDRS();
if (addFDRS(1000, fdrs_recv_cb))
subscribeFDRS(READING_ID);
Wire.begin(); // I/O expander

mcp.begin_I2C();
//set all pins to output
for (int i = 0; i <= 15; i++) {
mcp.pinMode(i, OUTPUT);

}

}

To make it look fancy, have the first one say "``` cpp"

void setup() {
pinMode(status_PIN, OUTPUT);//visualizing node ok

beginFDRS();
if (addFDRS(1000, fdrs_recv_cb))
subscribeFDRS(READING_ID);
Wire.begin(); // I/O expander

mcp.begin_I2C();
//set all pins to output
for (int i = 0; i <= 15; i++) {
mcp.pinMode(i, OUTPUT);

}

}

You might also want to try the "Auto Format" feature in Arduino IDE to help clean up your code.

You are not required to use sleepFDRS().

[shaffenmeister]

There are many implementations of data serialization formats.
CBOR may be an alternative:

• https://en.wikipedia.org/wiki/CBOR
• https://en.wikipedia.org/wiki/Comparison_of_data-serialization_formats

[timmbogner]

Thanks a ton, @shaffenmeister! As mentioned, I'm learning this as I go and often don't know where to start researching various concepts. I've been reading about CBOR and I'm getting excited about it.

From my understanding, a 32-bit float using CBOR would consist of a one byte header (containing the type 7 and short count 26), then a four byte extended count for the payload. This puts us at 5 bytes, which is larger than the existing structure. However...

The entire DataReading structure has always contained 7 bytes, and that's always bothered me. In the back of my mind, I've been waiting for a good reason to add a byte and make it an even 8. With 5 bytes available to the payload there is more than enough space to express an integer, float, or even small bits of text/binary data using CBOR. I think this will be a fantastic improvement for the system, and will put to rest some of its longest-running shortcomings.

Thanks again! I'm glad I didn't end up reinventing that wheel. If you have any advice more advice on this, I'm all ears. I've sifted through the examples in this branch of TinyCBOR by soburi and I think it all seems straightforward. I'm hoping to do it without breaking any code compatibility, although I don't think I can keep compatibility with old-version devices.

[shaffenmeister]

You are welcome! Payload wise there are some tradeoffs for sure. For my use cases (ESP NOW, less than ten values per message) these would not weigh too much. On the other hand, I think that tinyCBOR would be easy to integrate. CBOR provides custom tagging (which you do today as well)... You would probably break backwards compatibility protocol wise, but this would need a deeper look. The load/send API could stay untouched.