Show latest uplink message and up- & downlink fcount

[kschiffer]

MOVED ISSUE FROM https://github.com/TheThingsIndustries/lorawan-stack/issues/1971 original from @laurensslats

Summary

I love this feature of the V2 console, currently missing in V3.

...

Why do we need this?

Easy debugging as it gives insights on whether the device is operational and data is picked up by the network.

...

What is already there? What do you see now?

...

What is missing? What do you want to see?

What about something like this? Maybe with some magic UI sauce from Kevin

...

Environment

Chrome

...

How do you propose to implement this?

Add some fields in the console
...

Can you do this yourself and submit a Pull Request?

I need pure brainpower from @kschiffer
...

[johanstokking]

I'm not sure if this is the same, but quickly checking the NS's session for existence, activation date/time and potentially frame counters would be very helpful.

@rvolosatovs are we keeping a "last seen" timestamp in NS?

[rvolosatovs]

@rvolosatovs are we keeping a "last seen" timestamp in NS?

No, but it can be derived from RecentUplinks (RecentUplinks[len(RecentUplinks)-1].ReceivedAt)

[kschiffer]

@rvolosatovs are we keeping a "last seen" timestamp in NS?

No, but it can be derived from RecentUplinks (RecentUplinks[len(RecentUplinks)-1].ReceivedAt)

That's what I did, but I figured that for the complete functionality, we'd need to hook into the event stream to keep the numbers updated in real time.

[kschiffer]

That's what I did, but I figured that for the complete functionality, we'd need to hook into the event stream to keep the numbers updated in real time.

Before I go ahead and implement it this way, @htdvisser do you think that approach is thorough enough? The uplink downlink events do not contain the current frame count, so I'd have to increment them based on the initial value that I got from the session prop inside the end device.

[johanstokking]

That’s not the way to go I’m afraid.

Are we not sending the frame counters along? That would be the real issue then.

[rvolosatovs]

@rvolosatovs are we keeping a "last seen" timestamp in NS?

No, but it can be derived from RecentUplinks (RecentUplinks[len(RecentUplinks)-1].ReceivedAt)

That's what I did, but I figured that for the complete functionality, we'd need to hook into the event stream to keep the numbers updated in real time.

Unfortunately, it's not as simple as it seems at first sight.

1. (ABP) Device, for which ResetsFCnt==true may reset FCnt itself
2. (OTAA) Device may rejoin, which in itself shall not reset the FCnt yet - FCnt shall either be reset or increased depending on the session key ID used in the next received data uplink
3. FCnt value in the uplink payload is not necessarily the actual device FCnt, because only 16 LSB are sent in the uplink, while the frame counters may be 32-bit wide.

All of this is handled by NS (and 2. in part by AS), and I certainly do not think console shall do the same.

I think the way forward is to add events for (1.) - ns.device.reset and (2.) - ns.device.confirm_session (or maybe ns.device.rejoin ?).
For (3.) we need to figure out a way to deliver the full FCnt value to the console. We may want to introduce another event, e.g. ns.up.data.match, or ns.up.data.handle which indicate successful processing of the uplink and includes full FCnt (and possibly more). Note, that ns.up.data.forward is not related here and merely indicates uplink being sent to AS, it is currently mistakenly published on each uplink, it will be now published only when actually being forwarded to the AS(PR incoming), also, the AS uplink payload does not contain the full FCnt, but the FCnt sent in the uplink (so, 16 LSB)

[rvolosatovs]

Blocked until we arrive to a conclusion on NS events to drive this and that is implemented in NS.

[kschiffer]

Ok I get that this is indeed not feasible then at the moment. What about the last seen value then. Is it save to obtain an initial value from the device registry and then keeping it updated via relevant device events (like uplink, confirmed downlink, etc) ?

[rvolosatovs]

Ok I get that this is indeed not feasible then at the moment. What about the last seen value then. Is it save to obtain an initial value from the device registry and then keeping it updated via relevant device events (like uplink, confirmed downlink, etc) ?

Yes, ns.up.data.receive, ns.up.join.receive, ns.up.rejoin.receive is what you're looking for

[rvolosatovs]

Blocked by #2221 #2222

[rvolosatovs]

NS now publishes ns.up.data.process event, which contains the uplink with full FCnt as payload

Please use the value in the uplink as actual active DevAddr and FCntUp of the device

[rvolosatovs]

For downlinks we currently do not have similar functionality, but that's also not the focus of this issue, let's start with just the uplink FCnt and then add downlinks at a later stage(Note, that depending on LoRaWAN version we may actually have 2 distinct downlink frame counters - one for NS and one for AS, so it's much less trivial that uplink frame counter as well.)

[laurensslats]

When we know the last uplink in the application, let's replace Linked with Last seen in the application view (similar to the gateway overview page).

[kschiffer]

When we know the last uplink in the application, let's replace Linked with Last seen in the application view (similar to the gateway overview page).

It would definitely be nice to show that information globally for the application, but doing so would mean that the console needs to probe all end devices associated with the application one after the other, to be able to show the initial last seen status. I don't think this is viable when we have to assume applications with potentially hundreds or thousands of end devices.

Two things I could think of:

1. I could show the last seen info as it gets available through the incoming stream of device message events. Depending on the setup of the application it could however mean that there will be no such events available in a timely manner (think of end devices that send downlinks every hour or less frequent).
2. We could use a threshold of the device count after which we stop calculating the last seen value initially. If an application has 100+ devices, it is also very likely that the necessary device events arrive very soon after loading the application page.

@htdvisser @rvolosatovs Do I maybe overlook something? Alternatively, can you think of any way of aggregating that data and making it available through an API endpoint?

[htdvisser]

ApplicationLinkStats just has a last_up_received_at and last_downlink_forwarded_at:

https://github.com/TheThingsNetwork/lorawan-stack/blob/develop/api/applicationserver.proto#L56-L69

[kschiffer]

Closed via #2585