Is it possible to check that 'last timestamp' is the actual time from the corresponding GPS data?

[jata1]

Hey Gary - Congrats on the release of icloud3 v3. I have updated already.

I am having an ongoing battle with speed calcs in the last km from home related to the high frequency GPS data. I am working on a couple of solutions but still struggling to find one that works!

As an alternative to dropping some of the data for speed calculations, I wanted to explore if there was a way to improve the speed calculation accuracy. My GPS accuracy is high (4 or 7m) so I'm thinking that speed calcs should be quite accurate too.

Is there a way you can help me test/confirm that the last timestamp attribute is the actual/exact time relating to the GPS data?

The reason I ask is Jeff (composite author) is using GPSlogger with a high frequency GPS update rate and speed calcs are good.

[gcobb321]

If you set the log level to rawdata on the Event Log > Actions > xx or Configure Settings > Page 2 > Format Parameters screen, the actual data from iCloud is displayed in the iCloud-0.log file. The timestamp value and lat/long values are displayed. You could compare that value with the one in the devtrkr entity’s attribute for locations from famshr. The timestamp value is where the location timestamp and time comes from

[jata1]

OK thanks Gary. Congrats again on the release of v3.

I think I have cracked the solution from the other side. Getting good GPS data at 2min intervals when away from home using an automation/script and a template sensor triggered by the script.

The drive home tonight will be an good test as this is when I get the high frequency GPS data affecting speed calcs...

[jata1]

Working! Below is a chart with raw icloud and a composite that has combines HA app with corrected icloud3 data.

Thanks for all the help with this over the last week or so - much appreciated

And below is the end-to-end project working correctly - detecting when speed is >10% over speed limit.

[gcobb321]

Glad you got it working. I assume they they get their bicycles out if there is a 25% over spike.

I assume the spike in the top image is the short interval iCloud3 data.

[jata1]

Haha - good idea on the bikes. Currently only tracking myself and the wife. My son is not driving yet so I have a bit of time to get this all working.

I am getting an occasional false positive for my wife and I have her setup through my icloud account (family sharing). Some of her GPS accuracy data is lower than mine (50m) but we have the same phone so not sure why this is the case. Is family sharing GPS handled differently?

I will examine the raw logs and let you know if I find anything.

[gcobb321]

The operation of iCloud3 is handled by icloud3_main.

Every 5-secs, see if anything needs to be done [ _polling_loop_5_sec_device]

• Check is made to see if the mobile app issued an update trigger or the location changed. [_main_5sec_loop_update_tracked_devices_mobapp]
  • If so, validate the data (gps accuracy, not old, not already processed, etc). [_validate_new_mobapp_data]
    • If useable, update the device [process_updated_location_data]
      • Determine the interval to the next location update, Zone distance, stationary zone update, etc and update the sensors and device tracker entity. [determine_interval.determine_interval]
• Check the timer (next update Time, enter stationary zone, old location update, enter zone delay, etc) to see if an iCloud l location is needed [_main_5sec_loop_update_tracked_devices_icloud]
  • If so, get the location data from iCloud for all devices in the famshr account and check the data for all devices to see if it is useable [_validate_new_icloud_data]
    • If useable, update the device [process_updated_location_data]
      • Determine the interval to the next location update, Zone distance, stationary zone update, etc and update the sensors and device tracker entity. [determine_interval.determine_interval]

A lot of other things go on to handle other things dealing with bad Location data, Stationary zones, devices that are monitored, zone handling, etc. but that it’s the basic logic. Once the data is useable, it is always handled the same for all devices and all data sources.

Gps accuracy is different for each device, even sitting side-by-side. It depends on the cell signal, satellite signal, internal components, etc. Google can explain it in more detail.

Hope this helps. Dig through the code to see how things are handled if you are so inclined.

[jata1]

Thanks Gary.

I have been eyeballing the log and found something strange with these two items. They are in the log one after the other (2mins apart) but the GPS data is only 5 secs apart.

03-26 17:02:02 [pyicloud_ic3:1506] ICLOUD > ──────── FAMSHR DATA - <MAGDA'S IPHONE/MAGDA_IPHONE> ────────
{'▶ITEMS◀ (items)': {'id': 'AdpQUk+gea...', 'modelDisplayName': 'iPhone', 'lostModeCapable': True, 'name': "Magda's iPhone", 'deviceClass': 'iPhone', 'deviceStatus': '200', 'rawDeviceModel': 'iPhone14,2', 'batteryLevel': 0.4399999976158142, 'deviceDisplayName': 'iPhone 13 Pro', 'prsId': 'MTQ1MDE3NTcyNg~~', 'batteryStatus': 'not charging', 'deviceModel': 'iphone13Pro-1-1-0', 'data_source': 'FamShr'}, '▶LOCATION◀ (location)': {'isOld': False, 'isInaccurate': False, 'altitude': 0.0, 'latitude': -33.86737200527077, 'horizontalAccuracy': 13.59509039048701, 'timeStamp': 1711432921789, 'verticalAccuracy': 0.0, 'longitude': 151.20902364724404, 'timestamp': 1711432921, 'location_time': '5:02:01p'}}


03-26 17:04:00 [pyicloud_ic3:1506] GETLOC ⡇ ──────── FAMSHR DATA - <MAGDA'S IPHONE/MAGDA_IPHONE> ────────
{'▶ITEMS◀ (items)': {'id': 'AdpQUk+gea...', 'modelDisplayName': 'iPhone', 'lostModeCapable': True, 'name': "Magda's iPhone", 'deviceClass': 'iPhone', 'deviceStatus': '200', 'rawDeviceModel': 'iPhone14,2', 'batteryLevel': 0.4399999976158142, 'deviceDisplayName': 'iPhone 13 Pro', 'prsId': 'MTQ1MDE3NTcyNg~~', 'batteryStatus': 'not charging', 'deviceModel': 'iphone13Pro-1-1-0', 'data_source': 'FamShr'}, '▶LOCATION◀ (location)': {'isOld': False, 'isInaccurate': False, 'altitude': 0.0, 'latitude': -33.86806048337412, 'horizontalAccuracy': 35.0, 'timeStamp': 1711432925349, 'verticalAccuracy': 0.0, 'longitude': 151.20977275677794, 'timestamp': 1711432925, 'location_time': '5:02:05p'}}

[gcobb321]

Not strange at all. iCloud does this all the time. I guess it will return the data it already has if there are delays contacting the phone for its location.

Look at the Event Log and the Located items. It shows the time change for each item and sometimes does not c change.

[jata1]

It’s that these two gps data points create speed delta of 100km/h which was not remotely possible at that time

[jata1]

Sorry to be a pain. Just trying to work out what’s happening so I can see if I can use the data for speed tracking.

I suspect this might also be related to the speed jitter when close to home and there is high frequency gps data being requested.

[gcobb321]

For your use case, you have to analyze the data you are getting and filter out what you should not use. You can ignore a result if it doesn’t make sense (spike, x% change in short period of time, etc) because you can not control the data you are getting.

[jata1]

That is a good idea. I have been thinking the same but I need to catch this before I push the data into the composite integration but I need the composite integration to do the speed calculation that catches the error.

I will find a way but good to see if we can fix it at source. Then it will ‘just work’ (it is an Apple thing after all) haha

[jata1]

I am working on a slightly different approach with my speed sensor. I will only use GPS data if it is 1min plus old. This should help with random blips caused when the data is too close together. Also eliminates speed data being used during the high frequency period when getting close to home.

I will keep you posted!

And thanks again for all your help with this.