New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Send the data in batches #58
Comments
Maybe worth mentioning that this came up due to finding some odd gps points from @jamoross ’s geigiecast. I can post a kibana link later but it looks like the device averaged some valid and invalid gps coordinates and sent us 0.003,0.009 as the location. It’d be good if we can only send the same points that show up in the log, perhaps filtered to only points with reasonable gps confidence. |
Here's one of the points in question http://kibana.safecast.cc/app/kibana#/doc/ingest-measurements-*/ingest-measurements-2021-02-20?id=UFd7vXcBj2uUTIA-1wb7&_g=(refreshInterval:(pause:!t,value:0),time:(from:'2021-02-12T12:28:03.937Z',to:'2021-02-24T05:23:17.620Z')), see https://api.safecast.org/en-US/bgeigie_imports/51022 for the SD log. {
"device_urn": "geigiecast:62025",
"device_class": "geigiecast",
"device": 62025,
"when_captured": "2021-02-20T03:28:47Z",
"loc_lat": 0.00253,
"loc_lon": 0.00944,
"loc_olc": "6FG22235+2Q8",
"lnd_7318u": 104,
"dev_test": true,
"service_uploaded": "2021-02-20T03:29:05Z",
"service_transport": "geigiecast:172.58.27.225",
"service_handler": "i-051cab8ec0fe30bcd",
"@timestamp": "2021-02-20T03:28:47Z",
"ingest": {
"location": {
"lat": 0.00253,
"lon": 0.00944
}
} |
Which would be here, way of the actual drive path. |
That is probally a rounding error for "NO GPS" (meaning 0,0) and a valid location. |
It's interesting looking at the log leading up to Any idea how the 0 might have reached the geigiecast for averaging? |
There are some
|
If it's possible it'd be great to have the geigiecast post the same data we'd find in a log. If every 5s will drain too much battery, then perhaps we could send the data in batches. |
For now I think we can probably just ignore stuff stuff with lat/lon in the 0-0.01 range when making graphs and maps. |
This comment has been minimized.
This comment has been minimized.
I must have just had something weird going on in my browser for that last screenshot. Data is there, here's the section with the weird point that reached ingest. https://5bc3d4f7330a4459881197a85659caf5.us-west-2.aws.found.io:9243/goto/58d9e28960c21350967606230935dd69 |
Here's a CSV incase it's helpful (zipped for github) |
The bGeigieCast was not designed to be a life streaming thing. It was made to be equivalent to the pointcast. That is why the 5 minutes come in. The 30 seconds were made for testing only, but I used it as a kind of live streaming. |
Yeah, that's understandable. I think it still works pretty well for both purposes and filtering after the fact isn't hard. |
It would be a better way to get all the data from the bGeigeiNano in batches to the api and not average (as now) calculated over 30 seconds or 5 minutes.
The text was updated successfully, but these errors were encountered: