-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DB contains records with same state #63
Comments
I'm curious, if that entity is usually not "visible" (which I'm not entirely sure what you mean), and rarely changes, why are you using it as an input to a composite entity? For most FWIW, this integration was created well before it was possible to filter individual attributes from the DB. It currently does not use that feature at all. (In fact, most of my integrations are old enough that they didn't do this. I've only used it so far on two of them, sun2 & google_maps.) Anyway, I'll consider adding this feature. Probably several of the attributes shouldn't be recorded. However, I'm not entirely convinced |
Long story... Then I started using iphonedetect integration. For a mainly testing purpose I "linked" these 2 integration - ASUSWRT & iphonedetect - inside the Composite. After some update of ASUS router both integration (ASUSWRT & iphonedetect) stopped seeing Watch at all (although Watch still can sync mails w/o bluetooth via Wifi) - but it is another story... |
You might want to read the composite docs more closely, especially the part that talks about watched entities, and the |
Absolutely agree. If coords (+maybe accuracy), state, battery data are changed -> a new record.
Could you specify a way by which we can use this historical attribute in Lovelace? |
A composite tracker entity was defined w/o and these records in the "states" table with same state & same attributes (issue): So, based on DB see no reason for Composite to update |
That is not the only way historical data can be used.
That's not completely true. last_update_ts is changing. And since this entity does not supply a |
Yes.
No, that is already the default value of that option. It is using all states because there are no GPS-based inputs, and really no other input either.
It is directly related.
I couldn't say. I'd need to see the complete state history for both for that time period to say. Or maybe did HA restart at one of those times? Did the router entity become unavailable or something? Again, hard to say without knowing the exact state changes of both for that period. |
Hope you understood me properly, these "(may be unrelated)" words were about the "But why the composite entity has "last_changed" = yesterday" sentence.
I see. Not sure if it can help you:
I think I found it: My expectation was: if the state of composite tracker is not changed (see a History above) - then its "last-changed" should be some old value, not a time of last HA startup. Could it be related that config is still in yaml? As for the main issue: |
Yours, as well as many others. But, unfortunately, that's now how HA works. The last_changed & last_updated state fields are set by HA core code and is based on when the state is set by an integration. last_changed simply is the last time the state string is changed (e.g., home, not_home, etc.), whereas last_updated is the last time any part of the state (including attributes) is changed. The only thing an integration can change about that is that it can, if it wants, force last_changed to update even if the state string hasn't changed (assuming at least something else about the state has changed.) These fields initially get set when HA (and the integration) starts. Therefore, even if an entity is restored (with its state from before HA restarted), last_changed & last_updated can't be any value earlier than when HA starts. |
I see, like some other entities in HA (helpers). As for the initial issue: |
I took a look at the iphonedetect integration and the issue you raised there. That integration is pretty old and fairly simple. Reading through that code, and the associated device_tracker component level code, I'm not seeing how it would create state_changed events, and additional records in the DB, when its state remains not_home. Are you sure you're reading the DB tables correctly? What would help is if you could enable debug for homeassistant.core. Then, whenever any entity in the system changes, and a state_changed event is fired, there would be a record of it. Then you could search for all those messages that are related to Please add the following to your configuration.yaml file: logger:
logs:
homeassistant.core: debug Then restart HA. When you believe there have been unnecessary DB records created for these entities, search home-assistant.log in the same folder as configuration.yaml. If you can get to a command line and know how to use grep, you might try something like:
Then share that output here. If you're not familiar with how to do this from a command line, you could always go to Settings -> System -> Logs, then enter |
BTW, I'm also curious, do you still use the asuswrt integration? If so, it apparently adds a |
I use ASUSWRT, but for some devices (like Mikrotik switches) is shows I will answer your long post a bit later. Thanks a LOT for supporting! |
I will do it a bit later. |
FWIW, I have a Python script that can extract info from HA's database. It broke a while ago when the schema changed, but I just updated it. It should be good for HA versions 2023.4 and later. Anyway, it looks kind of like this:
You can specify which entity_id's you want to see, and optionally you can specify which attributes you want to see, and you can see events. All of those can optionally use regular expressions. You also have flexible control as to what time window you want to see. And the output is colorized to make it easier to browse over. If you're interested at all in this script, let me know and I can get it to you and explain how to install and use it. |
It looks like the data you got from the database is relevant, but it's not really sorted, and it doesn't show if/when HA restarted in that sequence. Getting the DEBUG state_changed events from the log, or using my script, either way would help to fully understand what is happening with that entity, and how that impacts the composite entity that uses its state. |
It's very kind of you! I do not know sql, have to analyse tables in my primitive way.
Check the "last-changed" column (or added datetime) - it is sorted by a date.
Will try to prepare info soon. |
Ok, I see that now. It is indeed sorted by time (and, of course, state_id, which is effectively is the same thing for sorting this data.) Whenever you can, no rush. As you say, at this point we're really trying to better understand the iphonedetect entity first. |
After looking at the iphonedetect code I realized it might work with my Android phone, too. So, I installed it on my test system and configured it to check my Android phone. Sure enough, it found it, and created a device_tracker entity whose state was home. I let it run a while, then turned off my phone's WiFi. Via DEBUG messages I could see the next time it checked the phone, it noticed it no longer responded. Then, after the configured consider_home period, the device_tracker changed to not_home. I checked the log and used my script that extracts state info about entities, and both showed the iphonedetect device_tracker entity behaved as I expected. I.e., it did not create state_changed events constantly. It only created them when it first changed to home, and when it later changed to not_home. Next, I added a composite entity whose only input is the iphonedetect device_tracker. It, too, behaved as I expected. I.e., it only created state_changed events, and DB entries, when it changed state based on the iphonedetect entity changing state. It did not create multiple entries with the same state except for last_seen. So, I could not reproduce what you're observing. |
And, FWIW, a direct query of the DB:
|
Interesting. Could it be related to a router? But anyway. Assume this is not iphonedetect but some xxxdetect. As I said earlier, this is a dilemma:
I do not think that currently it is possible to "skip a write to DB" if an object is updated (here - |
I doubt it. The integration is directly sending UDP packets to the phone. I doubt the switch function of the router would have any effect on that.
That is possible, but unlikely. An integration can force an update (where both last_updated & last_changed are updated), even if the state & attributes do not change, but most do not. I checked iphonedetect and it does not. I browsed the starline issue & code and I don't see where it does it either. I think the issue here is that you're seeing updates from different runs of HA, i.e., across restarts. When HA & integrations start, they will fire state_changed events for all the entity objects that are created, which will result in DB records. If those integrations restore state, then likely the state & attributes will be the same. If you look closely to the records you circled above, you'll see the values in the old_state_id column for those records are all NULL. This means all these records happened when the entity object was created, either when HA starts, or when the integration is reloaded. Bottom line, I think your interpretation of the DB records is incorrect. What would help, as I've said before, is messages from the log and/or using my hadb.py script to pull out not only the records associated with state changes, but also events that show when HA restarts. |
Absolutely not.
Yes, as you explained it here - #63 (comment)
Interesting. I will look closer.
OK, I will do it! Thanks again! |
@ildar170975, can you provide an update? |
Hi! Not yet... Was very far from a PC, just came back. Will try to give smth these days, thanks for not forgetting!!! |
Closing since there has not been an update for a couple of months. Also, I don't believe this integration is doing anything wrong here. Feel free to reopen with new data/info. |
Was not able to add any updates since was rather far from a home PC. |
Here is a "states" table for one
device_tracker
entity:Look at the "attributes_id" column.
Here just 2 entries with different
attribute_id
:A difference is only in "last_seen" attribute.
In this particular example it was a tracker for Apple watch - this device is usually not "visible" & rarely changes it's state.
In case of another device there could be LOTS of records in DB.
Is it possible to exclude the
last_seen
from DB & not store a new record when only this attribute is changed?The text was updated successfully, but these errors were encountered: