Unit of measurement as kkWh?

[bigbadoooff]

System Health details

System Information

version	core-2024.3.3
installation_type	Home Assistant Container
dev	false
hassio	false
docker	true
user	root
virtualenv	false
python_version	3.12.2
os_name	Linux
os_version	6.1.21-v8+
arch	aarch64
timezone	Europe/Berlin
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
GitHub API Calls Remaining	5000
Installed Version	1.34.0
Stage	running
Available Repositories	1411
Downloaded Repositories	31
HACS Data	ok

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Dashboards

dashboards	3
resources	17
views	9
mode	storage

Recorder

oldest_recorder_run	26. März 2024 um 21:08
current_recorder_run	4. April 2024 um 22:03
estimated_db_size	974.83 MiB
database_engine	sqlite
database_version	3.44.2

Checklist

• I have read the FAQ to see if there is a known solution to my problem.
• I have enabled debug logging for my installation.
• I have filled out the issue template to the best of my ability.
• This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).
• This issue is not a duplicate issue of currently previous issues..

Describe the issue

Since updating to Homeassistant 2024.3.3 i get this unit error for type energy from a real power sensor:
Somehow it has unit "kkWh" thus rendering it useless. The input sensor provides its value as "kW"

Reproduction steps

1. Setup new energy from real power sensor

Debug logs

2024-04-04 20:05:29.897 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Forced energy sensor generation for the power sensor 'sensor.verbrauchszahler_waschmaschine_2'
2024-04-04 20:05:29.898 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor: Waschmaschine energy
2024-04-04 20:05:29.898 DEBUG (MainThread) [custom_components.powercalc.sensors.utility_meter] Creating utility_meter sensor: Waschmaschine energy daily (entity_id=sensor.waschmaschine_energy_daily)
2024-04-04 20:05:29.898 DEBUG (MainThread) [custom_components.powercalc.sensors.utility_meter] Creating utility_meter sensor: Waschmaschine energy weekly (entity_id=sensor.waschmaschine_energy_weekly)
2024-04-04 20:05:29.899 DEBUG (MainThread) [custom_components.powercalc.sensors.utility_meter] Creating utility_meter sensor: Waschmaschine energy monthly (entity_id=sensor.waschmaschine_energy_monthly)
2024-04-04 20:05:29.901 WARNING (MainThread) [homeassistant.components.sensor] Entity sensor.waschmaschine_energy (<class 'custom_components.powercalc.sensors.energy.VirtualEnergySensor'>) is using native unit of measurement 'kkWh' which is not a valid unit for the device class ('energy') it is using; expected one of ['MJ', 'MWh', 'Wh', 'GJ', 'kWh']; Please update your configuration if your entity is manually configured, otherwise create a bug report at https://github.com/bramstroker/homeassistant-powercalc/issues

Diagnostics dump or YAML config

config_entry-powercalc-5a92eb5c910b2ee586e5481576c1e60c.json

[bramstroker]

Hmm it's strange. I am unable to reproduce yet.
Seems similar as bug reported long time ago: ttps://github.com//issues/1676
And actual fix here: https://github.com/bramstroker/homeassistant-powercalc/blob/master/custom_components/powercalc/sensors/energy.py#L154

When did you create this sensor? Recently or long time ago?

Could you also create an energy sensor using the HA helper Riemann Sum, and see if that gives the same problem.
Powercalc uses the same helper to create the energy sensors, so when you have the same issue using the HA core helper it is a bug in the core.

[ardeus-ua]

Same issue. Had to delete and re-add devices with proper unit of measurement

[bramstroker]

Do you both have this issue since that exact version HA 2024.3.3 or unsure?

[bigbadoooff]

I am unsure, I skipped a few versions since the beginning of the year.
Your hint to try it with a regular riemann-helper lead to the solution, because I had the same issue there.

I had my sensors set up as "kW", I switched them to "W" and all was working again.
But I'm still not sure what's the reason because it worked like this before.

Nevertheless, it's not a bug in powercalc. Thread can be closed. Thanks for your input and keep up the great work!

[TheSerapher]

I just had the same thing happen after the upgrade from 2024.4.0 to 2024.4.1 - after a restart the energy sensor created from a charge power meter of my car (read via car since my charger doesn’t allow reading data) was returning kkWh. I only noticed because the energy dashboard didn’t show any new data since the restart.

I removed a group that included the meter which allowed data to be refreshed for another group that included it too (All Rooms -> Garage -> Vehicle Energy -> Meter from power reading).

Re-added it and it broke again. Tried to add the Garage as its own „from net“ as energy source and it showed the „invalid unit“ error which lead me here.

tried to delete it all to recreate the meter but now it won’t create the entities for energy readouts. Not sure if that’s related or just an issue because there is no data (not charging so 0 power).

[bramstroker]

I have created a bug report in HA core issue tracker: home-assistant/core#115077

[bramstroker]

I'm not sure if it's a problem with the riemann sum core helper, as the author mentioned the is kkWh issue is not present when you omit the unit prefix when setting up.
I verified that by setting up a new sensor and the sensor was correctly created with kWh unit of measurements.

Powercalc also forces the unit prefix to be None, so I don't see why you guys have this issue all of a sudden.

Might be that it were sensors set up a long time ago and were always kkWh, but did not cause any problems.
There were a lot of changes in unit conversions and validations in HA lately, so it might be it's rejected by the validation mechanism now.

I have added some more extensive debug logging in 1.11.6 which I just released.

Would help if any of you still experiencing the issue install that version, enable debug logging for Powercalc and restart.
And share the logs here.

[bigbadoooff]

I set up my whole system in january, that's not so long ago. And it definitely worked before with the correct units. The log shows this, so it has ben kWh before.
2024-04-07 14:40:35.225 WARNING (Recorder) [homeassistant.components.sensor.recorder] The unit of sensor.waschmaschine_energy (kkWh) cannot be converted to the unit of previously compiled statistics (kWh). Generation of long term statistics will be suppressed unless the unit changes back to kWh or a compatible unit. Go to https://my.home-assistant.io/redirect/developer_statistics to fix this

Here is the log from before with extended debugging:

2024-04-07 20:18:32.271 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Forced energy sensor generation for the power sensor 'sensor.verbrauchszahler_waschmaschine_2'
2024-04-07 20:18:32.272 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.waschmaschine_energy, source_entity=sensor.verbrauchszahler_waschmaschine_2, unit_prefix=None)
2024-04-07 20:18:32.272 DEBUG (MainThread) [custom_components.powercalc.sensors.utility_meter] Creating utility_meter sensor: Waschmaschine energy daily (entity_id=sensor.waschmaschine_energy_daily)
2024-04-07 20:18:32.273 DEBUG (MainThread) [custom_components.powercalc.sensors.utility_meter] Creating utility_meter sensor: Waschmaschine energy weekly (entity_id=sensor.waschmaschine_energy_weekly)
2024-04-07 20:18:32.273 DEBUG (MainThread) [custom_components.powercalc.sensors.utility_meter] Creating utility_meter sensor: Waschmaschine energy monthly (entity_id=sensor.waschmaschine_energy_monthly)
2024-04-07 20:18:32.275 WARNING (MainThread) [homeassistant.components.sensor] Entity sensor.waschmaschine_energy (<class 'custom_components.powercalc.sensors.energy.VirtualEnergySensor'>) is using native unit of measurement 'kkWh' which is not a valid unit for the device class ('energy') it is using; expected one of ['Wh', 'GJ', 'MJ', 'kWh', 'MWh']; Please update your configuration if your entity is manually configured, otherwise create a bug report at https://github.com/bramstroker/homeassistant-powercalc/issues

Does this help?

[erkr]

I have seen this error before! Happened directly using the riemann integration sensors.

Reason was a mismatch between the unit of the source sensor and defining the unit_prefix: "k" for the riemann sensor.
So I guess powercalc assigns a source sensor in kW to an energy sensor where the unit_prefix: 'k' is defined. In that case you get 'kkWh'.
When you remove the unit_prefix for source sensors in kW, it's solved.

[bigbadoooff]

the current code should already do this. But for some reason it stopped working. Oddly enough after several restarts it now works again. I did not change anything in the setup of the affected sensors.

I still think that the problem lies with the underlying integration of the riemann sum and I don't like the way the unit handling is done. Imho all calculations should be done in the respective SI base unit. Everything else just changes the scale of the output.

[bramstroker]

Powercalc creates a Riemann sensor under the hood, just as you would by manually creating one from the HA helper page.
The unit_prefix will be set whatever you configured in powercalc with the energy_sensor_unit_prefix setting, this defaults to k.
When the source power sensor is in kW and the setting energy_sensor_unit_prefix is set to k, then Powercalc will set the unit_prefix to None to prevent the duplicate kkWh issue.
This has always worked, but some of you are experiencing issues since latest HA versions.
While diving some deeper the only reason kkWh can happen is that Powercalc sets unit prefix to k during sensor creation.

I might have an idea what causes this, because the new HA versions are very fast in startup time, which makes me believe the state of the source sensor is not available at that moment. This is where Powercalc tries to get the unit of measurement from. So it will be a hit and miss during startup, where sometimes it is kW or sometimes it is unknown.

Worked on a fix already yesterday (#2191) which you guys might check to see if that solves the problem, even after multiple restarts.
You can install this by clicking green Code button on main repository page and than Download Zip. Next copy custom_components/powercalc contents to config/custom_component/powercalc directory.

Also I am with you that I think this issue should really be solved in riemann sensor, where the conversion should happen. And prevent these kind of issues with wrong unit being constructed.
End users will also click prefix to k while setting up for a source sensor in kW, which will also cause the kkWh issue. Found multiple reports on that in discord channel. It's not user friendly and very error prone this way.

[TheSerapher]

@bramstroker i think you are right with your startup order. I’ve done quite a few lately (testing some things) and it hasn’t occurred but then suddenly units were back to kkWh. I’ve simply fixed them in the developer tools and measurements are coming in again.

Good to know what’s causing them - for now I can check after restarts if units are detected properly.

Thanks for you hard efforts on this integration for all of us 😃

[TheSerapher]

I just noticed that the state of my meter changed without a restart. I manually updated it to kWh after a restart and it suddenly went back to kkwh without doing anything. Not sure if that’s relevant but something I observed just now.

[TheSerapher]

Just happened again not 44 minutes later. Reset manually to kWh again

[bramstroker]

I have released v1.11.7, hope this solves anything for you all.

[TheSerapher]

I had the custom power sender meter removed and upgraded to the latest version. Re-created it but it showed the same error right away. Manually adjusted it to kWh and restarted but it showed it again.

maybe worth mentioning that this is a car charger which does log consumptions beyond 150.000 W.

Logger: homeassistant.components.sensor
Source: components/sensor/__init__.py:737
integration: Sensor ([documentation](https://www.home-assistant.io/integrations/sensor), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+sensor%22))
First occurred: 08:56:37 (1 occurrences)
Last logged: 08:56:37

Entity sensor.id_4_gtx_meter_energy (<class 'custom_components.powercalc.sensors.energy.VirtualEnergySensor'>) is using native unit of measurement 'kkWh' which is not a valid unit for the device class ('energy') it is using; expected one of ['GJ', 'Wh', 'kWh', 'MJ', 'MWh']; Please update your configuration if your entity is manually configured, otherwise create a bug report at https://github.com/bramstroker/homeassistant-powercalc/issues

[bramstroker]

@TheSerapher Are you also able to create a kWh for the same power meter without using powercalc?
Directly with the Riemann sum helper provided by HA.

And see if you have similar behaviour with that one?

[TheSerapher]

Hey @bramstroker,

Finally got around to test this. It's behaving the same way without me ever charging the car at all - so it didn't sum any data but still said 0.0 kkWh. When creating the Riemann sum integral helper I selected (k) as the scale. Maybe that was causing this?

[bramstroker]

@TheSerapher You'll need to create the rieman integral helper manually without selecting a scale. Because when you select it it will result in the wrong unit of measurements. Because the source is already in kW the riemann sum will just prepend k to it, causing this issue.
So create one without selecting k and see if you eventually will have the kkWh also.

Please also enable debug logging for powercalc.
And check for the following log line:
Creating energy sensor (entity_id=...., source_entity=..., unit_prefix=...)

This is logged after every startup where powercalc is creating the riemann sum sensor under the hood.

unit prefix can either be k or None. In your case it should always be None after every restart of HA.
When this is k at any moment, this can cause the issues with kkWh.

[TheSerapher]

Looks like it was created as you anticipated:

2024-05-03 13:31:02.090 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.id_4_gtx_powercalc_meter_energy, source_entity=sensor.id_4_gtx_charging_power, unit_prefix=None)

After a restart:

2024-05-03 13:31:02.090 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.id_4_gtx_powercalc_meter_energy, source_entity=sensor.id_4_gtx_charging_power, unit_prefix=None)

So hopefully this solved the issue - I will keep an eye on it once I charge the car again.