Unexpected error fetching FoxESS data: Expecting value: line 1 column 1 (char 0)

[guba91]

HA 2023.5.3
foxess-ha version: v0.26

Logger: custom_components.foxess.sensor
Source: custom_components/foxess/sensor.py:276
Integration: foxess (documentation, issues)
First occurred: 23 maggio 2023 alle ore 16:33:05 (4 occurrences)
Last logged: 23 maggio 2023 alle ore 16:58:25

Unexpected error fetching FoxESS data: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _async_refresh
self.data = await self._async_update_data()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 217, in _async_update_data
return await self.update_method()
File "/config/custom_components/foxess/sensor.py", line 141, in async_update_data
await getAddresbook(hass, headersData, allData, deviceID, username, hashedPassword,0)
File "/config/custom_components/foxess/sensor.py", line 276, in getAddresbook
response = json.loads(restAddressBook.data)
File "/usr/local/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[M31Galaxy15287]

I'm experiencing the same problem. Usually it happens a few times a day, every day. Very annoying because I have a couple of utility meters that depend on the data that I have to manually calibrate each day now. Sometimes I only loose a couple of readings of the sensor.battery_soc entity.

Here is my log:

Logger: custom_components.foxess.sensor
Source: custom_components/foxess/sensor.py:305
Integration: foxess (documentation, issues)
First occurred: 15:54:33 (1 occurrences)
Last logged: 15:54:33

Unexpected error fetching FoxESS data: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _async_refresh
self.data = await self._async_update_data()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 217, in _async_update_data
return await self.update_method()
File "/config/custom_components/foxess/sensor.py", line 147, in async_update_data
await getReport(hass, headersData, allData, deviceID)
File "/config/custom_components/foxess/sensor.py", line 305, in getReport
for item in json.loads(restReport.data)['result']:
File "/usr/local/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[guba91]

at least i'm not alone <3

[RubenCoppensOnGit]

i created a fix. I forked the code .. and requested for a pullrequest to get it back into here.
No errors anymore.. NONE. I added a lot of optimizations in it. works without hangs.. already for 2 days now.
The real issue was a problem in Hass. helper method for api is not stable. I created a workaround
Also the adressbook issue is solved.
if you want you can try the version in the fork from : https://github.com/RubenCoppensOnGit/foxess-ha/tree/Fixing-HA-hang-key-error

i assume it will be merged soon here.

Enjoy

[M31Galaxy15287]

Thank you for your work!
I copied over your fix. I will give it a few days and then I will reply again with my findings. Hope this fixes my problem indeed.

[guba91]

i created a fix. I forked the code .. and requested for a pullrequest to get it back into here.
No errors anymore.. NONE. I added a lot of optimizations in it. works without hangs.. already for 2 days now.
The real issue was a problem in Hass. helper method for api is not stable. I created a workaround
Also the adressbook issue is solved.
if you want you can try the version in the fork from : https://github.com/RubenCoppensOnGit/foxess-ha/tree/Fixing-HA-hang-key-error

i assume it will be merged soon here.

Enjoy

I'm going to test it now! Thanks!

[guba91]

i created a fix. I forked the code .. and requested for a pullrequest to get it back into here. No errors anymore.. NONE. I added a lot of optimizations in it. works without hangs.. already for 2 days now. The real issue was a problem in Hass. helper method for api is not stable. I created a workaround Also the adressbook issue is solved. if you want you can try the version in the fork from : https://github.com/RubenCoppensOnGit/foxess-ha/tree/Fixing-HA-hang-key-error

i assume it will be merged soon here.

Enjoy
triyed your repo but i got this error in the logs:

2023-06-25 22:03:46.756 ERROR (MainThread) [homeassistant.components.sensor] Error adding entities for domain sensor with platform foxess
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 471, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 749, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 850, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 590, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 654, in _async_write_ha_state
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 596, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/sensor/init.py", line 469, in state
value = self.native_value
^^^^^^^^^^^^^^^^^
File "/config/custom_components/foxess/sensor.py", line 1252, in native_value
if self.coordinator.data["reportDailyGeneration"]["value"] == 0:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
KeyError: 'value'
2023-06-25 22:03:46.762 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up foxess platform for sensor
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 328, in _async_setup_platform
await asyncio.gather(*pending)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 471, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 749, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 850, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 590, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 654, in _async_write_ha_state
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 596, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/sensor/init.py", line 469, in state
value = self.native_value
^^^^^^^^^^^^^^^^^
File "/config/custom_components/foxess/sensor.py", line 1252, in native_value
if self.coordinator.data["reportDailyGeneration"]["value"] == 0:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
KeyError: 'value'

[essenemari]

Thanks! Actually it looks very promising for me, as issue #156 does NOT persist with your changes!

Before it was like below and counters were raising up during the time:

Tue 06 Jun 2023 09:51:30 PM CEST

lsof for homeassistant user: 920
lsof for homeassistant user and  8.209.116.72: 506
lsof only for 8.209.116.72: 4554
lsof for hass: 8286

Now, after few hour of testing

Mon 26 Jun 2023 09:53:38 PM CEST

lsof for homeassistant user: 344
lsof for homeassistant user and  8.209.116.72: 0
lsof only for 8.209.116.72: 0
lsof for hass: 4803

Looks awesome!

[xrad]

Seeing this error as of last night around ~ 0200 UTC. Rebooted HA but still no dice:

Logger: custom_components.foxess.sensor
Source: helpers/update_coordinator.py:313
Integration: foxess ([documentation](https://github.com/macxq/foxess-ha), [issues](https://github.com/macxq/foxess-ha/issues))
First occurred: 09:18:08 (1 occurrences)
Last logged: 09:18:08
Unexpected error fetching FoxESS data: Expecting value: line 1 column 1 (char 0)

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 313, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 269, in _async_update_data
    return await self.update_method()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess/sensor.py", line 125, in async_update_data
    token = await authAndgetToken(hass, username, hashedPassword)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess/sensor.py", line 251, in authAndgetToken
    response = json.loads(restAuth.data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I didn't change any of the creditials recently. I had updated to the latest HA core and HAOS but the system had worked fine after this for about a day, so I don't think it's related.

[FozzieUK]

FoxESS appear to have disabled private access to their cloud again, this was always going to happen but the timing of it seems to have taken everyone by surprise, so I suspect once FoxESS UK support get involved with their opposite numbers in China that change will be reversed.

The OpenAPI version is written and working, it needs a personal_api_key to operate and you get that from the FoxESS cloud platform - but at the moment the request a key isn't working either, if you have a personal api key already their openAPI will work, but until they get that request screen working again there's not much to do - i've just given FoxESS UK support a nudge.

I'll post more when I have updates

[xrad]

I see. Couldn't obtain the API key either. Let me know if there is anything I can help with.

[xrad]

API key generation is back online and 0.33 appears to work fine! Thanks a lot!

[MartinBlackburn]

Can also confirm I've been able to update to the open API version and generate an API key which seems to be working so far.
Thanks for fixing this @FozzieUK

[xrad]

Thanks again, I have not seen any issues since the switch.

Since the OpenAPI server limits the number of accesses - could it be an interesting feature to do the queries in a non-linear way, for example do less queries during the night and more during daytime?

[FozzieUK]

@xrad i’ve been working on improving the pacing mechanism for api calls.
It is currently running at ~ 35api calls per hour, i’m testing a version that runs at ~25 api (next release) but as the openapi only returns the last ‘real time’ variables data you have to poll at the datalogger interval speed or risk losing data.
I think there is some mileage in slowing down the daily totals (feedin, generation, loads etc..), the battery parameters, and daily generation but once you get below 25 api calls they are only small improvements - if I can get it reliable at 20 api calls per hour, i’ll be happy 😊

[xrad]

Makes sense. I totally ignored the datalogger invervall in my speculation. Cheers!