Higher resolution for historical view

[corradio]

Is your feature request related to a problem? Please describe.
I often am interested in seeing what happened in the last few days, or looking at decarbonisation trends over longer periods.

Therefore, as a user, I'd like to be able to see historical data without too much loss of resolution. Currently, the Monthly view shows month-by-month data over a year, whereas it could be even more interesting to see daily/weekly data over a year.

The same applies to Yearly data, where it could be nice to see its evolution in a higher granularity (days?)

Describe the solution you'd like
I understand there might be both a need to

1. show data grouped by interval (e.g. monthly, yearly) to report aggregated statistics (e.g. the year 2024 had an intensity of X)
2. show long time intervals (e.g. a month) without loss of resolution

The two might be difficult to reconcile, but I feel that 2/ might be more important that 1/.
Maybe 1/ can be visualised somewhere else. One idea could be to put it in the tooltip? For example "The carbon intensity [for this hour] is X, which is Y% higher/lower than the 2022 average of Z"). That way, the user can scroll through the graph to find the yearly average they care about, while not loosing any resolution.

Another idea is to have

• a view that allows to go back X in the past (e.g. last day -- default, last week, last month, last year, last decade) with high resolution data
• a view that shows yearly/monthly/daily aggregates (I am actually not sure anyone cares about monthly or daily statistics in the past?). This might even be redundant with the data portal.

[VIKTORVAV99]

We really need to consider the payload sizes of the data here as well, there is a risk we will be sending huge payloads for this data if we increase the resolutions.

[corradio]

As inspiration, this is the "monthly" view from Ember. It shows monthly data but over a longer span than a year.

It's an interesting solution: we could simply increase the domain of the data shown for a given frequency:

• yearly -> show 10y
• monthly -> show 10y
• daily -> show 1y
• hourly -> show 5 days
• 5 or 15min -> show 2 days

The data payload would still increase however, but I guess this can be quantified.

[VIKTORVAV99]

As inspiration, this is the "monthly" view from Ember. It shows monthly data but over a longer span than a year.

It's an interesting solution: we could simply increase the domain of the data shown for a given frequency:

• yearly -> show 10y
• monthly -> show 10y
• daily -> show 1y
• hourly -> show 5 days
• 5 or 15min -> show 2 days

The data payload would still increase however, but I guess this can be quantified.

With the caching we are putting in place right now and the work that has gone into minimising the size of the state files this is becoming less of a concern performance wise on the backend parts of the system.

And if we only did it for say monthly data we wouldn't have to re-download the data every hour which would minimise the performance and data usage on the client.

Just to note though, for yearly we are displaying all the data we have that won't be making a lot of regions gray (We can look into if this is still the case, maybe it can be increased) and it will automatically keep incrementing and increasing each year.

[VIKTORVAV99]

The hourly state request is currently 127 kb data transfered and 702 kb total size.
Since the hourly state contains 25 datapoints we can conclude there is about 5kb transfered of data per datapoint. This includes exchanges and zones.

Using that as a baseline we can extrapolate the expected transfered sizes of the proposed horizons to be:

Resolution	Historical data points	Size
Yearly	10 years	50 kb
Monthly	120 months	600 kb
Daily	365 days	1 825 kb
Hourly	120 hours	600 kb
15-min	192 15-min intervalls	960 kb
5-min	576 5-min intervalls	2 880 kb

As we can see yearly is not an issue and will not be for many years.

Monthly is also an okay size and the same goes with hourly but we would likely have to introduce a last_month endpoint similar to the last_hour endpoint to reduce first paint in the app.

15-min is doable but I'd recommend lowering the horizon to just 1 day and add a last_15min endpoint.

Daily is way too big, that we would have to limit, possibly to 180 days or something similar and still add the last_day endpoint.

5 minutes is out of the question IMO, not only do we not have enough datasources that supply this data to justify it in the app the size and amount of datapoints would be huge meaning graphs and tooltips would be hard to navigate.

I hope this helps put things in perspective a little.

[madsnedergaard]

Here's some additional great context and discussion on this feature:

• This tweets suggest this feature would be useful - https://x.com/roobar99/status/1790953807305970163
• Additional tweet with more context and details on why: https://x.com/swbeeton/status/1791019973818851794

In the context of this use case - using your platform to inform - a single 12 month limit on the data you could provide could be argued as an "outlier".
Longer term trends could help inform the discussion.

Also, 2023 marked a record for Spain in particular as this was the first year they recorded greater than 50% of their yearly energy production being generated by renewables. The data as presented in your resource could be cherry-picked to infer differently (e.g., an outlier month rather than a yearly aggregate).

A historical annual trend, both by specific month (for seasonal trend insights) and overall annual could help establish your offering as a more authoritative resource.