Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Price devices distinctively in scheduling #618

Open
7 tasks
nhoening opened this issue Mar 28, 2023 · 20 comments · May be fixed by #654
Open
7 tasks

Price devices distinctively in scheduling #618

nhoening opened this issue Mar 28, 2023 · 20 comments · May be fixed by #654

Comments

@nhoening
Copy link
Contributor

nhoening commented Mar 28, 2023

Based on #615, this feature allows specifying price sensors per device, flexible or unflexible.

In the flex-contextparameter, we already have consumption-price-sensor and production-price-sensor. They will remain the default (see https://flexmeasures.readthedocs.io/en/latest/api/notation.html#flex-context).

New parameters consumption-price-sensors-per-device and production-price-sensors-per-device can optionally be added to overwrite this default, so that some devices are priced by other price sensors. These parameters are dicts, mapping the power sensor IDs of devices (assets) to price sensor IDs.

TODO:

  • Adapt FlexContextSchema (in data.schemas.scheduling.__init__.py). This should suffice to make the API parse these new parameters and pass them on.
  • Also add these parameters to the CLI command flexmeasures add schedule for-storage (in cli/data_add.py), so they end up in flex_context (line 1054ff). This should help with testing locally.
  • Get prices for these distinct price sensors from the database in data.models.planning.storage.py::StorageScheduler.compute_schedule() (also add the price slope trick for these) and pass them on to device_scheduler(). I'd say we keep them as a dict until here
  • In device_scheduler, we need to adapt the model so that per device (denoted by "d" in the ConcreteModel), the correct price is applied. I believe the best places are price_down_select() and price_up_select().
  • Add a scheduling unit test. Idea: In data.models.planning.test_solver.py::test_building_solver_day_2(), we could add a market_scenario parameter for the case that the price for solar (feed-in tariff) is much higher than the tariff used by the battery (the default). Then we can expect the battery to never charge. The pytest fixture create_test_tariffs might be a good place to add the extra price sensor and data.
  • Add documentation for the new parameters, in the flex context description.
  • Bonus: Adding an integration test with the API (api.v3_0.tests.test_sensor_schedules.py), which is crucial for the actual operation. However, the existing tests are not including the flex context a lot, anyway, so I can't demand it from you now.
@nhoening nhoening added this to the 0.13.0 milestone Mar 28, 2023
@nhoening nhoening added the API label Mar 28, 2023
@Flix6x
Copy link
Contributor

Flix6x commented Mar 30, 2023

I have only an implementation detail / clarification to add. I don't think the new prices per device should overwrite the price that can be set currently.

The current price is being applied to the aggregate power flow of the whole portfolio of devices being scheduled. This represents, for example, a building's net power flow to/from the grid.

The new price per device should be applied to the power flow to/from a single device and should be implemented as an additional costs, imo. This represents, for example, a device's usage-dependent depreciation costs.

@nhoening
Copy link
Contributor Author

I see how you came to this, but I guess one does not have to view the portfolio as aggregated (on one contract). That seems to be our default view (like, for a building or industrial site), but maybe it does not apply always. In fact, the intention here seems to be that the individually-priced devices are not part of the portfolio. Maybe that can be clarified by stakeholders.

If the two signals are vastly different (two different price curves), the original intention would become harder to maintain (one would need to compute the difference between these two independent price curves).

@nhoening
Copy link
Contributor Author

I added a TODO item, so this feature is also available through the CLI.

@Srieon
Copy link

Srieon commented Apr 12, 2023

hi @nhoening I was trying to implement the toy example using the API endpoints so that i can extend it to do the above changes discussed in the issue. All the endpoints are working fine except for the /api/v3_0/sensors/(id)/schedules/trigger which is used to create the schedule.
It gives an "[Error 99 connecting to localhost:6379. Cannot assign requested address]"
This seems to be an error regarding the redis configuration ( judging by the port number) but the toy example is working fine with CLI .

@nhoening
Copy link
Contributor Author

Hi @Srieon, the toy example using the CLI would by default not require Redis, as the --as-job parameter is False per default.

The API expects to be able to queue jobs for execution, so a Redis node is needed. You can also run one via Docker, btw, see the Docker-compose file.

@Srieon
Copy link

Srieon commented Apr 25, 2023

hey @nhoening I was trying to access the trigger endpoint "/api/v3_0/sensors/(id)/schedules/trigger" with the request body consisting of both consumption-price-sensor and production-price-sensor. It throws an error if one of them is missing . Ideally shouldn't they both be same. I can see that at various places you have implemented that if consumption-price-sensor is missing consumption-price-sensor =production-price-sensor. But this endpoint doesn't have that . Is there a reason for it ? Could I just add that line so that my request body only requires one price sensor?

@nhoening
Copy link
Contributor Author

Hi @Srieon, I agree that often they are the same.
I see that in the CLI command production prices default to consumption prices, but right now I don't see where else we do that.
The endpoint code would not be the best place to do that, as it doesn't parse the flex context. This point where we actually do that would be better. (and then the line in the CLI command might be able to go).

And we should document this defaulting better in the relevant documentation.

Supporting this defaulting better could be a small PR in itself, actually.

Does this help?

@anirudh-ramesh
Copy link

Hi @nhoening, the first couple of items to modify i.e. adding parameters in order for the API and CLI to parse the new fields for the sensor dictionary. We're stuck in knowing if this makes sense to you. Perhaps I could issue a PR into which we can keep adding commits and you could keep reviewing until all of these criteria are met?

Cc @Srieon

@nhoening
Copy link
Contributor Author

Yes a PR might be a good idea. You can set it to draft status if it's not ready to be merged, but still ask for a review.

P.S. the first sentence of your last post seems to be missing a verb, so I did not get what you were saying there 😃

@anirudh-ramesh
Copy link

Ah, yes, sorry, I didn't proofread it. I meant to say that we'd implemented the first two items on the list.

rajath-09 pushed a commit to Srieon/flexmeasures that referenced this issue Apr 26, 2023
@nhoening nhoening linked a pull request Apr 29, 2023 that will close this issue
2 tasks
@nhoening nhoening modified the milestones: 0.13.0, 0.14.0 May 1, 2023
@rajath-09
Copy link

Hello @nhoening

@anirudh-ramesh recently submitted a PR that made changes to parse new prices from distinct sensors to the concrete model. However, I've encountered an issue where the model is currently infeasible because the power selection is not being done. I understand that the objective function with sense=minimise should take care of the price selection, but I'm not sure how to proceed with the power selection to ensure that it happens according to the price selection using the sensor dictionary that associates the power and price sensors.

Can you please provide guidance on how I can make the model feasible and ensure that the power selection happens based on the price selection using the newly added sensor dictionary parameters? I would greatly appreciate any help or advice on this matter. Thank you."

@nhoening
Copy link
Contributor Author

nhoening commented May 9, 2023

Hi @rajath-09 ― glad to help, but "the power selection is not being done" is for me not specific enough.
Maybe you can share excerpts from a log file, a screenshot or (best) reproducible steps (e.g. a script constructing and running an example), so we can see the problem first hand.
And when you say "currently", what do you mean? In the current state of this PR?

@rajath-09
Copy link

Hi @nhoening
By "the power selection is not being done" ,what i mean is that ,isn't the model supposed to be scheduled according to both the power and price sensor data provided and the association between them?For instance,if for a particular date-time frame,price from particular sensor is selected,isn't the associated power sensor be only selected?
Yes by "currently" i mean the current state of this PR.

@rajath-09
Copy link

Hi @nhoening
I wanted to understand what the "planned_costs" imply in the scheduling algorithm.
For instance if I have solar plants,load,battery and grid, what all costs does the "planned_costs" involve?

@rajath-09
Copy link

Also is there a feature which could be used to get the planned costs for individual assets?

@nhoening
Copy link
Contributor Author

nhoening commented Jun 2, 2023

Tagging @Flix6x and @victorgarcia98 here who might get you an answer quicker than I can.

@rajath-09
Copy link

Could you help me out here? @Flix6x @victorgarcia98

@victorgarcia98
Copy link
Contributor

Hello Rajath!

The planned cost per device is returned by the function device_scheduler in line 196. You can see how it is computed here. As far as I can see, these planned costs are not saved.

@Flix6x
Copy link
Contributor

Flix6x commented Jun 3, 2023

Also is there a feature which could be used to get the planned costs for individual assets?

No. This in itself would be a great feature request, imo, though. It would require something like passing a cost_sensor_id to the scheduler and using it to save the computed costs as well (for total costs). The total costs can be gotten from the model results after it solved. Costs per device are not stored on the model yet, but can be gotten by multiplying the power of each device (which is stored on the model) with the price (also stored). Saving these would require passing some mapping of devices (columns passed to the device_scheduler) to cost_sensor_ids.

If you need to compute these costs with features currently available, you might run a reporter after the scheduler has saved its results. The reporter_config should multiply each schedule with the prices. Bare in mind that reporters are kind of a new feature. A dedicated CLI command for running reporters will be released in v0.14 (it has recently become available on main already).
@victorgarcia98 can sensor_search_configs be used to select only sensor data sourced by the scheduler?

@victorgarcia98
Copy link
Contributor

@victorgarcia98 can sensor_search_configs be used to select only sensor data sourced by the scheduler?

At the moment, the sensor_search_configs can take a source id in its values so you could get that.

This could be done using the PandasReporter with a reporter_config similar to the following:

{
    "beliefs_search_configs": [
        {
            "sensor": 1,
            "source" :  1,
            "alias" : "power",
            "resolution" : "PT1H"
        },
        {
            "sensor": 2,
            "alias": "price",
            "resolution" : "PT1H"
        }
    ],
    "transformations": [
        {
            "df_input": "price",
            "method": "droplevel",
            "args": [
                [
                    1,
                    2,
                    3
                ]
            ]
        },
        {
            "df_input": "power",
            "method": "droplevel",
            "args": [
                [
                    1,
                    2,
                    3
                ]
            ]
        },
        {
            "df_output" : "energy_cost_df",
            "df_input" : "power",
            "method" : "multiply",
            "args" : [ "@price" ]
        }
    ],
    "final_df_output": "energy_cost_df"
}

@nhoening nhoening linked a pull request Jun 11, 2023 that will close this issue
2 tasks
@Flix6x Flix6x modified the milestones: 0.14.0, 0.15.0 Jun 12, 2023
@Flix6x Flix6x removed this from the 0.15.0 milestone Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants