enable usage of Highs solver

[victorgarcia98]

Description

This PR adds the possibility to use the solver Highs in FlexMeasures. To do so, you need to set the configuration variable FLEXMEASURES_LP_SOLVER to appsi_highs.

In addition, I've run a benchmark to compare Highs agains our current solver by default, CBC. The benchmark is running the following command 10 times:

 flexmeasures add schedule for-storage --sensor-id 7 --consumption-price-sensor 2 --inflexible-device-sensor 3 --start ${TOMORROW}T00:00+01:00 --duration ${DURATION} --soc-at-start 0% --roundtrip-efficiency 90%

The benchmark uses the data from the Toy Tutorial Part II, repeating the pattern for the duration of the tests (12D).

Results

solver	duration	time
cbc	P12D	2.795 ± 0.07
cbc	P3D	0.269 ± 0.01
cbc	P6D	0.797 ± 0.01
highs	P12D	12.605 ± 0.11
highs	P3D	0.908 ± 0.01
highs	P6D	3.196 ± 0.03

Profile of computing a schedule with Highs (duration=12D)

Profile of computing a schedule with CBC (duration=12D)

Comment

Although the performance is worse in practice, Highs is being used in other projects (without Pyomo) providing interesting improvements:

• PyPSA: using linopy, in this post, they show Highs to be comparable to commercial solvers such as Gurobi.
• GridCal: using PuLP

Moreover, looking at the profile reports, we can see that Pyomo is consuming most of the time building the problem using Highs as it's using a LegacySolver, which turns out to be pretty slow. The actual solving time is taking a small fraction of the total computation (10%).

Further Improvements

Test with larger/more complex problems and wait for Pyomo to improve the creation of the model.

Related Items

#614

---

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

[nhoening]

Definitely deserves a changelog entry.

Maybe even a blog post on flexmeasures.io with these findings above?

Of course, a larger problem would be great, but do we have one available, e.g. in simulations? We can also wait for the next project which has one.

[Flix6x]

Does it make sense to ship FM with both solvers installed? (Not sure about their sizes.) Or should this become an installation choice at some point?

[Flix6x]

Very nice @victorgarcia98, thank you for checking this solver out!

("HiGHS" is the official typography it seems.)

[Flix6x]

[Columbo] Just one more thing: did you run our test suite against HiGHS?

[victorgarcia98]

Good point, thanks for reminding me to run the test suit.

The test flexmeasures/data/models/planning/tests/test_solver.py are failling with the following error stacktrace:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/victor/Work/Seita/flexmeasures/venv/lib/python3.10/site-packages/pyomo/contrib/appsi/base.py", line 1554, in solve
    results: Results = super(LegacySolverInterface, self).solve(model)
  File "/home/victor/Work/Seita/flexmeasures/venv/lib/python3.10/site-packages/pyomo/contrib/appsi/solvers/highs.py", line 250, in solve
    res = self._solve(timer)
  File "/home/victor/Work/Seita/flexmeasures/venv/lib/python3.10/site-packages/pyomo/contrib/appsi/solvers/highs.py", line 234, in _solve
    return self._postsolve(timer)
  File "/home/victor/Work/Seita/flexmeasures/venv/lib/python3.10/site-packages/pyomo/contrib/appsi/solvers/highs.py", line 657, in _postsolve
    raise RuntimeError(
RuntimeError: A feasible solution was not found, so no solution can be loaded.Please set opt.config.load_solution=False and check results.termination_condition and results.best_feasible_objective before loading a solution.

I tried to do opt.config.load_solution=False, but the exception still happens :S.

A temporal (and dirty) workaround would be to wrap the solve step in a try ... catch block, return None for the results and handle this properly at the StorageScheduler level. We could elevate this issue at the Pyomo level, but I don't find immediate benefit from doing so (it's going to take time to create a minimum example and following up the fix).

Given the status of the implementation and proven there's no apparent speed up (results above), I would consider disregarding the inclusion of this PR into FM.

[victorgarcia98]

Given the status of the implementation and proven there's no apparent speed up (results above), I would consider disregarding the inclusion of this PR into FM.

I retract from this as the benchmark that @Flix6x run show very interesting results, we should probably pursue using HiGHS.

[victorgarcia98]

Just created an issue in the PYOMO repo to ask for help.

[nhoening]

I like where this is going :)

[nhoening]

just a few textual changes