Recommended solver presets

[SteveDiamond]

Quoting from @knicklux on discord:
on that topic, I was thinking that some presets in CVXPY. Like cp.SOLVER_AUTO.{LARGE_SCLAE, FAST, ACCURATE, MULTIPRECISION} would make life a lot easier in cases, where one wants to run the solver again with a higher accuracy or switch from a second-order to a first-order method because of memory issues
For example, here is a preset with solver options for SDPA-multiprecision:

settings_sdpa = {
    "gammaStar": 0.5,
    "lambdaStar": 1e5,
    "lowerBound": -1e10,
    "upperBound": 1e10,
    "maxIteration": 200,
    "warm_start": True,
}

[knicklux]

For semidefinite programs:

FAST: solver CLARABEL with default options
LARGE_SCALE: solver SCS with default options
LARGE_SCALE_ACCURATE: SCS with

    options_scs = {
        "max_iters": 1000000,
        "verbose": True,
        "use_indirect": True,
        "eps": 1e-8,
        "eps_abs": 1e-8,
        "eps_rel": 1e-8,
        "eps_infeas": 1e-8,
        "acceleration_lookback": 0,
    }

ACCURATE: solver CVXOPT with options

    options_cvxopt = {
        "max_iters": 5000,
        "verbose": True,
        "abstol": 1e-8,
        "reltol": 1e-8,
    }

ROBUST: solver CVXOPT with options

    options_cvxopt = {
        "max_iters": 5000,
        "verbose": True,
        "abstol": 1e-21,
        "reltol": 1e-11,
        "refinement": 2, # higher number seemed to make things worse
        "kktsolver": "robust"
    }

MULTIPRECISION: solver SDPA-multiprecision with options

settings_sdpa = {
    "epsilonStar": 1e-30, # check, if this is always a good idea
    "epsilonDash": 1e-30, # # check, if this is always a good idea
    "gammaStar": 0.5,
    "lambdaStar": 1e5,
    "lowerBound": -1e10,
    "upperBound": 1e10,
    "maxIteration": 200
}

@SteveDiamond is CLARABEL feature-complete or going to be feature-complete soon?

[SteveDiamond]

What do you mean by feature complete? Clarabel is actively developed, so there are continuous improvements.

[knicklux]

I get an error when minimizing log_det with CLARABEL:

thread '<unnamed>' panicked at 'not implemented: Mixed PSD and Exponential/Power cones are not yet supported', src/solver/core/cones/psdtrianglecone.rs:292:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

so maybe, for now we stick with the default (which would be SCS with default settings) or wrap the call to CLARABEL in a try-catch

[SteveDiamond]

I get an error when minimizing log_det with CLARABEL:

thread '<unnamed>' panicked at 'not implemented: Mixed PSD and Exponential/Power cones are not yet supported', src/solver/core/cones/psdtrianglecone.rs:292:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

so maybe, for now we stick with the default (which would be SCS with default settings) or wrap the call to CLARABEL in a try-catch

Ah ok. Yeah this combination is not fully supported yet. @goulart-paul what's the current time frame for adding more support for PSD + exponential/power?

[rileyjmurray]

Clarabel’s log_det issue will be fixed in the near-ish future. I’ve spoken with Paul about this. Im very wary about having qualitative keywords influence solver dispatch.

…

On Mon, Jan 15, 2024 at 12:44 PM Nicolas Kessler ***@***.***> wrote: I get an error when minimizing log_det with CLARABEL: thread '<unnamed>' panicked at 'not implemented: Mixed PSD and Exponential/Power cones are not yet supported', src/solver/core/cones/psdtrianglecone.rs:292:9 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace so maybe, for now we stick with the default (which would be SCS with default settings) or wrap the call to CLARABEL in a try-catch — Reply to this email directly, view it on GitHub <#2324 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRLIFEJONC2FL5WC6ZPQH3YOVTJJAVCNFSM6AAAAABB3SHBNCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJSGU3TKOJZGU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[phschiele]

In principle, I think such qualitative keywords would be a nice addition. It could be great for users who have issues with the results of the 0 argument form of .solve() but don't want to get into all of the details of the solver options.

However, I also have some doubts about hard coding specific configurations of solver options into CVXPY, as it also creates some maintenance overhead. Also, I think we would need to have at least some process for how we determine such recommended settings, as they may very well depend on the problem domain.

Going in the direction that @rileyjmurray mentioned, I think we should distinguish between solver settings given the solver, and solver dispatch. We could have, per-solver groupings of settings, similar to the unified eps keyword recently introduced for the Mosek interface. For example, Clarabel has Full Accuracy and Reduced Accuracy parameters, although AFAIK they are currently not exposed to the user: Docs.

We could improve the information content of this table, e.g., by adding what type of algorithm is used:
https://www.cvxpy.org/tutorial/advanced/index.html#choosing-a-solver
We can then add some high-level guidance about accuracy vs. speed and memory usage, and which are the most common parameters to tweak.

TL;DR: A first approach could be to improve docs to guide solver choice + add solver-specific default groups.

[knicklux]

First, thank you for letting me know on the state of CLARABEL :)

Right now, the issue at core is that numerous users are heading to the discord to ask for solver settings, especially for SDPs. They all would benefit massively from either improving the documentation with tuning suggestions or providing solver presets; I agree with @phschiele that the first is more practical in the long run and I could help by providing tuning suggestions for SDPs and QPs. Is there already an open issue to add this to the documentation?

About groupings of settings, it could make the definition of automatic settings easier. And thinking about issue #434 , it would be tremendously useful, if the resulting metrics of infeasibility in that check are less than the user-specified epsilon. I was recently revisiting snapvx and thinking about issues in a distributed implementation and one major issue is that a solver may return a wrong solution, which could be avoided by having the workers (or a subset of trusted workers) verify each other's solution.

Maybe, a better wording would be cp.SOLVER_PRESET.{...} instead of auto, that would make it clearer to the user that the settings are hard-coded and may or may not work. Like @phschiele pointed out, that there should be a process to test the presets and maybe it would be a good idea, to implement those first and then revisit the issue of providing presets.