Updates to the CVXPY wishLists/projects ideas

[Transurgeon]

In light of an upcoming release centered around updates in the documentation, I would like to ask CVXPY maintainers and developers to chime in on the wishlist located here.

What do you think can be added/removed? Instead of linking to existing issues maybe we can give a summary of the project in the docs (doesn't need to be for all projects of course)?

The following are my very initial thoughts on this subject. Any objections/feedback is certainly welcome!
Small scope projects

• State required cone types for atoms [1]
• Specify a list of solvers sorted by priority rather than just a single solver [2]

[1]: I think there were plans to add something similar to this in the atomic functions table.
[2]: The complication that Riley pointed out still remains. There was a similar recent issue on adding solver presets, which could help towards the completion of this project.

Medium scope projects

• Support integer variables in geometric programming [3]
• Add an interface to COSMO.jl [4]
• Support more open-source interior-point SDP solvers [5]
• Post-solver feasibility checks
• Citation generator [6]
• Developer documentation for key reduction files [7]

[3]: The issue was closed, but it seems like the feature still isn't supported. If this is not really high in the priority, we could potentially remove it. More details on where to get started could be really helpful :)
[4]: A while ago, I opened an issue to have cosmo-python added to pip but there was no response. If there are no objections, I think we could remove this project from the list.
[5]: I think Usama helped with support for SDPA some time ago. Maybe this project could be removed?
[6]: This seems really cool, but I don't feel like it's that important. Maybe could also remove.
[7]: I personally would really like to further understand the reduction steps of CVXPY. Riley's linked PR shows some good progress on the documentation. Maybe we could update this project with remaining reductions to document?

Large scope projects

• Expand use of power cone constraints [8]
• Problem serialization
• Pretty print method for summarizing a solution and its dual variables [9]
• Support for n-dimensional expressions, variables, parameters, etc [10]
• 10x improvement in the speed and memory usage of cvxcore [11]
• Full compatibility with NumPy broadcasting rules. This will be a breaking change, and can only go in CVXPY 2.0 [12]

[8]: I saw some discussion about this at some places, but I don't think it has been tackled yet.
[9]: This could be nice, but more details would definitely be needed. What exactly does a pretty-print summary look like in this case. How does it differ from the current verbose output?
[10] & [12]: We can maybe add those to a new section on CVXPY 2.0 and try to give a lot more details/examples on the expected behaviour. Also some guidelines on where to look and how to get started could be tremendously helpful.
[11]: I think a lot of progress was made on this with the SciPy backend by Philipp and further improvements whom I contributed also ;). I propose that we replace this project with an updated one that can consist of the GRAPHBLAS backend (close to being done) along with the RUST backend (also WiP). The former will be an optional dependency while the latter could possibly replace the CPP backend (although it will take a lot of effort to ensure a smooth transition). I don't mind separating the two backends into separate issues also.

[SteveDiamond]

A few more:

• propagate variable bounds to solvers
• wrap up gradients with pytorch PR
• Match CVX in terms of variable types/attributes and allow variables to have multiple types/attributes. Same goes for parameters.
• QCQP support
• IPOPT interface to add non-linear non-convex support
• Vectorized quad_over_lin (https://ask.cvxr.com/t/proper-use-of-quad-over-lin/11691/4?u=nikku). This is much more important than it sounds because it is a good building block for many use cases. But it is also much more work than it sounds like because it affects the QP processing, which is a nasty mess.

[Transurgeon]

Thank you Steven! I also had a few comments.

• I would love to know more about the multiple attributes feature for variables, it looks really interesting (but difficult also).
  Would this feature bring a lot of changes to the design of the AST tree that cvxpy generates? Where should one start looking into to understand the current types/attributes functionality of variables?
• For the variable bounds, what happens if a solver doesn't take them as parameters? Do we need to select which solvers support that feature before passing the bounds?
• For QCQP support, I saw that there was another package also from cvxgroup, would the implementation look similar to the code over there? Also is the plan to have QCQP as its own reduction with its own set of solvers?

[SteveDiamond]

• For the multiple attributes, it wouldn't change much. Right now we only allow one attribute per variable because after the problem is solved we project onto the variable constraints. That projection is too complicated in the fully general case with arbitrarily many attributes. A simple solution is to keep doing the projection but only if the variable has one attribute, or a reasonable subset. This project is actually fairly easy compared with the others. Happy to explain more if you're interested.
• There is a flag now for solvers that can take in variable bounds (see the proxqp interface). So this is more about integrating the variable bounds info with more solvers.
• I'm not sure what the best approach is. The NAG, Gurobi, and CPLEX solvers can all take QCQPs. It could either be a new reduction or augmenting the main conic reduction, which already supports quadratic objectives. Technically the QP reduction pathway is totally unnecessary now but I was afraid to remove it in case it impacted performance.

[PTNobel]

On the easy to implement side:

• Named constants
• Implement cp.vdot as cp.scalar_multipy alias (or whatever it should alias) and deprecate the old one
• Sparse variables used to make constraints

Complicated projects:

• cp.trace(A @ B) transforms into cp.vdot(A, B)
• Sparse variables reduce problem dimension

[SteveDiamond]

Named constants is implemented

[PTNobel]

So it is; I had missed that PR

[Transurgeon]

Hey Parth, thanks for letting us know about these features. I have a few comments below.

• for the cp.vdot to replace cp.scalar_product, maybe we can add the "good first issue" tag to this issue?
  also just to make sure.. to complete this would we simply need to replace all instances of cp.scalar_multiply with cp.vdot in the codebase?
• for the trace being transformed into vdot, why exactly does this make the computation faster? Could you provide some math behind it? Also how would this feature look like internally? (im curious why you consider this a complicated project since I am not too familiar with these atom implementations)
• I will definitely add the sparse variables stuff! I saw that Steven mentioned to look into leaf.py and cvx_attr2constr.

[PTNobel]

I will definitely add the sparse variables stuff! I saw that Steven mentioned to look into leaf.py and cvx_attr2constr.

Please coordinate with @griffinbholt who is also interested in working on the sparse stuff! I can do an intro on the Discord.

for the cp.vdot to replace cp.scalar_product, maybe we can add the "good first issue" tag to this #2336?
also just to make sure.. to complete this would we simply need to replace all instances of cp.scalar_multiply with cp.vdot in the codebase?

More or less! And update the relevant documentation...

for the trace being transformed into vdot, why exactly does this make the computation faster? Could you provide some math behind it? Also how would this feature look like internally? (im curious why you consider this a complicated project since I am not too familiar with these atom implementations)

When CVXPY is canonicalizing Tr(A @ B), it parses this as trace(matmul(A, B)) and then writes this to Z = matmul(A, B) and then trace(Z), this introduces an extra $n^2$ variables and constraints to the problem, and most notably requires computing $n^2 - n$ values that aren't useful at all.

How would we implement it isn't totally obvious to me, we could do something similar to the magic for x.T @ A @ x to quad_form transformation where we just check for a special case in the syntax tree, or maybe we could do something fancier (but that might end up becoming a much bigger presolver project!)

[PTNobel]

Option for using QDLDL instead of Eigen for sparse Cholesky

[rileyjmurray]

Sure; that’d be fine with me.

…

On Mon, Feb 26, 2024 at 6:01 PM Parth Nobel ***@***.***> wrote: Option for using QDLDL instead of Eigen for sparse Cholesky — Reply to this email directly, view it on GitHub <#2359 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRLIFGQQWKGSAYKQQEL6JDYVUH4TAVCNFSM6AAAAABDSKIDJOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKOJYGUZDM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[SteveDiamond]

Another important one came up today: DPP support for problems with complex numbers