Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fix gh-15888 for HiGHs OOM crashes #18990

Closed
wants to merge 3 commits into from
Closed

Conversation

HaoZeke
Copy link
Contributor

@HaoZeke HaoZeke commented Jul 30, 2023

More of a dodge than a fix, but will essentially work with exceptions now.

Reference issue

Closes #15888.

What does this implement/fix?

This is an extension to the bindings for the exception handling introduced in highs here: scipy/HiGHS#61

The pybind11 bindings of highspy also support handling C++ exceptions so even after integrating directly with highspy this should not be an issue.

Additional information

N/A. I intend to also have an upstream PR on exceptions so the fork changes are minimized.

In [1]: from scipy.optimize import linprog
   ...: import numpy as np
   ...: np.random.seed(0)
   ...: n_ctr = 500000
   ...: n_var = 500
   ...: c_ = np.ones(n_var)
   ...: A_ub = np.random.random((n_ctr, n_var))
   ...: b_ub = np.zeros((n_ctr, ))
   ...: 
   ...: res = linprog(c_, A_ub=A_ub, b_ub=b_ub, method='highs-ds', bounds=(0, None), options={
   ...: 'disp': True, 'presolve': True})
   ...: 
   ...: print('End')
   ...: 
Running HiGHS 1.2.0 [date: 2021-07-09, git hash: n/a]
Copyright (c) 2022 ERGO-Code under MIT licence terms
Presolving model
Memory allocation failed while processing fixedColAtLower: Failed to resize the vector. Requested new size: -2142950640. Size to add is 24.. Current size: 2152016632.
The problem is too large to be solved. Details: The problem cannot be pre-solved, either lower the size or try without presolve
End

In [2]: 

To test this locally one has to update the submodule to point to scipy/HiGHS#61.

Draft until scipy/HiGHS#61 is completed and until a custom exception is raised and caught (instead of catching Exception in the cython binding).

Note: The build will be broken until the HiGHS PR is merged :)

@HaoZeke HaoZeke marked this pull request as ready for review July 31, 2023 02:36
@HaoZeke HaoZeke requested a review from rgommers as a code owner July 31, 2023 02:36
@j-bowhay j-bowhay added the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Jul 31, 2023
@rgommers
Copy link
Member

Thanks @HaoZeke. I had a quick look at both PRs, that seems like a concise and pragmatic solution for now.

Given that this PR is broken until the other PR is merge, let me mark it as draft for the time being.

@rgommers rgommers marked this pull request as draft July 31, 2023 08:01
@HaoZeke
Copy link
Contributor Author

HaoZeke commented Oct 21, 2023

After ERGO-Code/HiGHS#1467 goes in and scipy/HiGHS#63 is updated for #19255, this will not be required (highspy catches and reports C++ exceptions nicely anyway).

@HaoZeke HaoZeke closed this Oct 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.optimize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: linprog, with highs is killed by the OOM killer when called with dense and large matrices
3 participants