-
Notifications
You must be signed in to change notification settings - Fork 255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Matrix instance where UMFPACK 64 routines fails to factorize #577
Comments
Thanks for the detailed reproducible code. I was able to run it here. I'll upload my files as a PR, if you like. The matrix can be factorized, but when used as is stands, the matrix takes about 151 seconds on my 20-core desktop. It suffers a lot of fillin. I tried pulling the matrix into MATLAB (which also uses umfpack), and I noticed something interesting. The matrix has 530,379 entries but only 448,277 nonzeros. That is, 82,102 of the entries are exactly zero. Normally that's not a problem, but I do treat those as legitimate entries (it can happen that subsequent matrices need those entries). But when I tried it in MATLAB, the factorization took just 2 seconds, and L+U has just 19.6 million entries. MATLAB drops zeros by design. If I add the zeros with values equal to epsilon (1e-16), the factorization takes 258 seconds in MATLAB (an older version of UMFPACK perhaps explains that difference, or a different BLAS library), and the factorization L+U has 1 billion entries. I think the epsilon values I added are causing trouble there. I'm not factorizing the same matrix in MATLAB as in C, in this case. I don't see as many entries in L+U from the C code (where zeros are kept but equal to zero), but I still see a lot: about 90 million, and the time is 151 seconds. So I would consider dropping the explicit zeros from A, if you can do that. |
I'll try that and let you know what happens. Thanks. |
Here's my output with your original matrix: that's my output, with some added diagnostics, with this program: loadUMFPACK_long_version.c.txt and a build script: my MATLAB script: and the output: and its output: My desktop has 256GB of RAM, so it was able to handle this matrix. |
Hi, after dropping the zeros. It is able to factorize with just 32-bit interface on my 16GB RAM machine!
Maybe HiGHS has done something for the zero elements in the matrix, I will probably look more into that when I have time. Thanks for the detailed explanation! |
Glad to help! |
Can you tell me more about this matrix? It could make for an interesting problem to add to my collection, as a counter-example. |
Sure. I was doing work on adding new basis packages to the PATH solver, which is a specialized solver for the mixed complementarity problem. The basis packages are used to calculate the basic feasible direction for pivoting since the PATH solver adopts a method similar to the simplex method. When I was testing all the available packages in PATH with some problem sets, a problem failed (a dynamic equilibrium problem). The failed matrix I extracted corresponds to the basic feasible direction, The reason why it has zeros in the data is exactly what you said - it can happen that subsequent matrices need those entries (we indeed reuse the matrix structure). Essentially, we are doing things similar to extracting the columns of Jacobian of a nonlinear map to form a basis. When we are identifying the number of non-zero elements, we are identifying the entries that are "structurally nonzero." So an entry that has value |
Describe the bug
I encountered an instance of matrix where UMFPACK 32-bit routine is unable to factorize it because of out-of-memory error. I then tried to use the UMFPACK 64-bit routine but it get stuck at the _numeric stage. I am able to factorize this matrix using other basis routines such as the ones provided by HiGHS. Not sure if it’s a bug, but I hope reporting this is helpful for you.
To Reproduce
The following are the steps I do to reproduce the error.
The binary file that contains the matrix is failed_matrix.dat. All the files mentioned here can be found in:
https://github.com/WalterLu3/umfpack_edge_case
Step 1. Reproducing Out-Of-Memory error with UMFPACK 32-bit interface.
I compiled this by dynamically linking to libumfpack.dylib.
It gives me an out-of-memory error. According to the UMFPACK user guide, I proceeded to use UMPACK 64 routines.
Step 2. Reproducing error with UMFPACK 64-bit never ending.
I also compiled the following by dynamically linking to libumfpack.dylib.
The numeric steps seem to never stops.
Expected behavior
I expected the numeric step would end for the UMFPACK 64bit interface.
Desktop (please complete the following information):
Thanks.
The text was updated successfully, but these errors were encountered: