Issues running the latest versions of Python-MIP on AWS lambda functions

[jerak]

Bug
With the two latest versions of Python-MIP (1.14.0 and 1.15.0) I seem to have issues running on AWS lambda functions. When calling the lambda function, the issue that appears is the following:

import mip.cbc\n File \"/tmp/sls-py-req/mip/cbc.py\", line 603, in <module>\n Osi_getNumCols = cbclib.Osi_getNumCols\nNameError: name 'cbclib' is not defined\n"

This issue is also reported in other places, and, as far as I understand it, the proposed solution is to add the CBC binaries manually, as described here: https://python-mip.readthedocs.io/en/latest/install.html

This solved the aforementioned error. However, this seems to be resulting in much longer solving times (sometimes more than doubling). Is this a known or explainable issue? Are there alternatives for resolving the initial problem?

Important to mention is that the reason for wanting to upgrade from 1.13.0 to 1.15.0 is that when calling the solver multiple times in a row, the solver sometimes seems to get stuck in an infinite loop (reproducing the error doesn't seem to be easy, as it seems to be machine/OS-dependent). My guess was that that issue was solved in coin-or/Cbc@e008a36

Lambda function specifications:

• OS: Amazon Linux 2 (see https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html)
• Python version: 3.9
• Thanks in advance! Feel free to ask questions if anything is missing or unclear.

[jerak]

When digging deeper into this, I noticed that the following error occurred prior to the aforementioned error:
An error occurred while loading the CBC library: cannot load library '/var/lang/lib/python3.9/site-packages/mip/libraries/cbc-c-linux-x86-64.so': /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /var/lang/lib/python3.9/site-packages/mip/libraries/cbc-c-linux-x86-64.so).

[jerak]

When following the steps for adding custom binaries, as described on this page: https://docs.python-mip.com/en/latest/install.html#using-your-own-cbc-binaries-optional, I noticed that the files are very small (e.g. ligCbc.so is 0kb, libCbc.so.0.0 is the largest file, with 2294kb. Also, I get a lot of files with similar names (all with .so, so.0, so.0.0 and .la) extensions, instead of 1 large file named libCbcSolver.so. Any ideas if there is something that I am doing wrong?

[rschwarz]

The other files are probably symbolic links to the main file.

[jerak]

Thank you, that makes sense. So that is not completely unexpected? Is the fact that adding those custom binaries results in longer runtimes to be expected? Do you by any chance have another potential direction?

Thanks a lot for your help!

[rschwarz]

No, it is standard practice to provide these symlinks.

As to longer runtimes: There may be compilation options that enable certain optimizations, having an impact on runtime.

[jerak]

Thank you, that might explain it indeed. Are you able to indicate what those options might be, so that I can try the same options on my side?

[rschwarz]

No, sorry, I'm not familiar with the building of CBC.

[tkralphs]

The symbolic links are part of the standard way that versioning is done with virtually all Linux packages. It's a mechanism by which you can drop in a new version of a library without breaking everything that depends on that library. If you do something like ls /usr/lib on a Linux box, you will see all libraries have these symbolic links installed.

The default build options should be fine in general, but there are some third-party libraries that can improve performance. Take a look at the builds on Github Actions to see what libraries are installed, e.g., https://github.com/coin-or/Cbc/blob/84625a5eae4e367d4d165669fa8e532f0c5e0dac/.github/workflows/linux-ci.yml#L24.