Extension of the C API

[rschwarz]

While working on a wrapper for HiGHS, I felt like missing out on some functionality that is supported in principle, but not currently exposed through the C API.

For instance:

• It's not possible to set or retrieve names for variables or constraints, but such data seems to be stored with HighsLp. Not sure if it's possible to find row/column indices by name in general.
• The C API has changeColIntegrality, but there is no corresponding getColIntegrality. Even Highs_getColsByRange etc fetch the coefficients, bounds and objective cost, but not the variable type.

I also needed to query specific attributes of rows and columns (eg just upper bound) which is a bit awkward because when using, eg Highs_getColsByRange, a handful for arrays need be allocated, but this is just a minor inconvenience.

[jajhall]

The names issue requires coding in C corresponding to vectors of strings in C++. I'll have a go, but may need help as I'm not a C programmer (particularly when it comes to char!). I'm not 100% happy about creating a setNames, as HiGHS doesn't need names internally, and users will have the names corresponding to indices. I guess it allows folk to create more meaningful solution files. If the model has been read from a file, then I can see why getNames will be valuable.

I can add a getColIntegrality. I don't want to introduce integrality into Highs::getCols, as it won't be relevant to people solving continuous problems.

[rschwarz]

I agree that storage of names is not critical (although it also improves debugging when writing to .lp files), but I thought it would be easy to add since it looked already implemented on the C++ side.

It also makes sense not the change getCols as that would be breaking.

[jajhall]

Indeed, creating a .lp or .mps file with "helpful" names after a model's been created without a file is another good reason.

[jajhall]

I've started trying to write something to get and pass names in C++ - without using std::vector:std::string> because I know that I'm going to have to interact with C.

I can just about understand the difference between char char* and char**, but I'm floundering when trying to wok with arrays of such things.

In C

• How would you define the array of names that you want to pass through the HiGHS C API as an argument to Highs_passNames to be used to set the names for the incumbent model in HiGHS?
• How would you define the array of names to be received from the HiGHS C API as an argument to Highs_getNames to be used to extract the names in the incumbent model in HiGHS?

[rschwarz]

Another possibility is to ask the user to provide a char** (array) preallocated to the number of names, then let the getName method fill it with pointers to char* (C string) that HiGHS has allocated (eg as copies of s.c_str()?) and make it clear that the caller has ownership and needs to free?

[jajhall]

I'd wondered about this approach, and tried to create such a char** array, but failed to populate it with names. Here's my attempt

char** col_names = (char**)malloc(sizeof(char**) * lp.num_col_);
for (HighsInt iCol = 0; iCol < lp.num_col_; iCol++) {
std::stringstream ss;
ss.str(std::string());
ss << model << iCol << "\0";
const std::string name = ss.str();
// col_names[iCol] = name.c_str();
printf("Col %d name is to be %s\n", int(iCol), name.c_str());

}

[jajhall]

Maybe I need something (I'm not used to working with) like strcpy since I suspect that the string created in name will be destroyed each time through the loop

[rschwarz]

I don't see an issue, but my C++ is a bit rusty. I can give this a try some other time.

[jajhall]

I'll create based on get/setName, as this will be easier for some users, and get/setNames can be done later

[jajhall]

I've added Highs_pass/getCol/RowName and Highs_getColIntegrality to the C API.

I've even managed to test them in check/TestCAPI.c

#1168 will merged this work into latest

Do these work for you @rschwarz ?

[jajhall]

So, I can't get the C correct in TestCAPI.c to test the new code. If the loop in lines 446-456 (where I construct "consecutive" column and row names) is executed (see 73a755b) then 8 of the CI tests fail because capi_unit_tests times out, despite running fine with g++ on Linux in debug.

@rschwarz: Can you see what in TestCAPI.c might cause this?

Perhaps there's something wrong in the coding of highs_c_api.cpp, but it uses C++ that is very close to code that runs fine with other tests in TestCAPI.c

[rschwarz]

I was able to compile and run the tests on my (linux) machine without problem, but so did 2/3s of the runs in the CI, right?

If I now enable the full_api_names block in the test, it still passes, but running with valgrind gives some warnings:

==12900== Source and destination overlap in strncat(0x1ffefffdcb, 0x1ffefffdcf, 1)
==12900==    at 0x483BBFC: strncat (vg_replace_strmem.c:349)
==12900==    by 0x10C123: full_api (TestCAPI.c:452)
==12900==    by 0x10F39B: main (TestCAPI.c:1234)
==12900==

Rather than attempting to understand what is the issue here exactly (I think we need char name[5]; instead, to account for the 0 terminator), I replaced the code with something based on sprintf, which doesn't give any remaining valgrind warnings in this test.

The printed messages also look fine:

Column 0 has name Col0
Column 1 has name Col1
Row    0 has name Row0
Row    1 has name Row1

But: when I insert a call to Highs_writeModel(highs, "foo.lp");, I see the following output, not using the provided names:

\ File written by HiGHS .lp file handler
min
 obj: +1 x1 -2 x2 
st
 con1lo: +1 x1 +2 x2 >= +0
 con1up: +1 x1 +2 x2 <= +2
 con2lo: +1 x1 +3 x2 >= +0
 con2up: +1 x1 +3 x2 <= +1
bounds
 x1 <= 10
 x2 <= 10
end

See branch https://github.com/rschwarz/HiGHS/commits/rs/capi_names, and in particular commit rschwarz@798012b

[jajhall]

Thanks for all this: very helpful. I'll look at it later

[rschwarz]

Unfortunately, there still seem to be timeouts in CI.

[jajhall]

It just needed char name[5]; for the tests of Highs_getColName and Highs_getRowName. Branch now merged into latest. Thanks @rschwarz

[Thell]

I did a search and didn't find anything related to col/row names other than this discussion and another search didn't bring up getColName in any code but reading through this I thought I'd give a use case where a getColByName would be nice.

This is a snippet of a Python function using pywraplp with the CBC backend where I benefit from not having to fuss with tracking and passing around names or indices which allows for the model solving to be extracted from the building logic.

def subset_selection(items, item_reqs, weights, state_1_values, state_2_values,
                     state_1_sum_values_lb, state_2_sum_values_lb):
    solver = subset_solver(items, item_reqs, weights, state_1_values, state_2_values)

    state_1_constraint = solver.LookupConstraint("state_1_lb")
    state_2_constraint = solver.LookupConstraint("state_2_lb")
    state_1_constraint.SetBounds(state_1_sum_values_lb, solver.infinity())
    state_2_constraint.SetBounds(state_2_sum_values_lb, solver.infinity())

    result_status = solver.Solve()
    if result_status == pywraplp.Solver.OPTIMAL:
        return extract_solution(solver, items, weights, state_1_values, state_2_values)
    return (None, None, None, None, None)

the _par version of this is

def subset_selection_par(items, item_reqs, weights, state_1_values, state_2_values,
                         state_values_sum_lb_pairs):
    solutions = []
    solver = subset_solver(items, item_reqs, weights, state_1_values, state_2_values)

    state_1_constraint = solver.LookupConstraint("state_1_lb")
    state_2_constraint = solver.LookupConstraint("state_2_lb")
    for state_1_sum_values_lb, state_2_sum_values_lb in state_values_sum_lb_pairs:
        state_1_constraint.SetBounds(state_1_sum_values_lb, solver.infinity())
        state_2_constraint.SetBounds(state_2_sum_values_lb, solver.infinity())
        result_status = solver.Solve()

        if result_status != pywraplp.Solver.OPTIMAL:
            continue
        solutions.append(extract_solution(solver, items, weights, state_1_values, state_2_values))

    return solutions

They are each used to run the model over a variety of sets of data where the solutions for a variety of bound params are used for each dataset.

This is something readily solvable on the user's end with a dictionary/hashmap of Name: Col but then that has to be built and passed around.

[jajhall]

getColName is in the latest branch, but I can see that you want getColByName. I also see that Gurobi offers this functionality. It's not expensive to have it, so I'll add it.

[jajhall]

This is now written, will soon be pushed to latest, and will make it into the next release.