What is an interface / wrapper?

[zaikunzhang]

Consider a package that is implemented in C or Fortran. Take PRIMA as an example. As of September 2023, it is only implemented in modern Fortran. To make PRIMA available in other languages, we need to provide interfaces (or wrappers; I will not distinguish the two words in the sequel).

What is an interface?

In my opinion, there are two different types of interfaces, namely the basic interface and the advanced one, which will be detailed below.

Let us still take PRIMA as an example. It provides the following Fortran subroutines for solving nonlinear optimization problems without using derivatives.

subroutine uobyqa(calfun,  x, f, rhobeg, rhoend, ftarget, maxfun)  ! For unconstrained problems
subroutine newuoa(calfun,  x, f, rhobeg, rhoend, ftarget, maxfun)  ! For unconstrained problems
subroutine bobyqa(calfun,  x, f, xl, xu, rhobeg, rhoend, ftarget, maxfun)  ! For bound-constrained problems
subroutine lincoa(calfun,  x, f, cstrv, Aineq, bineq, Aeq, beq, xl, xu, rhobeg, rhoend, ftarget, maxfun)  ! For linearly constrained problems
subroutine cobyla(calcfc, x, f, cstrv, nlconstr, Aineq, bineq, Aeq, beq, xl, xu, rhobeg, rhoend, ftarget, maxfun)  ! For nonlinearly constrained problems

Here,

• calfun is a callback function that evaluates the objective function (calcfc evaluates the objective function and nonlinear constraints);
• x0 is the starting point;
• Aineq and bineq define the linear inequality constraints;
• Aeq and beq define the linear equality constraints;
• xl and xu define that bound constraints;
• rhobeg, rhoend, ftarget, and maxfun are some parameters needed by the solvers.

Note that I have deliberately removed some arguments in the signature to make things simpler.

Suppose that we want to make these solvers available in MATLAB. So a MATLAB interface is needed.

Basic interface

The basic interface makes the above-mentioned solvers callable in MATLAB, providing the following MATLAB functions.

[x, f] = uobyqa(calfun, x0, rhobeg, rhoend, ftarget, maxfun)
[x, f] = newuoa(calfun, x0, rhobeg, rhoend, ftarget, maxfun)
[x, f] = bobyqa(calfun, x0, xl, xu, rhobeg, rhoend, ftarget, maxfun)
[x, f, cstrv] = lincoa(calfun, x0, Aineq, bineq, Aeq, beq, xl, xu, rhobeg, rhoend, ftarget, maxfun)
[x, f, cstrv, nlconstr] = cobyla(calcfc, x0, Aineq, bineq, Aeq, beq, xl, xu, rhobeg, rhoend, ftarget, maxfun)

This is done by compiling the Fortran code using MEX provided by MathWorks.

Advanced interface

The basic interface is already nice. Users need to do the following when they want to use PRIMA to solve their problems.

1. Describe the problem, defining calfun or calcfc, x0, xl, xu, Aineq, bineq, Aeq, beq, which ever applicable.
2. Select a solver to use.
3. Set the parameters needed by the solver, namely rhobeg, rhoend, ftarget, and maxfun.

This seems not too difficult. However, in practice, things may not be so simple. Here are some examples.

1. The user may not know these solvers well. He may not have a good idea about which solver to choose. For example, which one to choose for an unconstrained problem? Note that both newuoa and uobyqa can solve such a problem; indeed, the other three solvers can do it as well.
2. The users may not know what the parameters rhobeg, rhoend, ftarget, and maxfun represent and how to set them. Note that I have already simplified the problem here. In reality, the solvers have many more parameters --- cobyla has more than 20. It will be challenging and tedious to set all the parameters properly.
3. The user's input may not be valid. Should we simply let the solver take the invalid input and behave improperly, or should we blame the user that the input is invalid and terminate? Shouldn't we do something more considerate?

The advanced interface is another layer between the basic interface and the users, handling all these problems in a user-friendly way.

Take the MATLAB interface of PRIMA as an example. The interface wraps everything under a single MATLAB function prima. This function has the following features.

1. Standard and user-friendly signature. prima has the same signature as the MATLAB function fmincon from MathWorks, so users do not need to learn how to use prima. Basically, they just need to change fmincon to prima in their existing MATLAB code. That's all.
2. Automatic solver selection. prima selects the solver according to the problem unless the user specifies the solver by setting the Algorithm option. If the solver specified by the user is invalid or cannot solve the problem, a warning will be raised and a solver will be selected.
3. Default parameters. prima defaults the parameters to values that work well in general except for those set by the user. If the user sets a parameter improperly, then a warning will be raised and the default value will be taken.
4. Preprocessing. prima preprocesses the inputs from the user before sending them to the solver. For example, lincoa expects x0 to be feasible. If the user provides an infeasible starting point, then prima will project x0 to the feasible region before passing it to lincoa. Otherwise, lincoa will not behave properly. Indeed, points 2 and 3 above can also be regarded as part of the preprocessing.
5. Postprocessing. prima postprocesses the outputs from the solver before returning them to the user. An important aspect of the postprocessing is to validate that the outputs are reasonable --- for example, the number of function evaluations is not more than maxfun (the maximum number of function evaluations allowed). This is done only in the debugging mode, which is off by default and can be turned on by setting the debug option to true. If ever some outputs are invalid, an error will be raised in the debug mode, so that the user will be informed and may inform the developer.

In the MATLAB interface of PRIMA, points 2, 3, and 4 are handled by preprima, while point 5 is done by postprima. prima calls preprima before solving the problem, and calls postprima afterward.

With an advanced interface like prima, users only need to focus on the definition of the optimization problem, at which they should be experts. The setup of the solver needs minimum care.

What should we do in other languages (Python, Julia, R, ...)?

Ideally, we should provide the advanced interface elaborated above.

However, this may take a bit of time, since many details need to be handled. Instead of doing things in one shot, we should first produce the basic interface, and then work out the advanced one gradually. In some languages (Julia, for example), the basic interface can be generated automatically. The advanced one, however, does need a human being to handcraft with some care and patience.

The basic interface can also be used to integrate our solvers into existing libraries, e.g., SciPy in Python and Optimization.jl in Julia. In this scenario, these libraries will likely handle the inputs and outputs in a systematic and mature way. So we only need to code a thin wrapper between the basic interface and the library to make them understand each other. This is much easier to produce than a user-friendly interface for humans.

[emmt]

I think that, as dicussed in #62 (comment), the Julia interface to the algorithms of the PRIMA library implemented in PRIMA.jl is quite similar to the basic interface that you propose here perhaps with different default settings.

Thanks to the expressiveness of Julia, I do not see the needs (for now) of a higher level interface. Except maybe the single optimizer which automatically choose the best method considering the contraints (as what you have done for Python). The only difficulty I can see is to check the signature of the user-defined function (which is different for COBYLA)/

[zaikunzhang]

the only difficulty I can see is to check the signature of the user-defined function (which is different for COBYLA)

In the MATLAB interface, the nonlinear constraints are passed with an argument nonlcon separate from the objective function f.

That said, a standard call of cobyla would look like

cobyla(f, x0, Aineq, bineq, Aeq, beq, lb, ub, nonlcon, options)

which is exactly the same as the signature of fmincon provided by MathWorks.

Indeed, the signature of prima is the same as cobyla.

I suppose this is the natural way of interfacing the nonlinear constraints in Julia, MATLAB, Python, R ... By doing so, we are moving towards the "Advanced Interface" specified above. As mentioned, there is no need to do it in one shot. As long as we make the interface more user-friendly and more "native", we are getting closer the the "Advanced Interface".

[zaikunzhang]

Thanks to the expressiveness of Julia, I do not see the needs (for now) of a higher level interface

This is interesting. Could you elaborate on it a bit more?

For example, why the preprocessing and postprocessing mentioned above are not necessary?

The preprocessing is basically to interpret, validate, and clean up the inputs from the user and then send them to the selected solver, the postprocessing is to validate and decorate the outputs and then return them to the user. See the MATLAB interface for example:

https://github.com/libprima/prima/blob/main/matlab/interfaces/private/preprima.m
https://github.com/libprima/prima/blob/main/matlab/interfaces/private/postprima.m

I cannot see how another language can make such procedures completely unnecessary.

Thanks.

[emmt]

Thank you for your comments.

My point was just that the current Julia interface to the different PRIMA algorithms is sufficiently simple/readable (all optional arguments prodived by keywords) that it may be sufficient for users, like me, who want to exactly know which algorithm is called. At least this (not so) low-level interface can be used to replace Powell's methods in OptimPack and OptimPackNextGen.

Otherwise, I agree that the higher level interface is very helpful to make good algorithm choices for the end-users (e.g. considering the constraints of the problem), to perform pre- and post-processing of the variables, etc. This is probably what most users expect. I will surely consider adding such an interface in the Julia package mimicking what you have done in MATLAB but this represent substancuial work and I would put higher priority on extensively testing the current Julia interface.

This discussion makes me realize that, in Julia, It could be staightforward to require that the user-defined objective function all have the same syntax and that the non-linear constraints in COBYLA be specified as another function. Then a wrapper function embedding the two functions (the objective and the constraints) can be passed to the real COBYLA.

[emmt]

Something else that the high level interface can handle is the scaling of variables. I have open an issue here for the Julia interface to PRIMA algorithms with a proposition to cope with scaling. The proposition is not specific to the Julia case. The only remaining issue not handled by this proposition is the printing of the variables when iprint says to do that.

[zaikunzhang]

Thank you @emmt for your elaboration and for discussing the scaling. I think we have similar understandings about high-level interfaces. I have made some comments about scaling: https://github.com/emmt/PRIMA.jl/issues/4 , #95.

Apologies for my slow response.

Thanks,
Zaikun