C interface to vinecopulib

[kdheepak]

Are there any plans to write a C interface for vinecopulib? C++ is great for this sort of stuff, but writing code in Python/Julia/MATLAB that link to vinecopulib is going to be very challenging if the functions are just exposed in C++. It would be very easy to write these interfaces if commonly used functions were exposed using CDECL. Thoughts?

[tvatter]

I don't know about Julia, but regarding the other two:

• A "long" time ago, @slayoo started working on a Python interface to vinecopulib based on the Boost.Python package. Since neither @tnagler nor I are Python experts, we were not able to pursue this further. But we would be very interested in resurrecting the project if someone (who knows Python better than we do) were willing to assist.
• I think that MATLAB can handle both C and C++ seamlessly. @MalteKurz interfaced his own C++ vine copula library (albeit containing only a very small fraction of vinecopulib's functionality), see https://github.com/MalteKurz/VineCopulaMatlab.

[slayoo]

Re pyvinecopulib, there are two main issues:

• AFAIK, no one has ever provided any feedback (and I'm myself not really a user)!
• Boost.Python had a breaking API change after we wrote pyvinecopulib, hence it does not compile with recent Boost versions.
  The first problem is by far the key one!

[tvatter]

Yeah, pyvinecopulib was started at a time where vinecopulib itself was still at a very early stage, and @tnagler and myself were more focused on the C++ library and R interface. Given that the C++ library is now mature/stable enough, it would probably be a good time to resume working on this. I've already met a few other researchers that were interested in a Python interface to the vinecopulib, but time and Python skills are issues for both @tnagler and myself.

[kdheepak]

Thanks for the comments. I'll just add some additional thoughts on based on my experiences writing Python, Julia and MATLAB wrappers to C and C++ code.

1. Using swig solves a lot of problems. It's an additional burden early on but it's very customizable and can generate all the code + documentation you need in Python, R, MATLAB, Java, C#, Go, Ruby etc. You get all these languages supported by swig largely for free, just by writing one swig interface file. There's some additional customization you can do to make more specific language friendly features, i.e. if you want to make your Python interface more Pythonic you can do that through swig interface files. Swig supports both C and C++. So it may be worth considering if you want to redo the Python interface. Swig handles almost all the reference counting and dealing with the Python C API so that you don't have to.

2. For my second point, most of my thoughts are based on this talk and this talk. Having the ability to access a C ABI in the interface will be very helpful. For example, Julia bindings will work best with C interfaces. There is a package called Cxx.jl that allows interfacing with C++ directly but it appears to not be maintained at the moment. I wasn't able to get it working on my machine when trying to prototype an interface to vinecopulib yesterday. Additionally, I've been able to successfully compile the C interface on Travis-CI and Appveyor and distribute it to users running Linux, Mac and Windows with lesser issues than I have with C++ interfaces. My understanding is that with C++, compiling with multiple different versions of compilers is required to guarantee that it works across all the use cases a library can have. Distribution of a library with a C interface can be easier than a library with a C++ interface.

I can certainly help with the Python interface and Julia interface if that is of interest. Also, unfortunately, I'm much more familiar with C than C++, so there's that (and maybe that's where my bias for having a C interface is coming from). At the moment, I'm interested in a Julia interface to vinecopulib and at the moment I think that will not be possible with just the C++ interface in this repository.

[tnagler]

Many thanks for the input @kdheepak. I had a brief look at the hourglass pattern; it sounds quite interesting and might be something we want to do in the future. At least for our R interface, C++ was way easier than taking a detour to C, though.

In any case, we're happy about any help we can get. If you want to contribute anything specific, just let us know.

P.S.: We will presumably resume working on the Python interface during a short hackathon in July.

[tvatter]

OK, so we worked recently on the Python interface: pyvinecopulib. It turned out that exposing directly the C++ classes was also much easier than the detour to C. Essentially, the whole interface was written in slightly less than 600 lines, basic documentation included.

Now, the reason is that the OOP way in which vinecopulib is implemented was straightforward to translate to Python. For pyvinecopulib, we used pybind11, which is very similar to the "legacy" Boost.Python. For Julia, it is probably quite easy to mimick what we did using CxxWrap, as explained https://github.com/JuliaInterop/CxxWrap.jl#whats-the-difference-with-cxxjl.

The R interface required comparatively more work, as we not only exposed stuff from C++, but rather wrote a full API that feels more like what R users are accustomed to. I think that the Julia equivalent would be Cxx.jl.