Skip to content

Design decisions

Maxim edited this page May 16, 2019 · 9 revisions

Principle of work

A user wants to create C bindings for library, say, libfoobar. He creates the project directory foobar-go/, inside he creates foobar/ with foobar headers and the project manifest file foobar.yml. Inside that project manifest he specifies his preferences for each of three sections (see below) and runs c-for-go foobar.yml. By default, the results will be written in a directory named after the package, i.e. the same where headers are residing in this case. It's also convenient to have a Makefile with instructions to run the generator and clean the resulting code. The resulting code is splitted among different files: cgo_helpers.* are considered internal and are not suitable for human eyes, they mostly contain helpers for type unpacking and packing (i.e. passing between Go and C in both directions) and callback proxies. Others like foobar.go, docs.go, types.go and const.go are human-readable and c-for-go strives to get them as much idiomatic as possible. There are underscores to avoid name collisions, otherwise go lint would be completely silent.

Everything is organised the way that will ease the process of patching. If you're not satisfied how your type ha been wrapped, you can ignore it from the configuration manifest and add your manual implementation in file like util.go. And any getters and setters for unions can be done that way too, because CGo considers them as a contiguous array of bytes. Also, consider adjusting the configuration manifest adding memory and pointer tips, so the translator would know the difference between a pointer and an array argument in you functions.

The GoDoc documentation being generated based on type and function comments, it's useful if headers are accessible from the project root, so you can easily jump to the original line in C header just within your editor. There is an option to specify the documentation prefix URL in config manifest, that way the comments would link to the official documentation for a particular method or struct.

In cases when API is being too low-level and the library is hardly usable even with human-readable function names and types, consider writing a wrapper that would wrap everything into idiomatic Go code with support of objects composition, channels, goroutines, thread synchronisation primitives and so on. Consider replacing methods that can be easily implemented within Go standard library. See examples for the further reference.

Project architecture overview

The c-for-go project consists of 3 parts: translator, generator and parser in order of importance.

The first part done was the translator that is responsible for name conversions from stuff_like_this_id() into StuffLikeThisID(), for type conversions from unsigned char * to []byte, from MyClass ** into **MyClass or even []*MyClass. Translator does a lot of things, some parts of its engine are done right (name substitution, rule patterns, hints/tips for type conversions), other ones are done poorly because things turned out to be different in practice when I finished the first prototype and moved to experiments, however the quality is decent so I won't bother to rewrite that part.

Generator is mostly fine, however I don't like how pointer and references are handled, a better model from translator would fix that, but that's not so critical. The generator is based on fmt.Sprintf mechanism and that helped a lot to avoid code bloat. Also it's very flexible, for an example, when Go 1.6 announced pointer rules, the required changes were including switching all allocations to C.malloc calls, have a reference count object, implementing pointer borrowing and automatic frees with Finalizers, do nested allocations while unpacking slices like [][]string and so on. I added all this to generator in a couple of days without a single hack. Thanks to the dynamic helpers mechanism, see gen_bindings.go if curious.

Parser is actually a wrapper for cznic/cc C99 compiler frontend, by coincidence it has been under active development at the same time as I started this project (summer 2015), it was in alpha state and was unstable as hell. So I freezed the version that was stable enough and wrote the translator models based on the capabilities of CC at that moment. There was a lot stuff to guess. Then Jan has completed his CC project and I switched to the upstream master, at cost of having rewriting half of the translator logic, that's the true reason why its models are in so miserable state now. Parser being configured partly from the configuration manifest and partly from predefined options in predefined.go.

All three modules are doing their best-outcome guesswork to handle the common cases in the most common ways, however when some configuration is needed, c-for-go consults the configuration manifest for the project that usually has tree sections corresponding to the modules: GENERATOR, PARSER and TRANSLATOR sorted in order of contents size.

The main executable provides the glue for all of three modules, it does config parsing, runs external utils like in-house pkg-config lightweight clone to discover more headers, manages the output buffers, formats the code using imports.Process after generation is done and so on.