Use of bitfields in the public ABI

[kornelski]

git_fetch_options store flags as a bitfield.

This is a challenge for the Rust wrapper for this library (and possibly other non-C languages), because Rust's FFI doesn't understand C bitfields. Layout of common C types is generally well-known and easy to match, but bitfields are a less common feature, and I don't know how much their layout may vary across compilers, architectures, and endianness.

Could you provide getter and setter functions for manipulating these fields? Or perhaps you could guarantee that the library's ABI will always use the least significant bits of the unsigned int field?

[ethomson]

Yeah I was trying to be clever here and preserve api and abi backcompat. (I had wanted an anonymous union but alas).

Let me think about the right path forward here, in the meantime — yes, I can make that guarantee.

[ehuss]

Sorry, I'm not sure I am following. Since git2 is using bitflags, for example:

	unsigned int update_fetchhead : 1,
	             report_unchanged : 1;

from what I understand, there isn't any guarantee about how the C compiler will lay out these fields within the unsigned int. They could appear in the most significant bits, or the least significant bits, or in any order. How can git2 guarantee the ABI in this case?

[ethomson]

Oh sorry, I misunderstood the question. I was going to guarantee that we not break the signature. You're right that this is a bit more of a mess than I had thought.

[ehuss]

Yea, I imagine this is more of a problem for Rust than most other languages. I presume most bindings like Python are written in C which don't have this problem. Rust's C bindings try to guess how the native platform's C compiler will lay out its data types, and usually (but not always) gets it right. Unfortunately Rust hasn't extended that support for bitfields, which has been a long-standing issue.

In most cases, one can guess that the C compiler will lay out bitfields in a specific way, but I don't feel particularly comfortable doing that. It's not guaranteed, and I am not an expert enough on every C compiler and platform out there to be confident about those guesses.

I don't think it is too big of an issue. I will look into adding some small C functions which will take care of setting those bitfields. If it looks like that would be helpful for other languages, it might be considered to include those in libgit2. Otherwise, if this is just a Rust-specific thing, I think we can keep those on our side.

[ethomson]

I don't think it is too big of an issue. I will look into adding some small C functions which will take care of setting those bitfields.

Yeah but you can imagine that a user will have a clang compiled version of libgit2 and compile their app with gcc. I bet it will work but there's no guarantee and we shouldn't have this mess in our public API/ABI.

We need a 1.8.1 shortly, let me see how we can fix this mess without hopefully making things worse.

[ehuss]

Yea, good point. Thanks for taking a look!

[ethomson]

See #6806. This preserves ABI compatibility, but breaks API compatibility with any consumers who had started using report_unchanged. I'm loathe to break API compat in a point release but given how new report_unchanged is, I suspect that I can tear off the bandaid without impacting anyone.