make bitfields part of D

[WalterBright]

This removes the -preview=bitfields requirement.

Not having D bitfields impairs the usability of ImportC, as there isn't an easy way to be compatible with C bitfields (which are surprisingly complicated). The same code in the compiler that implements bitfields for ImportC implements them for D, ensuring compatibility.

D's bitfields support has also improved the ImportC bitfields, as use of it uncovered bugs in it. After all, the implementation is the same code.

Spec documentation: dlang/dlang.org#3190

[dlang-bot]

Thanks for your pull request, @WalterBright!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#16084"

[RazvanN7]

What we normally do in such situations is to deprecate the preview flag and simply state that the behavior now has become the default one, no need for preview switches. That way we can eventually get rid of the code that handles the preview switch without actually breaking anyone's code.

[tgehr]

I understand the benefit of easy compatibility with C, but C bitfields are still a bad way to do bitfields because they are not portable. This adds a footgun with convenient built-in syntax to D proper. It would be good to at least require e.g. extern(C) on the struct declaration.

[schveiguy]

I don’t have the energy to fight this. Just log my objection to adding c bitfields here (which I don’t expect to matter at this point). I will repost the obvious objections to c bitfields and what I think could be a good addition to d: https://dpldocs.info/this-week-in-d/Blog.Posted_2022_09_12.html#bitfields

[tgehr]

Also note that the main benefit of this interoperability is for incremental porting of existing C code to D. No sane C library would have C bitfields in the public API.

[WalterBright]

@tgehr there are two kinds of portability:

1. file format portability
2. C compiler portability

This provides the latter, as you mentioned: "the main benefit of this interoperability is for incremental porting of existing C code to D". As I discovered when implementing ImportC, this is not trivial to accomplish.

No sane C library would have C bitfields in the public API.

Sane or not, I encounter insane C code all the time. People still expect it to be supported.

[WalterBright]

@schveiguy I do understand Adam's objections. I see bitfields used a lot in C code. Anything reasonable we can do to make it easy for people to ease into using D with their C code is worth doing. I've wanted to use bit fields in the dmd source code many times, as it is the easiest way to compact the memory footprint of a struct. Yes, I could use functions to do that, but bitfields are just easier. Not having bitfields causes us to use flags instead, which is an unnecessary increase in complexity.

[WalterBright]

Although pedantically C bitfields are not portable, in practice they are if one sticks with int bitfields. It's similar to the size of an int not being reliable in C according to the Standard, but in practice, for the machines D targets, int is 32 bits.

Another non-portable aspect of C is the struct layout. Different C compilers lay things out differently.

[ibuclaw]

My sense is that in place of bitfields, it would be better to support N2763 - Nbit integers - instead (part of C23 which is due to be ratified this year). But apart from allowing both variables and fields to be bit sized, it's just a cosmetic difference. How it looks has nothing to do with the oft repeated concern of inconsistent layouts between platforms - I can't say that D bitfields are compatible with all platforms I do testing on.

[tgehr]

@schveiguy I do understand Adam's objections. I see bitfields used a lot in C code. Anything reasonable we can do to make it easy for people to ease into using D with their C code is worth doing.

The key word here is reasonable. It is good to support bitfields, just as it is good to support arrays. D has however still not copied C's absymal array design.

I've wanted to use bit fields in the dmd source code many times, as it is the easiest way to compact the memory footprint of a struct. Yes, I could use functions to do that, but bitfields are just easier. Not having bitfields causes us to use flags instead, which is an unnecessary increase in complexity.

You are arguing against a straw man. I (and probably Steve too) want:

• Bitfields that fit into the D language well
• Interopability with C (also with bitfields)

What you are arguing against:

• Not having bitfields at all.

The obvious solution is:

• Slightly change the syntax of this feature to make sure it's obviously related to C interop.
• Merge C bitfield interop.
• Keep open the door for a bitfields proposal that is better for D.

[WalterBright]

interoperability

Using bitfields in the D part of the compiler will effortlessly interoperate with bit fields in the gdc and ldc backends.

[tgehr]

interoperability

Using bitfields in the D part of the compiler will effortlessly interoperate with bit fields in the gdc and ldc backends.

Yes right (except for bootstrapping and on more exotic architectures). My objection is about using syntax that suggests this is a first-class D feature. It would be much better if it did not.

[schveiguy]

@schveiguy I do understand Adam's objections. I see bitfields used a lot in C code. Anything reasonable we can do to make it easy for people to ease into using D with their C code is worth doing.

Adding C bitfields to D is not required for this. It already works via ImportC.

and probably Steve too

I want C bitfields to be available, via ImportC, and D bitfields to be portable (i.e. not C bitfields) so I can know that when I specify a bitfield layout, the specification is unambiguous. It can be enough to pick one C implementation (the most intuitive one hopefully), and make that the D implementation.

I like Adam's solution, because it's immediately obvious what it does, how it does it, and has a clear path to fit into D's metaprogramming features. Have you considered how one is to iterate over bitfield members? How does typeof work? etc.

In the spec for bitfields, it basically says "implementation defined, go read the C compiler implementation". Is that really what we want to put in there? Why can't we specify an exact ABI for this?

[WalterBright]

There's been a suggestion to make this into a DIP, which I concur with. So save your points for that!