Releases and Breaking Changes: MSRV, half 2.3 intrinsics, exr 2.0.0

[johannesvollmer]

Versioning is hard, let's discuss it. Some events that sparked discussion:

• half released a new version with intrinsic conversion operations, which should be a lot faster
• the new half version uses the newest Rust version, 1.70.0 in a minor release, which sparked a lot of debate on its own (Requiring Rust 1.70.0 and changing backwards compatibility policy in a minor version bump seems a mistake starkat99/half-rs#97)
• we prevented old projects failing by restricting the version of half to our own MSRV (Get <1.70 Working Again #213), which excludes the new intrinsics, in a minor exr release

See #191, where a lot of previous discussion already happened.

Maybe some kind of strategy should established for this repository? Maybe I just need to get all the details right about how to release new versions of Rust libraries in general?

[johannesvollmer]

Furthermore I have done some more benchmarking. The results are a little bit worrying.

Compared to older releases, the last few optimizations (we profiled and changed code) produced a speedup of about 10% - 20%. However, just by upgrading to the newest version of half, we lose most of that performance again. I'll post the results of these benchmarks when I'm back at my computer again (not before Monday)

[Shnatsel]

Huh. 1.7.0 is half 2.2 plus use-intrinsics feature, right?

[johannesvollmer]

No feature flags anywhere. So, no intrinsics except for 2.0.0.

From 1.6.3 to 1.7.0, there were various small exr optimizations, but nothing about half changed

[johannesvollmer]

It's really strange though, there must be another reason, because using the new half version degrades performance of benchmarks that don't even have any f16 conversions in them! maybe code bloat led to cache misses in the code pages?

2.0.0 read_f32_as_u32_uncompressed_1thread ... bench:  46,673,270 ns/iter (+/- 22,380,346)
1.7.0 read_f32_as_u32_uncompressed_1thread ... bench:  42,306,660 ns/iter (+/- 3,177,281)
1.6.3 read_f32_as_u32_uncompressed_1thread ... bench:  44,077,260 ns/iter (+/- 8,076,883)

[Shnatsel]

Okay, first, the file used for read_f32_as_f32 benchmarks must actually be an f16 file, because on my machine it has the exact same performance as f16 conversions. It rises and falls exactly in tandem with the f16 benchmarks, so it must be actually f16.

Second, I cannot reproduce the regression on my machine. Please make sure you're benchmarking with the same compiler version at all times. I tried half 2.2, 2.2 with use-intrinsics feature and 2.3 using rustc 1.73.0-nightly (8ca44ef9c 2023-07-10). For me the results are exactly the same using 2.2 with use-intrinsics feature and 2.3, and both beat the basic 2.2.

[johannesvollmer]

Oh, this would explain it! I'll do it again and double check the files and compiler version, good idea :) how can I make sure the same compiler is used? isn't it enough to just do rustup update once?

[Shnatsel]

The crux of the issue is that cargo does not take MSRV into account, so it can upgrade to crates that no longer build on your current rustc. Making cargo do it is the only good solution. There is simply nothing the crate authors can do to make the problem go away.

There are options with various trade-offs, but all of them have issues:

1. Major semver bump
   • Drawback: makes anything depending on 1.x and 2.x incompatible, requiring the entire ecosystem of dependent crates to upgrade in lock-step. A huge headache for everyone involved.
2. Minor version bump
   • Drawback: breaks builds on older Rust without a Cargo.lock, or when running cargo update, but at least a manual workaround is possible.
3. Introduce an optional, default feature for using newer compiler features
   • Drawback: same as (2). May not even be possible when transitive dependencies like half are involved.
4. Introduce an optional, non-default feature for using newer compiler features
   • Drawback: Users on new compilers get sub-par performance without taking additional steps. May not even be possible when transitive dependencies like half are involved.
5. Wait until the required Rust version is reasonably widely used
   • Drawback: improvements delayed by half a year or more for everyone just because of a minority on an older Rust version. Still doesn't make everyone happy.
6. Use build.rs to detect the Rust version and toggle features accordingly
   • Drawback: users find this behavior surprising. Also, the presence of build.rs complicates integration into other build systems, required e.g. when calling Rust code from another language.

But without first-class MSRV support in Cargo this is always a losing battle. Even if you've opted for one of the solutions, some transitive dependency will issue a new release with a higher MSRV and screw up all your hard work, and then it turns out you've done all these trade-offs for nothing.

[johannesvollmer]

Thank you for this summary <3 I think I got the bigger picture now. Considering that the new release of half does not improve performance, but regress instead, I suggest the solution today is to wait (5). And possibly find out why performance regressed.

In general however, I feel like (2) would be the least exhausting and least error-prone way for library developers, even though it might cause users to investigate why their build failed after running cargo update.

[Shnatsel]

I also prefer (2)

[johannesvollmer]

The last plan was to release a new major version of exr (1.8.0, not 2.0.0) that allows the newest version of half. But the benchmarks show a regression. Maybe we should just leave it as-is?