binutils: 2.39 -> 2.40 #211126

trofi · 2023-01-16T18:22:43Z

A few potentially disruptive changes:

binutils does not embed ${binutils-unwrapped}/lib as a default library search path anymore. This will cause link failures for -lbfd -lopcodes users that did not declare their dependency on those libraries. They will need to add libbfd and libopcodes attributes to build inputs.
libbfd and libopcodes attributes now just reference binutils-unwrapped.{dev,lib} pair of attributes without patching binutils build system.

We don't patch build system anymore and use multiple outputs out of existing binutils build. That makes the result more maintainable: no need to handle ever growing list of dependencied of libbfd. This time new addition was libsframe.

To accomodate out / lib output split I had to remove lib -> bin backreference by removing legacy lookup path for plugins.

I also did not enable zstd just yet as nixpkgs version of zstd package pulls in cmake into bootstrap sequence.

Changes: https://lists.gnu.org/archive/html/info-gnu/2023-01/msg00003.html

Description of changes

Things done

trofi · 2023-01-16T19:40:34Z

…in on wasi In NixOS#211126 I simplified `binutils` and `libbfd` derivations to follow upstream binutils build system closer. As a result of `./configure --target=wasm32-unknown-wasi` `binutils` does not install plugin headers by default. This causes `pkgsCross.wasi32.llvm_12` (used by `firefox`) to fail the build as: [ 81%] Building CXX object tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o /build/llvm/tools/gold/gold-plugin.cpp:38:10: fatal error: plugin-api.h: No such file or directory 38 | #include <plugin-api.h> | ^~~~~~~~~~~~~~ The change accomodates this constraint to disable plugin support for `wasi` targets.

trofi · 2023-01-18T19:30:14Z

Fixed llvm12 by avoiding plugin use for wasi default targets.

…in on wasi In NixOS#211126 I simplified `binutils` and `libbfd` derivations to follow upstream binutils build system closer. As a result of `./configure --target=wasm32-unknown-wasi` `binutils` does not install plugin headers by default. This causes `pkgsCross.wasi32.llvm_12` (used by `firefox`) to fail the build as: [ 81%] Building CXX object tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o /build/llvm/tools/gold/gold-plugin.cpp:38:10: fatal error: plugin-api.h: No such file or directory 38 | #include <plugin-api.h> | ^~~~~~~~~~~~~~ The change accomodates this constraint to disable plugin support for `wasi` targets.

We need newer pahole to support `binutils-2.40` which started generating fresh DWARF tags that pahole-1.24 does not yet understand and fails as: $ nix log /nix/store/ckjr3sbsh13y1prigppk2y0jpf0p4icm-linux-6.1.6.drv ... BTF .btf.vmlinux.bin.o Unsupported DW_TAG_unspecified_type(0x3b) Encountered error while encoding BTF. Upstream thread to add the support for it: https://lore.kernel.org/all/YzwkazNc6wNCpQTN@kernel.org/t/

trofi · 2023-01-18T23:15:37Z

Built my system and booted it successfully in qemu. I think it's good enough for review.

SuperSandro2000 · 2023-01-19T14:25:21Z

Way out of my compiler knowledge but this reminds me to share the code between llvm/gcc versions again.

trofi · 2023-01-21T12:34:34Z

pkgs/development/compilers/llvm/10/llvm/default.nix

@@ -5,6 +5,7 @@
 , cmake
 , python3
 , libffi
+, enableGoldPlugin ? (!stdenv.isDarwin && !stdenv.targetPlatform.isWasi)


I expected the change to trigger firefox (and llvm targeting wasm) to rebuild:

] ++ optionals (enableGoldPlugin) [ "-DLLVM_BINUTILS_INCDIR=${libbfd.dev}/include"

It's not too much of a rebuld, but I'd still like to have it here if it's not too much of a complexity for the whole PR.

ghost · 2023-01-21T11:47:40Z

pkgs/development/tools/misc/binutils/default.nix

+  # Ideally we would like to always install 'lib' into a separate
+  # target. Unfortunately cross-compiled binutils installs libraries
+  # across both `$lib/lib/` and `$out/$target/lib` with a reference
+  # from $out to $lib. Probably a binutils bug: all libraries should go


I agree.

I fought with this for a few hours trying to fix up all the contorted paths that binutils' make install creates and finally gave up.

trofi · 2023-01-21T12:30:28Z

pkgs/development/tools/misc/binutils/default.nix

+  # to $lib as binutils does not build target libraries. Let's make our
+  # life slightly simpler by installing everything into $out for
+  # cross-binutils.
+  ++ lib.optionals (targetPlatform == hostPlatform) [ "lib" ];


Yeah, I'd like to fix it upstream first as binutils does not seem to install any target-specific libraries outside lib/. The split between libbfd.so and libsframe.so locations does not look intentional. Once it's sorted we could bring both cases closer (either by using or not by using target prefixes consistently).

I can understand why FHS system wants to distinguish two binutils installs for two different targets in a single prefix. Maybe it's worth fixing as well.

pkgs/development/tools/misc/binutils/default.nix

+  outputs = [ "out" "info" "man" "dev" ]
+  # Ideally we would like to always install 'lib' into a separate
+  # target. Unfortunately cross-compiled binutils installs libraries
+  # across both `$lib/lib/` and `$out/$target/lib` with a reference
+  # from $out to $lib. Probably a binutils bug: all libraries should go
+  # to $lib as binutils does not build target libraries. Let's make our
+  # life slightly simpler by installing everything into $out for
+  # cross-binutils.
+  ++ lib.optionals (targetPlatform == hostPlatform) [ "lib" ];


vcunat · 2023-01-21T13:03:33Z

Hydra jobset for stabilization: https://hydra.nixos.org/jobset/nixpkgs/pr-211126-binutils-2.40

trofi · 2023-01-24T22:59:42Z

Fixing minor snag on darwin where allowedRequisites had to be extended slightly to cover split binutils as:

--- a/pkgs/stdenv/darwin/default.nix
+++ b/pkgs/stdenv/darwin/default.nix
@@ -733,6 +733,7 @@ rec {
         pcre.out
         gettext
         binutils.bintools
+        binutils.bintools.lib
         darwin.binutils
         darwin.binutils.bintools
         curl.out

It does not affect existing linux builds.

trofi · 2023-01-27T23:13:29Z

Failures:

(aarch64-linux) libbfd and libopcodes (log):

A few potentially disruptive changes: - binutils does not embed ${binutils-unwrapped}/lib as a default library search path anymore. This will cause link failures for -lbfd -lopcodes users that did not declare their dependency on those libraries. They will need to add `libbfd` and `libopcodes` attributes to build inputs. - `libbfd` and `libopcodes` attributes now just reference `binutils-unwrapped.{dev,lib}` pair of attributes without patching `binutils` build system. We don't patch build system anymore and use multiple outputs out of existing `binutils` build. That makes the result more maintainable: no need to handle ever growing list of dependencied of `libbfd`. This time new addition was `libsframe`. To accomodate `out` / `lib` output split I had to remove `lib` -> `bin` backreference by removing legacy lookup path for plugins. I also did not enable `zstd` just yet as `nixpkgs` version of `zstd` package pulls in `cmake` into bootstrap sequence. Changes: https://lists.gnu.org/archive/html/info-gnu/2023-01/msg00003.html

trofi · 2023-01-28T08:17:26Z

Fixed wrappers as:

--- a/pkgs/development/tools/misc/binutils/libbfd.nix
+++ b/pkgs/development/tools/misc/binutils/libbfd.nix
@@ -8,6 +8,7 @@ stdenv.mkDerivation {

   dontUnpack = true;
   dontBuild = true;
+  dontInstall = true;
   propagatedBuildInputs = [
     binutils-unwrapped-all-targets.dev
     binutils-unwrapped-all-targets.lib

trofi · 2023-01-28T14:06:51Z

Skimmed through https://hydra.nixos.org/jobset/nixpkgs/pr-211126-binutils-2.40 and did not find any build failure related to binutils. I would say it is looking great!

martinetd · 2023-03-12T23:20:34Z

BTW not complaning here, but does any of you know why ofborg didn't add me as reviewer on this PR that upgraded pahole?

(I recall something about it looking only at PR title? I guess when doing some mass modification we wouldn't want it to notify everyone, but while I appreciate the work done I didn't like finding out pahole was on a random commit when someone upgraded it again because it failed build 6.3-rc1 and I had to find out what happened... Well in that case as well I wonder what maintainers are for if I'm only notified after someone pushes another upgrade and not when there's a problem, but I guess that second upgrade was straightforward enough to do directly)

trofi · 2023-03-13T00:10:02Z

ofborg should look at all the commits. The green arrow against e1ef521 has this detail on maintainers specified: https://gist.github.com/GrahamcOfBorg/03fefe23cc2d730a4ac58f66d28872ed

martinetd: pahole, pahole

I would expect you to be CCed. Worth asking ofborg maintainers?

martinetd · 2023-03-13T00:29:53Z

Thanks, I've asked in NixOS/ofborg#636

trofi requested review from lovesegfault and Ericson2314 January 16, 2023 19:37

ofborg bot added 10.rebuild-darwin-stdenv 10.rebuild-linux-stdenv labels Jan 16, 2023

ofborg bot requested review from SuperSandro2000 and adevress January 16, 2023 22:17

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Jan 16, 2023

trofi marked this pull request as draft January 17, 2023 00:14

ofborg bot requested review from dtzWill, 7c6f434c, lovek323 and primeos January 18, 2023 21:55

trofi added 2 commits January 18, 2023 22:04

trofi force-pushed the binutils-update branch from 1aab85e to 08748c9 Compare January 18, 2023 22:04

trofi marked this pull request as ready for review January 18, 2023 23:15

trofi requested a review from matthewbauer as a code owner January 18, 2023 23:15

trofi mentioned this pull request Jan 19, 2023

Create binutils-2.40 compatible pahole release? acmel/dwarves#37

Closed

ghost approved these changes Jan 21, 2023

View reviewed changes

ghost reviewed Jan 21, 2023

View reviewed changes

trofi force-pushed the binutils-update branch from 08748c9 to 81eb44f Compare January 24, 2023 22:58

github-actions bot added the 6.topic: stdenv Standard environment label Jan 24, 2023

trofi force-pushed the binutils-update branch from 81eb44f to e1ef521 Compare January 27, 2023 23:16

trofi merged commit 0ba9da4 into NixOS:staging Jan 29, 2023

trofi deleted the binutils-update branch January 29, 2023 09:51

martinetd mentioned this pull request Mar 13, 2023

maintainers detected but not pinged? NixOS/ofborg#636

Open

martinetd self-requested a review March 13, 2023 10:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

binutils: 2.39 -> 2.40 #211126

binutils: 2.39 -> 2.40 #211126

trofi commented Jan 16, 2023 •

edited

trofi commented Jan 16, 2023 •

edited

trofi commented Jan 18, 2023

trofi commented Jan 18, 2023

SuperSandro2000 commented Jan 19, 2023

This comment was marked as resolved.

trofi Jan 21, 2023

ghost Jan 21, 2023

This comment was marked as resolved.

trofi Jan 21, 2023

This comment was marked as resolved.

vcunat commented Jan 21, 2023

trofi commented Jan 24, 2023 •

edited

trofi commented Jan 27, 2023 •

edited

trofi commented Jan 28, 2023

trofi commented Jan 28, 2023

martinetd commented Mar 12, 2023

trofi commented Mar 13, 2023

martinetd commented Mar 13, 2023

binutils: 2.39 -> 2.40 #211126

binutils: 2.39 -> 2.40 #211126

Conversation

trofi commented Jan 16, 2023 • edited

Description of changes

Things done

trofi commented Jan 16, 2023 • edited

trofi commented Jan 18, 2023

trofi commented Jan 18, 2023

SuperSandro2000 commented Jan 19, 2023

This comment was marked as resolved.

trofi Jan 21, 2023

Choose a reason for hiding this comment

ghost Jan 21, 2023

Choose a reason for hiding this comment

This comment was marked as resolved.

trofi Jan 21, 2023

Choose a reason for hiding this comment

This comment was marked as resolved.

vcunat commented Jan 21, 2023

trofi commented Jan 24, 2023 • edited

trofi commented Jan 27, 2023 • edited

trofi commented Jan 28, 2023

trofi commented Jan 28, 2023

martinetd commented Mar 12, 2023

trofi commented Mar 13, 2023

martinetd commented Mar 13, 2023

trofi commented Jan 16, 2023 •

edited

trofi commented Jan 16, 2023 •

edited

trofi commented Jan 24, 2023 •

edited

trofi commented Jan 27, 2023 •

edited