guix: build with glibc 2.31 #29987

fanquake · 2024-04-28T12:59:31Z

Set minimum required glibc to 2.31.
The glibc 2.31 branch is still maintained: https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.

Remove the stack-protector check from test-security-check, as the test
no-longer fails, and given the control we have of the end, the actual
security-check test seems sufficient (this might also be applied to some
of the other checks).

Drops runtime support for Ubuntu Bionic 18.04 and RHEL-8 from the release binaries.

DrahtBot · 2024-04-28T12:59:34Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	laanwj, hebasto

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#27038 (security-check: test for _FORTIFY_SOURCE usage in release binaries by fanquake)
#25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)
#24123 (guix: Pointer Authentication and Branch Target Identification for aarch64 Linux (Guix) by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

laanwj · 2024-04-28T13:11:20Z

Concept ACK.

For context: 2.31 is the version in Ubuntu 20.04 LTS: https://packages.ubuntu.com/search?keywords=libc6&searchon=names&suite=focal&section=all

hebasto · 2024-04-28T15:03:05Z

Concept ACK.

DrahtBot · 2024-04-29T03:05:43Z

Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

File	commit `3aaf732` (master)	commit `f7476db` (master and this pull)
SHA256SUMS.part	`8ce4f4b597436a2f...`	`e9203b2d7289235e...`
*-aarch64-linux-gnu-debug.tar.gz	`19c81ef4929cb615...`	`225aeb49b50cda0e...`
*-aarch64-linux-gnu.tar.gz	`ff703ec621a73e3d...`	`a16c03cc9a6483af...`
*-arm-linux-gnueabihf-debug.tar.gz	`c94e8a9a48ba461d...`	`38e2f9eda7afcf35...`
*-arm-linux-gnueabihf.tar.gz	`fae274acd21e12a5...`	`9fab28af22a4520d...`
*-arm64-apple-darwin-unsigned.tar.gz	`7f6368310d01f063...`	`101efbfa96e21ee3...`
*-arm64-apple-darwin-unsigned.zip	`a13f6058d091c9c4...`	`222710293e3b3984...`
*-arm64-apple-darwin.tar.gz	`b67f21e1175a1474...`	`1f06a3da97ff99eb...`
*-powerpc64-linux-gnu-debug.tar.gz	`6784df54895f62f1...`	`ba331003298af9e9...`
*-powerpc64-linux-gnu.tar.gz	`76186c9b67fc37b9...`	`86383b946efcecbb...`
*-riscv64-linux-gnu-debug.tar.gz	`37407f08315cb072...`	`890e8e119c743ec5...`
*-riscv64-linux-gnu.tar.gz	`c850f117f2c47f30...`	`bfdd2fa0afe3c50a...`
*-x86_64-apple-darwin-unsigned.tar.gz	`864a4371d95076e4...`	`509c743cfc870db8...`
*-x86_64-apple-darwin-unsigned.zip	`f83ac30e770da990...`	`7fa43da8ef969d2a...`
*-x86_64-apple-darwin.tar.gz	`af4aa412d08a1fe4...`	`89e6c3ea23bdb034...`
*-x86_64-linux-gnu-debug.tar.gz	`e33e9a3326fa8005...`	`8e7df665dc995689...`
*-x86_64-linux-gnu.tar.gz	`40cfaebaff94fed3...`	`14dae8ea1b16b505...`
*.tar.gz	`9c9a35e818ddc287...`	`052832cd94a134bb...`
guix_build.log	`50c3d8042667f717...`	`991fe935dacfb37b...`
guix_build.log.diff		`090587a012de9f4a...`

maflcko · 2024-04-29T07:18:20Z

It would be good to mention that this drops support for Ubuntu Bionic 18.04 and RHEL-8 (and forks) completely, going forward.

fanquake · 2024-04-29T07:20:50Z

That is shown in the changes in symbol-check, but I'll add it to the op, and can add a rel note.

laanwj · 2024-04-29T09:39:44Z

contrib/guix/manifest.scm

              (sha256
               (base32
-                "0azpb9cvnbv25zg8019rqz48h8i2257ngyjg566dlnp74ivrs9vq"))
-              (patches (search-our-patches "glibc-2.27-riscv64-Use-__has_include-to-include-asm-syscalls.h.patch"


Good to get rid of these libc patches, especially the rv64 one.

fanquake · 2024-05-03T02:46:47Z

Guix Build (aarch64)

d5b65771bb9c1d2e4e49fbb78305e97a99ca43e608ced495f8c437855701d6af  guix-build-f5a949b90881/output/aarch64-linux-gnu/SHA256SUMS.part
113d77f864bac9382aa7598cb038a822aff2c961348ed9ca747a168ec87199bb  guix-build-f5a949b90881/output/aarch64-linux-gnu/bitcoin-f5a949b90881-aarch64-linux-gnu-debug.tar.gz
4a25c348e2921d27542f39cd5ccb6c7cf3aba72c89fecf69f4f66d63c4d5b623  guix-build-f5a949b90881/output/aarch64-linux-gnu/bitcoin-f5a949b90881-aarch64-linux-gnu.tar.gz
6157c1dc24f370b008c605cbb3ecffa1457b667673056d890833645547544482  guix-build-f5a949b90881/output/arm-linux-gnueabihf/SHA256SUMS.part
e02ef1dd8163ae03742bc80612574da4f23578b15b5902a7e76e74c90cbecc10  guix-build-f5a949b90881/output/arm-linux-gnueabihf/bitcoin-f5a949b90881-arm-linux-gnueabihf-debug.tar.gz
d0d249f86275b79b328525f2c30388a6ada66a82c37192bec48808d5d38343fe  guix-build-f5a949b90881/output/arm-linux-gnueabihf/bitcoin-f5a949b90881-arm-linux-gnueabihf.tar.gz
d6794a51de7d2dd025ead7056f6773da0b7a80178bf7d08a57d91590162792f8  guix-build-f5a949b90881/output/arm64-apple-darwin/SHA256SUMS.part
133bde0050ac3c6cee703129af75e0bc92936bce665a65f53e56238ebe248549  guix-build-f5a949b90881/output/arm64-apple-darwin/bitcoin-f5a949b90881-arm64-apple-darwin-unsigned.tar.gz
cc9d3c5e592e5de91f489be8c40575f90b7f288a9107bcd221ef47b7086d2f8c  guix-build-f5a949b90881/output/arm64-apple-darwin/bitcoin-f5a949b90881-arm64-apple-darwin-unsigned.zip
dca1a961714a91e5cabdb44b0faa47d74300a6c5ef9ee2ce6fb6e2275a59e810  guix-build-f5a949b90881/output/arm64-apple-darwin/bitcoin-f5a949b90881-arm64-apple-darwin.tar.gz
25b54cf81380bc68e9cba9416f68d1b0c2d4df1441b48f09d10ece07d70a3e5c  guix-build-f5a949b90881/output/dist-archive/bitcoin-f5a949b90881.tar.gz
49ae2bff0116a54cc70015ab05009b27b4173954c39873fa0836be6bf5a28ed8  guix-build-f5a949b90881/output/powerpc64-linux-gnu/SHA256SUMS.part
2f40f73f6cd49241a438d09e4efbd02d9893f4d53ae6a0f88487f6ea898dd8e4  guix-build-f5a949b90881/output/powerpc64-linux-gnu/bitcoin-f5a949b90881-powerpc64-linux-gnu-debug.tar.gz
b472a460abd20967df1d6c3b7d02dbf2fd539b10345abb1a7a3fd60d83b5d657  guix-build-f5a949b90881/output/powerpc64-linux-gnu/bitcoin-f5a949b90881-powerpc64-linux-gnu.tar.gz
64f33247b61e227d317b4da06b018ebbbdee296ea434b2c7512e929e9f7c51e7  guix-build-f5a949b90881/output/riscv64-linux-gnu/SHA256SUMS.part
610610460f9ec9f12751864eaee1dedfa7ff2ce9ce2b48dccc247ab8a3faa48c  guix-build-f5a949b90881/output/riscv64-linux-gnu/bitcoin-f5a949b90881-riscv64-linux-gnu-debug.tar.gz
271c55eb8fb0be2d8a426f8d9f62f706e5847a0fa9ddf8e358ab37292c5f8d35  guix-build-f5a949b90881/output/riscv64-linux-gnu/bitcoin-f5a949b90881-riscv64-linux-gnu.tar.gz
bfd6ec13657b37872bfccd75760ee8dd433853bb725fd4d4be0eba27b548b587  guix-build-f5a949b90881/output/x86_64-apple-darwin/SHA256SUMS.part
896d93cd00d9f6ffe21847f936544cf36d19f82d83807008ee152ce698f1e28f  guix-build-f5a949b90881/output/x86_64-apple-darwin/bitcoin-f5a949b90881-x86_64-apple-darwin-unsigned.tar.gz
0c1d29f669f2d343b3a263de9adb94da55c7c86517d568a79af462ad9516718a  guix-build-f5a949b90881/output/x86_64-apple-darwin/bitcoin-f5a949b90881-x86_64-apple-darwin-unsigned.zip
76a8d4718003430b26b8769c9326a5ed7b771fdb14fd5e9bb085dbcc84e19266  guix-build-f5a949b90881/output/x86_64-apple-darwin/bitcoin-f5a949b90881-x86_64-apple-darwin.tar.gz
033700117ea5d9553b8af3526acbead59f2ff35353f9842bc57987920a76add2  guix-build-f5a949b90881/output/x86_64-linux-gnu/SHA256SUMS.part
dfa35d5c203bd05823d4a4978b70ec4f21d3c6a5bab28f9e03befea67b7386e0  guix-build-f5a949b90881/output/x86_64-linux-gnu/bitcoin-f5a949b90881-x86_64-linux-gnu-debug.tar.gz
8ca02c111530e62a18a31d9aea92b0c7b0f4ec061a58b53f27d5e92c3974829c  guix-build-f5a949b90881/output/x86_64-linux-gnu/bitcoin-f5a949b90881-x86_64-linux-gnu.tar.gz
1b705f13584f9ad6dbd4a0fd3f51adb3302cb7da07cb0c115805628ccd57d981  guix-build-f5a949b90881/output/x86_64-w64-mingw32/SHA256SUMS.part
152aa747ad4a59882ce266a3958779b6fe49027ce5675789fb3a60777c30f56c  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64-debug.zip
a6fd72d4a5f4518e927ca871d88594adeb6a95fcfe76e395352e5d35d52c56ec  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64-setup-unsigned.exe
46ef6ed1d165ac6699c293a8c7c36d8f6f97c4d0e960f255b8ca4e0e1639fd60  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64-unsigned.tar.gz
90f92900af3b39e3c0a629b4ad279e793427f68da0930472487468408fb469ca  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64.zip

Set minimum required glibc to 2.31. The glibc 2.31 branch is still maintained: https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master. Remove the stack-protector check from test-security-check, as the test no-longer fails, and given the control we have of the end, the actual security-check test seems sufficient (this might also be applied to some of the other checks).

fanquake · 2024-05-04T01:56:18Z

Rebased and pulled in 1 more commit from the 2.31 branch.

laanwj · 2024-05-05T07:32:26Z

Same output as @fanquake (just stock ubuntu 24.04 amd64)

d5b65771bb9c1d2e4e49fbb78305e97a99ca43e608ced495f8c437855701d6af  guix-build-f5a949b90881/output/aarch64-linux-gnu/SHA256SUMS.part
113d77f864bac9382aa7598cb038a822aff2c961348ed9ca747a168ec87199bb  guix-build-f5a949b90881/output/aarch64-linux-gnu/bitcoin-f5a949b90881-aarch64-linux-gnu-debug.tar.gz
4a25c348e2921d27542f39cd5ccb6c7cf3aba72c89fecf69f4f66d63c4d5b623  guix-build-f5a949b90881/output/aarch64-linux-gnu/bitcoin-f5a949b90881-aarch64-linux-gnu.tar.gz
6157c1dc24f370b008c605cbb3ecffa1457b667673056d890833645547544482  guix-build-f5a949b90881/output/arm-linux-gnueabihf/SHA256SUMS.part
e02ef1dd8163ae03742bc80612574da4f23578b15b5902a7e76e74c90cbecc10  guix-build-f5a949b90881/output/arm-linux-gnueabihf/bitcoin-f5a949b90881-arm-linux-gnueabihf-debug.tar.gz
d0d249f86275b79b328525f2c30388a6ada66a82c37192bec48808d5d38343fe  guix-build-f5a949b90881/output/arm-linux-gnueabihf/bitcoin-f5a949b90881-arm-linux-gnueabihf.tar.gz
d6794a51de7d2dd025ead7056f6773da0b7a80178bf7d08a57d91590162792f8  guix-build-f5a949b90881/output/arm64-apple-darwin/SHA256SUMS.part
133bde0050ac3c6cee703129af75e0bc92936bce665a65f53e56238ebe248549  guix-build-f5a949b90881/output/arm64-apple-darwin/bitcoin-f5a949b90881-arm64-apple-darwin-unsigned.tar.gz
cc9d3c5e592e5de91f489be8c40575f90b7f288a9107bcd221ef47b7086d2f8c  guix-build-f5a949b90881/output/arm64-apple-darwin/bitcoin-f5a949b90881-arm64-apple-darwin-unsigned.zip
dca1a961714a91e5cabdb44b0faa47d74300a6c5ef9ee2ce6fb6e2275a59e810  guix-build-f5a949b90881/output/arm64-apple-darwin/bitcoin-f5a949b90881-arm64-apple-darwin.tar.gz
25b54cf81380bc68e9cba9416f68d1b0c2d4df1441b48f09d10ece07d70a3e5c  guix-build-f5a949b90881/output/dist-archive/bitcoin-f5a949b90881.tar.gz
49ae2bff0116a54cc70015ab05009b27b4173954c39873fa0836be6bf5a28ed8  guix-build-f5a949b90881/output/powerpc64-linux-gnu/SHA256SUMS.part
2f40f73f6cd49241a438d09e4efbd02d9893f4d53ae6a0f88487f6ea898dd8e4  guix-build-f5a949b90881/output/powerpc64-linux-gnu/bitcoin-f5a949b90881-powerpc64-linux-gnu-debug.tar.gz
b472a460abd20967df1d6c3b7d02dbf2fd539b10345abb1a7a3fd60d83b5d657  guix-build-f5a949b90881/output/powerpc64-linux-gnu/bitcoin-f5a949b90881-powerpc64-linux-gnu.tar.gz
64f33247b61e227d317b4da06b018ebbbdee296ea434b2c7512e929e9f7c51e7  guix-build-f5a949b90881/output/riscv64-linux-gnu/SHA256SUMS.part
610610460f9ec9f12751864eaee1dedfa7ff2ce9ce2b48dccc247ab8a3faa48c  guix-build-f5a949b90881/output/riscv64-linux-gnu/bitcoin-f5a949b90881-riscv64-linux-gnu-debug.tar.gz
271c55eb8fb0be2d8a426f8d9f62f706e5847a0fa9ddf8e358ab37292c5f8d35  guix-build-f5a949b90881/output/riscv64-linux-gnu/bitcoin-f5a949b90881-riscv64-linux-gnu.tar.gz
bfd6ec13657b37872bfccd75760ee8dd433853bb725fd4d4be0eba27b548b587  guix-build-f5a949b90881/output/x86_64-apple-darwin/SHA256SUMS.part
896d93cd00d9f6ffe21847f936544cf36d19f82d83807008ee152ce698f1e28f  guix-build-f5a949b90881/output/x86_64-apple-darwin/bitcoin-f5a949b90881-x86_64-apple-darwin-unsigned.tar.gz
0c1d29f669f2d343b3a263de9adb94da55c7c86517d568a79af462ad9516718a  guix-build-f5a949b90881/output/x86_64-apple-darwin/bitcoin-f5a949b90881-x86_64-apple-darwin-unsigned.zip
76a8d4718003430b26b8769c9326a5ed7b771fdb14fd5e9bb085dbcc84e19266  guix-build-f5a949b90881/output/x86_64-apple-darwin/bitcoin-f5a949b90881-x86_64-apple-darwin.tar.gz
033700117ea5d9553b8af3526acbead59f2ff35353f9842bc57987920a76add2  guix-build-f5a949b90881/output/x86_64-linux-gnu/SHA256SUMS.part
dfa35d5c203bd05823d4a4978b70ec4f21d3c6a5bab28f9e03befea67b7386e0  guix-build-f5a949b90881/output/x86_64-linux-gnu/bitcoin-f5a949b90881-x86_64-linux-gnu-debug.tar.gz
8ca02c111530e62a18a31d9aea92b0c7b0f4ec061a58b53f27d5e92c3974829c  guix-build-f5a949b90881/output/x86_64-linux-gnu/bitcoin-f5a949b90881-x86_64-linux-gnu.tar.gz
1b705f13584f9ad6dbd4a0fd3f51adb3302cb7da07cb0c115805628ccd57d981  guix-build-f5a949b90881/output/x86_64-w64-mingw32/SHA256SUMS.part
152aa747ad4a59882ce266a3958779b6fe49027ce5675789fb3a60777c30f56c  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64-debug.zip
a6fd72d4a5f4518e927ca871d88594adeb6a95fcfe76e395352e5d35d52c56ec  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64-setup-unsigned.exe
46ef6ed1d165ac6699c293a8c7c36d8f6f97c4d0e960f255b8ca4e0e1639fd60  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64-unsigned.tar.gz
90f92900af3b39e3c0a629b4ad279e793427f68da0930472487468408fb469ca  guix-build-f5a949b90881/output/x86_64-w64-mingw32/bitcoin-f5a949b90881-win64.zip

luke-jr · 2024-05-07T17:19:27Z

Is there a benefit to this? Just dropping patches?

fanquake · 2024-05-08T04:13:08Z

Is there a benefit to this? Just dropping patches?

No, it's not just dropping patches. It's about us not having to maintain an EOL branch of glibc, us getting bugfixes (if relevant) to the branch we are using, us getting closer to properly supporting hardening features, fully static builds etc, by using a glibc that supports them.

laanwj · 2024-05-08T09:39:14Z

New used symbols since 2.28:

 81: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND fcntl64@GLIBC_2.28 (15)

fcntl function that can handle large file offsets on 32 bit platforms. Used in unixFileLock. unixShmLock and other posix/unix file functions.

195: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND statx@GLIBC_2.28 (15)

Extended file stat. Used in QFileSystemEngine::fillMetaData.

533: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND renameat2@GLIBC_2.28 (15

Renaming function with flag for atomic exchange and no-overwrite. Used in QFileSystemEngine::renameFile.

 85: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND log@GLIBC_2.29 (12)
750: 0000000000116400     0 FUNC    GLOBAL DEFAULT  UND exp@GLIBC_2.29 (12)
776: 0000000000116c50     0 FUNC    GLOBAL DEFAULT  UND pow@GLIBC_2.29 (12)

Newer libm symbols (apparently more optimized, i can't find much about it).

485: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND pthread_cond_clockwait@GLIBC_2.30 (32)

Wait on a condition variable until a specific clock (eg the monotonic one) reaches a certain value. Used in std::cv_status std::condition_variable::wait_for.

fanquake added the DrahtBot Guix build requested label Apr 28, 2024

laanwj added Build system Linux/Unix labels Apr 28, 2024

DrahtBot removed the DrahtBot Guix build requested label Apr 29, 2024

fanquake added the Needs release note label Apr 29, 2024

fanquake force-pushed the build_with_glibc_2_31 branch from ab62bf4 to 5c27f21 Compare April 29, 2024 09:12

fanquake removed the Needs release note label Apr 29, 2024

laanwj reviewed Apr 29, 2024

View reviewed changes

laanwj mentioned this pull request Apr 29, 2024

guix: re-enable exported symbol checking for RISC-V #28095

Open

This was referenced Apr 29, 2024

security-check: test for _FORTIFY_SOURCE usage in release binaries #27038

Draft

guix: Pointer Authentication and Branch Target Identification for aarch64 Linux (Guix) #24123

Draft

fanquake force-pushed the build_with_glibc_2_31 branch from 5c27f21 to f5a949b Compare May 3, 2024 01:15

DrahtBot mentioned this pull request May 3, 2024

[POC] guix: produce a fully -static-pie bitcoind #25573

Draft

fanquake force-pushed the build_with_glibc_2_31 branch from f5a949b to 39e6c7a Compare May 4, 2024 01:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

guix: build with glibc 2.31 #29987

guix: build with glibc 2.31 #29987

fanquake commented Apr 28, 2024 •

edited

DrahtBot commented Apr 28, 2024 •

edited

laanwj commented Apr 28, 2024 •

edited

hebasto commented Apr 28, 2024

DrahtBot commented Apr 29, 2024

maflcko commented Apr 29, 2024

fanquake commented Apr 29, 2024

laanwj Apr 29, 2024

fanquake commented May 3, 2024

fanquake commented May 4, 2024

laanwj commented May 5, 2024

luke-jr commented May 7, 2024

fanquake commented May 8, 2024

laanwj commented May 8, 2024

guix: build with glibc 2.31 #29987

Are you sure you want to change the base?

guix: build with glibc 2.31 #29987

Conversation

fanquake commented Apr 28, 2024 • edited

DrahtBot commented Apr 28, 2024 • edited

Code Coverage

Reviews

Conflicts

laanwj commented Apr 28, 2024 • edited

hebasto commented Apr 28, 2024

DrahtBot commented Apr 29, 2024

Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

maflcko commented Apr 29, 2024

fanquake commented Apr 29, 2024

laanwj Apr 29, 2024

Choose a reason for hiding this comment

fanquake commented May 3, 2024

fanquake commented May 4, 2024

laanwj commented May 5, 2024

luke-jr commented May 7, 2024

fanquake commented May 8, 2024

laanwj commented May 8, 2024

fanquake commented Apr 28, 2024 •

edited

DrahtBot commented Apr 28, 2024 •

edited

laanwj commented Apr 28, 2024 •

edited