Fix hermetic toolchain linking before Go 1.21

[siddharthab]

Go toolchain checks which linker flags are available by running the
linker on a temporary file. Before Go 1.21, this command was run in a
temporary directory, but this has now been fixed upstream.
golang/go#59952

This change will not be needed for Go versions 1.21 and above, but will
be beneficial for users on lower versions and using a hermetic
toolchain.

See bazel-contrib/toolchains_llvm#183 for some more
context.

[fmeum]

@tyler-french Could you test this as well?

[tyler-french]

I am noticing some serious degradation in our GoLink speed when testing. Have you seen anything similar?

I may need to dig a bit deeper into this to see why this might be happening.

[siddharthab]

I did not do any performance regression testing. But some performance hit may be expected. Go toolchain is now running a small linker command before running the actual link command. Previously, that small linker command was failing trivially.

[tyler-french]

I did not do any performance regression testing. But some performance hit may be expected. Go toolchain is now running a small linker command before running the actual link command. Previously, that small linker command was failing trivially.

Our CI is experiencing a 2x overall build time (without cache) with this change applied, compared to without this change.

I would like to do a little more investigation, and see if there's a way we can fix this issue without adding latency to the build. Otherwise, we should guard this feature behind a config flag so that users are not impacted unless they need this feature.

[siddharthab]

Which hermetic toolchain do you use in your CI? Maybe I can help with this investigation if I find time.

[siddharthab]

It is also very likely that you would hit the increased build time issue when you move to Go 1.21, which has a similar change as in this PR, but built into the toolchain itself.

[tyler-french]

It is also very likely that you would hit the increased build time issue when you move to Go 1.21, which has a similar change as in this PR, but built into the toolchain itself.

I tested it with Go 1.21.1

Which hermetic toolchain do you use in your CI? Maybe I can help with this investigation if I find time.

We use zig_toolchain at version 0.11.0

http_archive(
    name = "hermetic_cc_toolchain",
    sha256 = "28fc71b9b3191c312ee83faa1dc65b38eb70c3a57740368f7e7c7a49bedf3106",
    urls = [
        "https://github.com/uber/hermetic_cc_toolchain/releases/download/{0}/hermetic_cc_toolchain-{0}.tar.gz".format(HERMETIC_CC_TOOLCHAIN_VERSION),
    ],
)

load("@hermetic_cc_toolchain//toolchain:defs.bzl", zig_toolchains = "toolchains")

zig_version = "0.11.0"
zig_toolchains(
    host_platform_sha256 = {
        "linux-aarch64": "956eb095d8ba44ac6ebd27f7c9956e47d92937c103bf754745d0a39cdaa5d4c6",
        "linux-x86_64": "2d00e789fec4f71790a6e7bf83ff91d564943c5ee843c5fd966efc474b423047",
        "macos-aarch64": "c6ebf927bb13a707d74267474a9f553274e64906fd21bf1c75a20bde8cadf7b2",
        "macos-x86_64": "1c1c6b9a906b42baae73656e24e108fd8444bb50b6e8fd03e9e7a3f8b5f05686",
    },
    url_formats = [
        "https://ziglang.org/builds/zig-{host_platform}-{version}.{_ext}",
    ],
    version = zig_version,
)

[siddharthab]

I tried to reproduce with bazel build --incompatible_enable_cc_toolchain_resolution --extra_toolchains @zig_sdk//toolchain:linux_amd64_gnu.2.34 --remote_cache= --disk_cache= //tests/core/cgo:c_srcs and the updated toolchain you posted above.

I can not reproduce what you are describing; maybe I am not calling with the right toolchain target. In the example above, the link step took 20 seconds on first invocation (clean /tmp dir), and then 4 seconds for each repeated run (with bazel clean in between). This is both with and without this change.

The issue in your CI could be more specific to the setup you have.