Questions about the release process

[david2278]

I'm looking through the release process and I've noticed a few things.

1. gitian-detached-sigs hasn't been updated in 9 years, but is still part of the documention for the release process. Do we not use this anymore?
2. Running gitian-build.sh yields. I think I've followed the instructions. I'm not sure why that image is missing.

--- Building for focal amd64 ---
Stopping target if it is up
Error response from daemon: No such container: gitian-target
Error response from daemon: No such container: gitian-target
Making a new image copy
Starting target
Checking if target is upUnable to find image 'base-focal-amd64:latest' locally
docker: Error response from daemon: pull access denied for base-focal-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

The release-process.md seems out of date.

In the "Fetch and Create Inputs" section at the bottom I see this

Create the OS X SDK tarball, see the OS X readme for details, and copy it into the inputs directory.

But if you run ./gitian-build.sh --docker --setup --build --sign SIGNER --verify 1.0, the script downloaded MacOSX10.11.sdk.tar.gz into the gitian-builder/inputs folder.

I also read that we submit binaries to Apple but the binaries that are submitted to them isn't documented anywhere from what I can tell. Do we submit the unsigned binaries?

I'm basically wondering just how out of date the release process is.

Thanks.

[patricklodder]

1. gitian-detached-sigs hasn't been updated in 9 years, but is still part of the documention for the release process. Do we not use this anymore?

Unfortunately, when I took over signing for the osx binaries with the 1.14.6 release, I found that the described process was defunct. This is an open item for the next release to be fixed, as this is a tooling issue more than a certificate issue. I've given a bit more info about that here

We have not ever released signed Windows binaries, so that is subject to future enhancement. I do not plan on doing this for the next minor release - but am willing to consider it for the next major release.

2. Running gitian-build.sh yields. I think I've followed the instructions. I'm not sure why that image is missing.

You'll need to run that script with --setup before running with --build.

The release-process.md seems out of date.
[..]
I'm basically wondering just how out of date the release process is.

Correct, it is out of date because the tooling does not work within the current signing validation by Apple; it will create binaries that will be rejected by Apple notarization. Instead, I manually signed and notarized the osx binaries this time around, which is unfortunately how all 1.14 releases had been done.

[david2278]

All binaries are deterministically built and the outcome can be attested using PGP signatures, but this isn't the same as signing the binary using the schemes from the target OSs. For Dogecoin, only the MacOS binaries are signed with Apple developer signatures historically.

Ok that makes sense.

This is because you're using KVM, which is, like LXC (!), upstream defunct when used on recent Ubuntu releases. See the current docs for how to do Docker builds, and add --docker to both the setup and the build runs.

Alright. I ran ./gitian-builder.sh --docker --setup --commit version <branch_name> and it worked. I'm had to set the MIRROR_HOST to host.docker.internal for it to connect to the apt-cacher-ng service.

When running ./gitian-builder.sh --docker --commit --build --sign ` I'm getting

--- Building for focal amd64 ---
Stopping target if it is up
Error response from daemon: No such container: gitian-target
Error response from daemon: No such container: gitian-target
Making a new image copy
Starting target
Checking if target is up.
Preparing build environment
Successfully copied 47.3MB to gitian-target:/home/ubuntu/build/
Successfully copied 4.19MB to gitian-target:/home/ubuntu/build/
Successfully copied 1.54kB to gitian-target:/home/ubuntu/cache/
Successfully copied 1.54kB to gitian-target:/home/ubuntu/cache/
Updating apt-get repository (log in var/install.log)
Installing additional packages (log in var/install.log)
Upgrading system, may take a while (log in var/install.log)
Creating package manifest
Creating build script (var/build-script)
Successfully copied 103MB to gitian-target:/home/ubuntu/build/
Running build script (log in var/build.log)
Traceback (most recent call last):
        6: from ./bin/gbuild:347:in `<main>'
        5: from ./bin/gbuild:347:in `each'
        4: from ./bin/gbuild:349:in `block in <main>'
        3: from ./bin/gbuild:349:in `each'
        2: from ./bin/gbuild:354:in `block (2 levels) in <main>'
        1: from ./bin/gbuild:187:in `build_one_configuration'
./bin/gbuild:23:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)

I looked for that log file var/build.log but couldn't find it. I assume it's in a docker container.

Sorry for the late response. Trying lots of things as to not ask silly questions.

I am running on WSL2 Ubuntu 20.04

[patricklodder]

The var/build.log is located inside your gitian-builder dir. From the script, the path should be gitian-builder/var/build.log

[david2278]

I thought I looked there, but I must've missed it. Ok I got the binaries built. Thanks Patrick!

One last question, you mentioned that you had to sign and notarize the osx binaries yourself because gitian-builder signs them in a way Apple will reject. Would you mind explaining how to do this since it's not documented anywhere?

[patricklodder]

I followed (a previous version of) the command-line guide from Apple.

You'll need a recent macOs and Xcode toolset for this, because the notarization requirements change often (it was changed last November 2023).

[david2278]

Ahh I see. Ok I was just curious. I'm going to go ahead and mark this as answered. Thanks Patrick!