Skip to content

Commit

Permalink
Trigger preproduction sync only after merge to main
Browse files Browse the repository at this point in the history 
In #916, we restructured the sync scripts. As part of this, the sync to
preprod after a ceremony completed would occur from the ceremony branch.
We only allow workload impersonation (which is needed to push to GCS and
update the CDN cache) from main, so this breaks.

To fix this, we simply split the workflow into two: The first triggers
on a push to ceremony and creates a PR to merge to main. After merging
from main and updating the root, we sync all contents from the
repository directory.

I also removed the cron job because I don't think it's needed. Also
updated documentation for post-merge steps.

Signed-off-by: Hayden Blauzvern <hblauzvern@google.com>
  • Loading branch information
@haydentherapper
haydentherapper committed Mar 14, 2024
1 parent 5fafb17 commit b6d2ef5
Show file tree
Hide file tree
Showing 3 changed files with 121 additions and 77 deletions.
73 changes: 3 additions & 70 deletions .github/workflows/sync-ceremony-to-main.yml
View file
Expand Up @@ -13,18 +13,15 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# This workflow is triggered when a ceremony branch is completed, and will:
# - Create a PR that merges the completed ceremony branch to main
# - Sync the ceremony branch to the GCS preprod bucket
# This workflow is triggered when a ceremony branch is completed,
# creating a PR that merges the completed ceremony branch to main

name: Sync Published Ceremony Branch to Main and Preprod

# Declare default permissions as none.
permissions: {}

on:
schedule:
- cron: '0 */12 * * *' # every 12 hours
workflow_dispatch:
inputs:
branch:
Expand Down Expand Up @@ -63,73 +60,9 @@ jobs:
pr_body: "Merge ceremony branch to main"
pr_reviewer: bobcallaway,haydentherapper,joshuagl,kommendorkapten

sync:
permissions:
id-token: 'write'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2
with:
fetch-depth: 0
- uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version-file: './go.mod'
check-latest: true
# Setup OIDC->SA auth
- uses: google-github-actions/auth@55bd3a7c6e2ae7cf1877fd1ccb9d54c0503c457c # v2.1.2
id: auth
with:
token_format: 'access_token'
workload_identity_provider: 'projects/306323169285/locations/global/workloadIdentityPools/github-actions-pool/providers/github-actions-provider'
service_account: 'tuf-gha@project-rekor.iam.gserviceaccount.com'
create_credentials_file: true
- uses: google-github-actions/setup-gcloud@98ddc00a17442e89a24bbf282954a3b65ce6d200 # v2.1.0
with:
project_id: project-rekor
- name: Login
run: |
gcloud auth login --brief --cred-file="${{ steps.auth.outputs.credentials_file_path }}"
gcloud auth list
# sync
- name: sync
run: |
check_expiration() {
expiry=$(jq -r '.signed.expires' $1)
expires=$(date -d $expiry +%s)
current=$(date +%s)
if (( expires < current )); then
echo "Detected expired metadata file $1 at $expiry!"
exit 1
fi;
}
# Upload all but TUF timestamp. Once timestamp is uploaded, all other files must have been uploaded.
for f in $(ls repository/repository/ -I *timestamp.json)
do
# Check for expiration if this is a non-versioned metadata file.
# Versioned metadata like 1.root.json may be expired.
# TODO(asraa): When consistent snapshots are enabled, this logic must be changed so that
# only old versioned metadata can be expired.
if [[ $f == [^0-9]*.json ]]; then
check_expiration repository/repository/$f
fi;
gcloud --quiet storage cp --cache-control=no-store -r repository/repository/$f gs://sigstore-preprod-tuf-root/
done
# Upload timestamp after checking latest timestamp expiration
check_expiration repository/repository/timestamp.json
gcloud --quiet storage cp --cache-control=no-store -r repository/repository/*timestamp.json gs://sigstore-preprod-tuf-root/
# delete any files present in sigstore-preprod-tuf-root not in repository/repository
gcloud --quiet storage cp -r gs://sigstore-preprod-tuf-root/ .
diff -qr repository/repository sigstore-preprod-tuf-root | while read l; do
if [[ $l =~ "Only in sigstore-preprod-tuf-root" ]]; then
path=$(python3 -c "import re; s='$l'; pattern=r'^Only in sigstore-preprod-tuf-root(\/?)(.*): (.*)$'; match=re.search(pattern, s); print('/'.join([match.group(2), match.group(3)]).lstrip('/'))")
gcloud --quiet storage rm gs://sigstore-preprod-tuf-root/$path
fi;
done
gcloud compute url-maps invalidate-cdn-cache tuf-preprod-repo-cdn-lb --path "/*" --async
if-failed:
runs-on: ubuntu-latest
needs: [sync]
needs: [push]
permissions:
issues: 'write'
actions: 'read'
Expand Down
109 changes: 109 additions & 0 deletions .github/workflows/sync-main-to-preprod.yml
View file
@@ -0,0 +1,109 @@
#
# Copyright 2024 The Sigstore Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This workflow is triggered a root signing completes and is merged
# into main, syncing the repository to the GCS preprod bucket

name: Sync Published Root Signing to Preprod

# Declare default permissions as none.
permissions: {}

on:
workflow_dispatch:
push:
# When a root signing ceremony completes
branches:
- main
paths:
- 'repository/repository/root.json'

jobs:
sync:
permissions:
id-token: 'write'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2
with:
fetch-depth: 0
- uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version-file: './go.mod'
check-latest: true
# Setup OIDC->SA auth
- uses: google-github-actions/auth@55bd3a7c6e2ae7cf1877fd1ccb9d54c0503c457c # v2.1.2
id: auth
with:
token_format: 'access_token'
workload_identity_provider: 'projects/306323169285/locations/global/workloadIdentityPools/github-actions-pool/providers/github-actions-provider'
service_account: 'tuf-gha@project-rekor.iam.gserviceaccount.com'
create_credentials_file: true
- uses: google-github-actions/setup-gcloud@98ddc00a17442e89a24bbf282954a3b65ce6d200 # v2.1.0
with:
project_id: project-rekor
- name: Login
run: |
gcloud auth login --brief --cred-file="${{ steps.auth.outputs.credentials_file_path }}"
gcloud auth list
# sync
- name: sync
run: |
check_expiration() {
expiry=$(jq -r '.signed.expires' $1)
expires=$(date -d $expiry +%s)
current=$(date +%s)
if (( expires < current )); then
echo "Detected expired metadata file $1 at $expiry!"
exit 1
fi;
}
# Upload all but TUF timestamp. Once timestamp is uploaded, all other files must have been uploaded.
for f in $(ls repository/repository/ -I *timestamp.json)
do
# Check for expiration if this is a non-versioned metadata file.
# Versioned metadata like 1.root.json may be expired.
# TODO(asraa): When consistent snapshots are enabled, this logic must be changed so that
# only old versioned metadata can be expired.
if [[ $f == [^0-9]*.json ]]; then
check_expiration repository/repository/$f
fi;
gcloud --quiet storage cp --cache-control=no-store -r repository/repository/$f gs://sigstore-preprod-tuf-root/
done
# Upload timestamp after checking latest timestamp expiration
check_expiration repository/repository/timestamp.json
gcloud --quiet storage cp --cache-control=no-store -r repository/repository/*timestamp.json gs://sigstore-preprod-tuf-root/
# delete any files present in sigstore-preprod-tuf-root not in repository/repository
gcloud --quiet storage cp -r gs://sigstore-preprod-tuf-root/ .
diff -qr repository/repository sigstore-preprod-tuf-root | while read l; do
if [[ $l =~ "Only in sigstore-preprod-tuf-root" ]]; then
path=$(python3 -c "import re; s='$l'; pattern=r'^Only in sigstore-preprod-tuf-root(\/?)(.*): (.*)$'; match=re.search(pattern, s); print('/'.join([match.group(2), match.group(3)]).lstrip('/'))")
gcloud --quiet storage rm gs://sigstore-preprod-tuf-root/$path
fi;
done
gcloud compute url-maps invalidate-cdn-cache tuf-preprod-repo-cdn-lb --path "/*" --async
if-failed:
runs-on: ubuntu-latest
needs: [sync]
permissions:
issues: 'write'
actions: 'read'
if: always() && needs.sync.result == 'failure'
steps:
- name: Open issue or add comment on failure
uses: sigstore/sigstore-probers/.github/actions/open-workflow-issue@main
with:
comment_for_each_failure: true
16 changes: 9 additions & 7 deletions playbooks/ORCHESTRATION.md
View file
Expand Up @@ -389,21 +389,23 @@ required.

## Step 7: Publication

Once the PR from [Step 3](#step-3-snapshotting-and-timestamping) is merged, a [workflow](../.github/workflows/sync-ceremony-to-main.yml) will automatically create a PR merging the changes on the completed ceremony branch to main.
Once the PR from [Step 6](#step-6-snapshotting-and-timestamping) is merged, a [workflow](../.github/workflows/sync-ceremony-to-main.yml) will automatically create a PR merging the changes on the completed ceremony branch to main.

Submitting this PR will trigger a push to the preproduction GCS bucket, so ensure that this PR is verified and ready to be pushed!
Submitting this PR will trigger [a workflow](../.github/workflows/sync-main-to-preprod.yml) to push to the preproduction GCS bucket for further testing, so ensure that this PR is verified and ready to be pushed!

## Post-ceremony Steps

1. The preproduction GCS bucket will need to be manually synced to the GCS production bucket as of [916](https://github.com/sigstore/root-signing/pull/916).
1. Perform manual testing against the preproduction environment. Assuming no issues, continue with the following steps.

2. If any root keyholders have changed, update the [current root keyholders](https://github.com/sigstore/root-signing#current-sigstore-root-keyholders) with their name, key ID, and location of their key material.
1. To push to the production environment, run [this workflow](../.github/workflows/sync-preprod-to-prod.yml), which syncs the preproduction and production GCS buckets.

3. If any targets have changed, update them and their usage in the table containing the [repository structure](https://github.com/sigstore/root-signing#tuf-repository-structure).
1. If any root keyholders have changed, update the [current root keyholders](https://github.com/sigstore/root-signing#current-sigstore-root-keyholders) with their name, key ID, and location of their key material.

4. Announce the root rotation on twitter and the community meeting, and thank the keyholders!
1. If any targets have changed, update them and their usage in the table containing the [repository structure](https://github.com/sigstore/root-signing#tuf-repository-structure).

5. Schedule the next root signing event one month before expiration on the calendar. Check [here](https://github.com/sigstore/root-signing/blob/e3f1fe5e487984f525afc81ac77fa5ce39737d0f/cmd/tuf/app/init.go#L29) for root expiration. Schedule a testing event for the week before.
1. Announce the root rotation on twitter and the community meeting, and thank the keyholders!

1. Schedule the next root signing event one month before expiration on the calendar. Check [here](https://github.com/sigstore/root-signing/blob/e3f1fe5e487984f525afc81ac77fa5ce39737d0f/cmd/tuf/app/init.go#L29) for root expiration. Schedule a testing event for the week before.

### Other

Expand Down

0 comments on commit b6d2ef5

Please sign in to comment.