[RELEASE 0.2] Update test-infra to include knative/test-infra#309 and k…

…native/test-infra#277 (#2746)

These are required due to changes in the GCR ACLs and service accounts.
Because of the update, #2625 must be also cherrypicked or `release.sh` will be broken.

From knative/test-infra#309

> **When creating a test cluster on Prow, use the default service account**
> This way the Prow config file is the canonical way to specify the service account.

From knative/test-infra#277

> **Don't delete images after an E2E test finishes**
> Part of #276

From #2625

> **Update test-infra to include latest features of release.sh**
> Major changes:
>
> * use flags instead of environment variables for setting GCR/GCS
> * log configuration at the start of the process
> * be more verbose when tagging/publishing

hack/release.md

@@ -3,34 +3,38 @@
 The `release.sh` script automates the creation of Knative Serving releases,
 either nightly or versioned ones.
 
-By default, the script creates a nightly release but does not publish anywhere.
+By default, the script creates a nightly release but does not publish it
+anywhere.
 
 ## Common flags for cutting releases
 
-The following flags affect the behavior of the script, no matter the type of
-the release.
-
-* `--skip-tests` Do not run tests before building the release. Otherwise,
-build, unit and end-to-end tests are run and they all must pass for the
-release to be built.
-* `--tag-release`, `--notag-release` Tag (or not) the generated images
-with either `vYYYYMMDD-<commit_short_hash>` (for nightly releases) or
-`vX.Y.Z` for versioned releases. *For versioned releases, a tag is always
-added.*
-* `--publish`, `--nopublish` Whether the generated images should be published
-to a GCR, and the generated manifests written to a GCS bucket or not. If yes,
-the destination GCR is defined by the environment variable
-`$SERVING_RELEASE_GCR` (defaults to `gcr.io/knative-releases`) and the
-destination GCS bucket is defined by the environment variable
-`$SERVING_RELEASE_GCS` (defaults to `knative-releases/serving`). If no, the
-images will be pushed to the `ko.local` registry, and the manifests written
-to the local disk only (in the repository root directory).
+The following flags affect the behavior of the script, no matter the type of the
+release.
+
+- `--skip-tests` Do not run tests before building the release. Otherwise, build,
+  unit and end-to-end tests are run and they all must pass for the release to be
+  built.
+- `--tag-release`, `--notag-release` Tag (or not) the generated images with
+  either `vYYYYMMDD-<commit_short_hash>` (for nightly releases) or `vX.Y.Z` for
+  versioned releases. _For versioned releases, a tag is always added._
+- `--release-gcs` Defines the GCS bucket where the manifests will be stored. By
+  default, this is `knative-nightly/serving`. This flag is ignored if the
+  release is not being published.
+- `--release-gcr` Defines the GCR where the images will be stored. By default,
+  this is `gcr.io/knative-nightly`. This flag is ignored if the release is not
+  being published.
+- `--publish`, `--nopublish` Whether the generated images should be published to
+  a GCR, and the generated manifests written to a GCS bucket or not. If yes, the
+  `--release-gcs` and `--release-gcr` flags can be used to change the default
+  values of the GCR/GCS used. If no, the images will be pushed to the `ko.local`
+  registry, and the manifests written to the local disk only (in the repository
+  root directory).
 
 ## Creating nightly releases
 
 Nightly releases are built against the current git tree. The behavior of the
-script is defined by the common flags. You must have write access to the GCR
-and GCS bucket the release will be pushed to, unless `--nopublish` is used.
+script is defined by the common flags. You must have write access to the GCR and
+GCS bucket the release will be pushed to, unless `--nopublish` is used.
 
 Examples:
 
@@ -44,29 +48,87 @@ Examples:
 
 ## Creating versioned releases
 
-*Note: only Knative admins can create versioned releases.*
-
 To specify a versioned release to be cut, you must use the `--version` flag.
 Versioned releases are usually built against a branch in the Knative Serving
-repository, specified by the `--branch` flag. 
-
-* `--version` Defines the version of the release, and must be in the form
-`X.Y.Z`, where X, Y and Z are numbers.
-* `--branch` Defines the branch in Knative Serving repository from which the
-release will be built. If not passed, the `master` branch at HEAD will be
-used. This branch must be created before the script is executed, and must be
-in the form `release-X.Y`, where X and Y must match the numbers used in the
-version passed in the `--version` flag. This flag has no effect unless
-`--version` is also passed.
-* `--release-notes` Points to a markdown file containing a description of the
-release. This is optional but highly recommended. It has no effect unless
-`--version` is also passed.
-
-If this is the first time you're cutting a versioned release, you'll be prompted
-for your GitHub username, password, and possibly 2-factor authentication
-challenge before the release is published.
-
-The release will be published in the *Releases* page of the Knative Serving
-repository, with the title *Knative Serving release vX.Y.Z* and the given
-release notes. It will also be tagged *vX.Y.Z* (both on GitHub and as a git
+repository, specified by the `--branch` flag.
+
+- `--version` Defines the version of the release, and must be in the form
+  `X.Y.Z`, where X, Y and Z are numbers.
+- `--branch` Defines the branch in Knative Serving repository from which the
+  release will be built. If not passed, the `master` branch at HEAD will be
+  used. This branch must be created before the script is executed, and must be
+  in the form `release-X.Y`, where X and Y must match the numbers used in the
+  version passed in the `--version` flag. This flag has no effect unless
+  `--version` is also passed.
+- `--release-notes` Points to a markdown file containing a description of the
+  release. This is optional but highly recommended. It has no effect unless
+  `--version` is also passed.
+- `--github-token` Points to a text file containing the GitHub token to be used
+  for authentication when publishing the release to GitHub. If this flag is not
+  used and this is the first time you're publishing a versioned release, you'll
+  be prompted for your GitHub username, password, and possibly 2-factor
+  authentication challenge (you must be a Knative admin to have the required
+  publishing permissions).
+
+The release will be published in the _Releases_ page of the Knative Serving
+repository, with the title _Knative Serving release vX.Y.Z_ and the given
+release notes. It will also be tagged _vX.Y.Z_ (both on GitHub and as a git
 annotated tag).
+
+Example:
+
+```bash
+# Create and publish a versioned release.
+./hack/release.sh --publish --tag-release \
+  --release-gcr gcr.io/knative-releases \
+  --release-gcs knative-releases/serving \
+  --version 0.3.0 \
+  --branch release-0.3 \
+  --release-notes $HOME/docs/release-notes-0.3.md
+```
+
+## Creating incremental build releases ("dot releases")
+
+An incremental build release (aka "dot release") is a versioned release built
+automatically based on changes in the latest release branch, with the build
+number increased.
+
+For example, if the latest release on release branch `release-0.2` is `v0.2.1`,
+creating an incremental build release will result in `v0.2.2`.
+
+To specify an incremental build release to be cut, you must use the
+`--dot-release` flag. The latest branch and release version will be
+automatically detected and used.
+
+_Note 1: when using the `--dot-release` flag, the flags `--nopublish` and
+`--notag-release` have no effect. The release is always tagged and published._
+
+_Note 2: if the release branch has no new commits since its last release was
+cut, the script successfully exits with a warning, and no release will be
+created._
+
+The following flags are useful when creating incremental build releases:
+
+- `--branch` Restricts the incremental build release to the given branch. If not
+  passed, the latest branch will be automatically detected and used.
+- `--release-notes` Points to a markdown file containing a description of the
+  release. If not passed, the notes will be copied from the previous release.
+- `--github-token` Points to a text file containing the GitHub token to be used
+  for authentication when publishing the release to GitHub. If this flag is not
+  used and this is the first time you're publishing a versioned release, you'll
+  be prompted for your GitHub username, password, and possibly 2-factor
+  authentication challenge (you must be a Knative admin to have the required
+  publishing permissions).
+
+Like any regular versioned release, an incremental build release is published in
+the _Releases_ page of the Knative Serving repository.
+
+Example:
+
+```bash
+# Create and publish a new dot release.
+./hack/release.sh \
+  --dot-release \
+  --release-gcr gcr.io/knative-releases \
+  --release-gcs knative-releases/serving
+```

hack/release.sh

@@ -16,15 +16,6 @@
 
 source $(dirname $0)/../vendor/github.com/knative/test-infra/scripts/release.sh
 
-# Set default GCS/GCR
-: ${SERVING_RELEASE_GCS:="knative-releases/serving"}
-: ${SERVING_RELEASE_GCR:="gcr.io/knative-releases"}
-readonly SERVING_RELEASE_GCS
-readonly SERVING_RELEASE_GCR
-
-# Set the repository
-export KO_DOCKER_REPO="${SERVING_RELEASE_GCR}"
-
 # Script entry point
 
 initialize $@
@@ -34,28 +25,23 @@ set -o pipefail
 
 run_validation_tests ./test/presubmit-tests.sh
 
+# Build the release
+
 banner "Building the release"
 
 # Build should not try to deploy anything, use a bogus value for cluster.
 export K8S_CLUSTER_OVERRIDE=CLUSTER_NOT_SET
 export K8S_USER_OVERRIDE=USER_NOT_SET
 export DOCKER_REPO_OVERRIDE=DOCKER_NOT_SET
 
-echo "- Destination GCR: ${KO_DOCKER_REPO}"
-if (( PUBLISH_RELEASE )); then
-  echo "- Destination GCS: ${SERVING_RELEASE_GCS}"
-fi
-
-# Build the release
-
 # Run `generate-yamls.sh`, which should be versioned with the
 # branch since the detail of building may change over time.
 readonly YAML_LIST="$(mktemp)"
 $(dirname $0)/generate-yamls.sh "${REPO_ROOT_DIR}" "${YAML_LIST}"
 readonly YAMLS_TO_PUBLISH=$(cat "${YAML_LIST}" | tr '\n' ' ')
 readonly RELEASE_YAML="$(head -n1 ${YAML_LIST})"
 
-tag_images_in_yaml "${RELEASE_YAML}" "${KO_DOCKER_REPO}" "${TAG}"
+tag_images_in_yaml "${RELEASE_YAML}"
 
 echo "New release built successfully"
 
@@ -68,8 +54,7 @@ fi
 # Publish the release
 # We publish our own istio.yaml, so users don't need to use helm
 for yaml in ${YAMLS_TO_PUBLISH}; do
-  echo "Publishing ${yaml}"
-  publish_yaml "${yaml}" "${SERVING_RELEASE_GCS}" "${TAG}"
+  publish_yaml "${yaml}"
 done
 
 branch_release "Knative Serving" "${YAMLS_TO_PUBLISH}"