Trigger artifact and docker release workflows on published event

[siladu]

NOTE: Leaving as draft as this is untested. Would like feedback on approach first.

Context

Currently, we need to create a pre-release first in order to trigger these workflows.

If we burn-in from a source build and are ready to release then this is an unnecessary step. In fact, we may forget to do it which breaks the release workflow (currently just publishing docker latest tags but they rely on this docker workflow having completed).

@jflo I'm not sure whether this is actually an improvement or always going through pre-release would be better...
For example, in the case where we do a pre-release and then "promote" it to a release, would these workflow run a second time or is pre-release -> release not counted as another "publish" step?

...If that is a problem, then we could account for this PR in our process by only ever using prerelease for -RCx builds and ensure we re-tag as a full release for the final release publish.

Impl Details...

https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#release

Note: The prereleased type will not trigger for pre-releases published from draft releases, but the published type will trigger. If you want a workflow to run when stable and pre-releases publish, subscribe to published instead of released and prereleased.

[jflo]

if we are ok with forfeiting the ability to provide users with pre-releases, this is fine. I think i'm ok with that, if folks want something bleeding edge, they can get a nightly. If we want to actually provide something, we just name it an RC and publish it.

[siladu]

if we are ok with forfeiting the ability to provide users with pre-releases, this is fine. I think i'm ok with that, if folks want something bleeding edge, they can get a nightly. If we want to actually provide something, we just name it an RC and publish it.

Yeh I was trying to get at that with this point

...If that is a problem, then we could account for this PR in our process by only ever using prerelease for -RCx builds and ensure we re-tag as a full release for the final release publish.

So a possible approach if this PR were merged would be:

• Always burn-in using an -RCx tag, and optionally create a DRAFT release
• Once burn-in is signed off, we have two paths:
  1. Create a second tag for the same commit and go straight to full release
  2. Create a prerelease with the -RCx tag and have a public prerelease phase for some period, before doing (1).

Worse case scenario, if a pre-release is "converted" to a full release and hence the same tag is republished then we might re-run the jobs with different dates...and so possibly different SHAs which isn't great but hopefully we could make the deterministic on commit hash, rather than date.

[siladu]

The reality is, if we create a pre-release then some users will use it, regardless of what we say, especially since external notification systems are alerting them about it. So if we intend to hold back releases until we are satisfied with a burn-in then I think the above approach is good.

[siladu]

Testing:

• [PASS] Creating pre-release on tag1 triggers artifacts
• [PASS] Updating pre-release to full release on tag1_commit1 does not re-trigger artifacts
• [PASS] Updating pre-release on tag1_commit1 to full release tag2_commit1 does not re-trigger artifacts
• [FAIL] Creating full release directly on tag1 triggers all workflows but "release besu" doesn't depend on the others when it should

[siladu]

[FAIL] Creating full release directly on tag1 triggers all workflows but "release besu" doesn't depend on the others when it should

In 46709d2 (#6963), I introduced a dependency on docker completing successfully before the "full release" would run.

However, I think this was acting a second trigger (an OR instead of an AND) alongside the existing:

  release:
    types: [released]

I have tried and failed to get a conditional trigger working on another branch: https://github.com/siladu/besu/blob/trigger-on-published-test/.github/workflows/release.yml

I therefore don't think the idea for this PR is workable and will close it.

[siladu]

After discussing with the team, @jframe suggested just having all 3 workflows (artifacts, docker, release) simply trigger when we do the full release, i.e. only publish anything (and everything) upon a full release.

We could merge the artifacts and docker workflows into release, then it would be easier for the latest docker job to depend on the previous docker job.

That would leave no workflows triggering (and therefore no artifacts generating) on pre-release, although we don't really need pre-release currently. However, we could just leave artifacts and docker untouched so they would trigger, and just duplicate their behaviour in release.

The intention here would be that we never convert a pre-release into a full release, only create one or the other.

This approach is captured here: #6994