Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Proposal to implement CDEvents (cdevents.dev) notifications for Harbor. Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
- Loading branch information
1 parent
b4c27b4
commit 1701b3f
Showing
2 changed files
with
198 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,198 @@ | ||
| # Proposal: CDEvents Support | ||
|
|
||
| Authors: | ||
| - Andrea Frittoli (afrittoli), IBM | ||
|
|
||
| Discussion: [github.com/goharbor/community/issues/225][cdevents-discussion] | ||
|
|
||
| ## Overview | ||
| - [Proposal: CDEvents Support](#proposal-cdevents-support) | ||
|
|
||
| ## Abstract | ||
| The Harbor open source registry recently [announced][harbor-28] support for [CloudEvents][cloudevents] based webhooks. CloudEvents enrich Harbor with a tool agnostic event payload format, which is a great addition to the project in the direction of interoperability. | ||
| To further increase interoperability, we propose the adoption of [CDEvents][cdevents]. CDEvents is an open source project, incubated at the [Continuous Delivery Foundation (CDF)][cdfoundation], that aims to standardize the content of events in the continuous delivery space, thus introducing interoperability across different tools. | ||
|
|
||
| ## Background | ||
|
|
||
| The CDF recently announced the [adoption of CDEvents][cdevents] by several open source tools, like Jenkins, Tekton, Spinnaker and Testkube. Since then, the implementation of CDEvents in those tools continues to progress and more open source communities expressed interest, like Harbor, ArgoCD, Tracetest and more. CDEvents is supported by several companies like Fidelity, Apple, Ericsson, IBM, SAS and others. The project is setting up collaborations with other groups, like the CNCF TAG App Delivery and the Value Stream Management Interoperability group at Oasis. | ||
|
|
||
| ## Goals | ||
|
|
||
| - Harbor emits CDEvents payload as an alternative to the existing notification formats | ||
| - Update the Harbor portal component to allow for configuration of CDEvents notification format | ||
| - Align, if possible, to any existing compatibility policy for notification format, or document any relevant exception | ||
|
|
||
| ## Non-Goals | ||
|
|
||
| - Replace existing notification formats | ||
| - Harbor consumes CDEvents. This may be addressed as a separate proposal is relevant. | ||
| - Harbor supports CloudEvents bindings other than webhook. This may be a separate conversation about whether it makes sense for Harbor to produce events for other transport types. While CDEvents are typically transported over the CloudEvents HTTP binding, the webhook binding allows for similar interoperability benefits | ||
| - No changes are expected for the helm and operators components | ||
| - The terraform provider may be updated after this proposal is implemented | ||
|
|
||
| ## Use Cases | ||
|
|
||
| A non-exhaustive list of use cases: | ||
|
|
||
| - Audit trail: Harbor is used to store artifacts as part of software development lifecycle (SDLC) workflows. The CDEvents produced by Harbor are stored along with those produced by the SCM tools used to manage the source code, the build and test tools used to build it and the deployment tools used to deploy the artifacts. All these events can be used to build an audit trail for artifacts stored in Harbor. | ||
|
|
||
| - GitOps: tools like Argo CD and Flux can poll image registries and detect a new version or tag for an image associated with a deployment they manage. They react by pushing a PR to the SCM repository which in turn triggers a deployment of the new image version. Both tools could react to an event rather than polling. If such event was in CDEvents format they can implement support for one event type rather than integrating with every possible registry. Argo CD has an RFC open to implement support of CDEvents. The CDEvent community reached out to Flux as well to discuss CDEvents support. | ||
|
|
||
| ## Proposal | ||
|
|
||
| ### Supported Events | ||
|
|
||
| Harbor events support today are documented in the [webhook notification documentation][harbor-webhook]. | ||
| CDEvents todays supports an equivalent of `PUSH_ARTIFACT` event. More events could be added to the CDEvents specification to match what Harbor does. | ||
|
|
||
| | Harbor Event | CDEvents Event | Notes | | ||
| |---|---|---| | ||
| | `PUSH_ARTIFACT` | [`artifact.published`][cdevent-artifact-published] | | | ||
| | `PULL_ARTIFACT` | `artifact.pulled` | Not defined in CDEvents, to be added | | ||
| | `DELETE_ARTIFACT` | `artifact.deleted` | Not defined in CDEvents, to be added | | ||
| | `SCANNING_COMPLETED` | tbd | We could consider using testing events, or define a new event | | ||
| | `SCANNING_STOPPED` | tbd | Does this mean that the scanning was interrupted by a user? | | ||
| | `SCANNING_FAILED` | tbd | We could consider using testing events, or define a new event | | ||
| | `QUOTA_EXCEED` | tbd | Not sure there is an interop use case for this? | | ||
| | `QUOTA_WARNING` | tbd | Not sure there is an interop use case for this? | | ||
| | `REPLICATION` | tbd | Not sure there is an interop use case for this? | | ||
| | `TAG_RETENTION` | tbd | Not sure there is an interop use case for this? | | ||
|
|
||
| #### PUSH_ARTIFACT | ||
|
|
||
| The CloudEvent notification, from the [documentation][harbor-webhook] looks like: | ||
|
|
||
| ```json | ||
| { | ||
| "specversion": "1.0", | ||
| // id is a generated UUID to make sure the unique | ||
| "id": "66e18103-09c1-41f6-982f-37df223f3eeb", | ||
| // requestid is the http request id for tracing the source request of this event | ||
| "requestid": "51c0b694-0168-4f3c-b0db-282565455d7b", | ||
| "source": "/projects/2/webhook/policies/15", | ||
| "type": "harbor.artifact.pushed", | ||
| "datacontenttype": "application/json", | ||
| "time": "2023-04-03T06:04:46Z", | ||
| "data": { | ||
| "resources": [ | ||
| { | ||
| "digest": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4", | ||
| "tag": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4", | ||
| "resource_url": "localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4" | ||
| } | ||
| ], | ||
| "repository": { | ||
| "date_created": 1680501893, | ||
| "name": "alpine", | ||
| "namespace": "harbor", | ||
| "repo_full_name": "harbor/alpine", | ||
| "repo_type": "private" | ||
| } | ||
| }, | ||
| "operator": "harbor-jobservice" | ||
| } | ||
| ``` | ||
|
|
||
| This is a CloudEvent in [structured][ce-structured], i.e. the event metadata and data are kept together in the payload in [JSON format][ce-json-format]. The `requestid` and `operator` fields are not part of the CloudEvents specification, and are thus Harbor own extensions. | ||
|
|
||
| The [CDEvent CloudEvents binding][cde-ce-binding] describes how CDEvents field map to CloudEvents metadata, which results in the following event payload: | ||
|
|
||
| ```json | ||
| { | ||
| "specversion": "1.0", | ||
| // id is a generated UUID to make sure the unique | ||
| "id": "66e18103-09c1-41f6-982f-37df223f3eeb", | ||
| // requestid is the http request id for tracing the source request of this event | ||
| "requestid": "51c0b694-0168-4f3c-b0db-282565455d7b", | ||
| "source": "/projects/2/webhook/policies/15", | ||
| "type": "dev.cdevents.artifact.published.0.1.1", | ||
| "datacontenttype": "application/json", | ||
| "time": "2023-04-03T06:04:46Z", | ||
| "data": { | ||
| "context": { | ||
| "version": "0.3.0", | ||
| "id": "66e18103-09c1-41f6-982f-37df223f3eeb", | ||
| "source": "/projects/2/webhook/policies/15", | ||
| "type": "dev.cdevents.artifact.published.0.1.1", | ||
| "timestamp": "2023-04-03T06:04:46Z" | ||
| }, | ||
| "subject": { | ||
| // id is in pURL format | ||
| "id": "pkg:localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4", | ||
| "source": "/projects/2/webhook/policies/15", | ||
| "type": "artifact", | ||
| "content": {} | ||
| } | ||
| "customData": { | ||
| "repository": { | ||
| "date_created": 1680501893, | ||
| "name": "alpine", | ||
| "namespace": "harbor", | ||
| "repo_full_name": "harbor/alpine", | ||
| "repo_type": "private" | ||
| } | ||
| } | ||
| }, | ||
| "operator": "harbor-jobservice" | ||
| } | ||
| ``` | ||
|
|
||
| The `repository` section does not have an equivalent today in the CDEvents data model. We could discuss on CDEvents side weather to include that. In the meanwhile, the `repository` data may be included in the `customData` section. | ||
|
|
||
| CDEvents users [`pURL`][purl] to describe artifact IDs. The `pURL` format can be used to describe the tag, sha and location in one string. It is still possible to include them as separate strings as part of the `customData`. | ||
|
|
||
| The `resources` field in the Harbor CloudEvent is represented as an array. If this means that a single event can be used to represent the push of multiple resources, this means that in CDEvents this would be mapped to a series of events. | ||
|
|
||
| ### Implementation Design | ||
|
|
||
| [TBD] | ||
|
|
||
| ### User Interface | ||
|
|
||
| #### Configuration | ||
|
|
||
| An additional CDEvents format will be added to the list of available format. | ||
| Some of the event types, those circled in red, would be disabled and could not be selected, at least until we define an equivalent on CDEvents side. | ||
|
|
||
|  | ||
|
|
||
| #### Compatibility Policy | ||
|
|
||
| CDEvents follows [semantic versioning][semver], and its versioning strategy is further described in the [CDEvents primer][cdevents-versions]. The current version of the specification is v0.3. Event though the CDEvents projects strives to avoid backward-incompatible changes, events in version v0.4 of the spec may include backward-incompatible changes compared to v0.3. | ||
|
|
||
| A new version of Harbor may include a new version of CDEvents which implies that CDEvents based notification may include backward-incompatible changes. This needs to be clarified in the Harbor compatibility policy. | ||
|
|
||
| #### Evolution | ||
|
|
||
| Future versions of Harbor will continue to include newer versions of CDEvents. | ||
| CDEvents SDK should be updated to benefit from security patches (if any). | ||
|
|
||
| ## Implementation | ||
|
|
||
| The proposed implementation plan will be as follows: | ||
| - Implement experimental support with CDEvents v0.3 | ||
| - `PUSH_ARTIFACT` support only | ||
| - Other events are not available for selection when CDEvents payload format is selected | ||
| - \[List of Harbor PRs] | ||
| - Implement new events and spec changes required in CDEvents v0.4 (planned for 31.08) | ||
| - \[List of CDEvents PRs] | ||
| - Implement full CDEvents support | ||
| - Full test and docs coverage | ||
| - \[List of Harbor PRs\] | ||
|
|
||
| ## Known Issue | ||
|
|
||
| [harbor-28]: https://goharbor.io/blog/harbor-2.8/ | ||
| [cdevents]: https://cdevents.dev | ||
| [cdfoundation]: https://cd.foundation | ||
| [cloudevents]: https://cloudevents.io/ | ||
| [adoption of CDEvents]: https://cd.foundation/blog/2023/05/08/cdevents-real-world-adoption/ | ||
| [harbor-webhook]: https://goharbor.io/docs/main/working-with-projects/project-configuration/configure-webhooks/ | ||
| [cdevent-artifact-published]: https://github.com/cdevents/spec/blob/v0.3.0/continuous-integration.md#artifact-published | ||
| [ce-structured]:https://github.com/cloudevents/spec/blob/main/cloudevents/bindings/http-protocol-binding.md#32-structured-content-mode | ||
| [ce-json-format]: https://github.com/cloudevents/spec/blob/main/cloudevents/formats/json-format.md | ||
| [cde-ce-binding]: https://github.com/cdevents/spec/blob/v0.3.0/cloudevents-binding.md | ||
| [purl]: https://github.com/package-url/purl-spec | ||
| [semver]: https://semver.org/ | ||
| [cdevents-versions]: https://cdevents.dev/docs/primer/#versioning | ||
| [cdevents-discussion]: https://github.com/goharbor/community/issues/225 |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.