Skip to content

Commit

Permalink
Layered operator (#272)
Browse files Browse the repository at this point in the history
* yaml/configMap default configuration

* fix make test

* fix with new objects

* fix with new objects

* config small fixes

* fix for #51

* fix for #58

* init next (design improvement)

* initial model

* initial model

* initial

* initial

* initial

* format and license

* factory and pswd generator

* delete onCreate handler

* support configmapfiles, dynamic-plugins

* initial model test framework

* configurations

* more comments and tests

* add more tests, remove old logic of object creation

* add more tests, remove old logic of object creation

* add support of keys, integration tests passed

* add support of keys, integration tests passed

* fix npe

* cm envs

* tmp

* maintain images env var

* fix lint

* remove unused params in status

* fix make release-build

* fix default images

* several fixes

* fix route.Spec.To.Name

* fix image env vars

* fix dynamic plugins

* fix

* remove ownership of depl, ss, service

* DbSecret and Route

* clean up

* make test

* clean db

* status

* fix gosec

* cleanup

* route fix

* patch and route

* fix lint

* fix

* working...

* temp

* refactor runtime

* temp

* temp

* tmp

* temp

* dbsecret

* fix

* fix

* operator-script

* fix

* test

* fix

* rename module

* types

* ctrl test fixed

* ctrl test fixed

* fix

* container permissions

* chore: gosec check is looking for a build stage, so give it one (#163)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: only generate PR previews and next... (#161)

* chore: only generate PR previews and next builds for paths listed in the GH action (exclude changes to doc, etc.)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* indent

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* use a check-changes stage to set an env.CHANGES with either a list of changed files or a nullstring; if null, don't build anything

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* run 'PR Publish' stage for all PRs, but if no changes, skip the subsequent setup/build/publish stages

Signed-off-by: Nick Boldt <nboldt@redhat.com>

---------

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* no-op to test if new PR check will skip... (#164)

* no-op to test if new PR check will skip building container images for a readme update

Signed-off-by: Nick Boldt <nboldt@redhat.com>

must checkout before we can git diff, obviously

Signed-off-by: Nick Boldt <nboldt@redhat.com>

must checkout before we can git diff, obviously

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Update README.md

---------

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: multiline env var; explicitly check diff against HEAD~1 (#167)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: skip the golang build if there's no... (#168)

* chore: skip the golang build if there's no changes to the golang files (see regex)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* don't fail if nothing returned by grep

Signed-off-by: Nick Boldt <nboldt@redhat.com>

---------

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: use multiline github env; check HEAD~1 for diff; reorder regexes (#170)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: no auth needed to run tests (#171)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* move env.CHANGES check to substages as that's where env is defined (#173)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* bump to latest actions (node 16 -> 20) (#172)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: move commit check into the same job as the build as it seems env vars do not cross job boundaries (#174)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: fix: remove dep on other job (#175)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: move commit check into the same job as the build as it seems env vars do not cross job boundaries; remove dep on other job (#176)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Security mitigation: remove secret get from RBAC (#160)

* Security mitigation: remove secret get from RBAC

* Security migtigation: update the description for the custom image and extraFile secrets in the CRD

* Security compliance: remove create and update from RBAC for PV and PVC

* Code cleanup: remove unused clientset

* chore: label every new issue with jira label (#181)

* chore: bump csv to 1.2 in main

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: RHIDP-855 tweak csv/operator/subscription descriptions

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Add instructions for installing CI Builds and move install scripts here (#184)

* Move CI Builds install script from personal gist to upstream repo

* Add instructions for installing CI Builds of the RHDH operator

* Reference the CI Builds instructions from the main install doc

* Use single script rather than 2 nearly identical ones

This is largely inspired from the installCatalogSourceFromIIB.sh script in the internal GitLab repo.

Co-authored-by: Nick Boldt <nboldt@redhat.com>

* Update .rhdh/scripts/install-rhdh-catalog-source.sh

* Apply suggestions from code review

Co-authored-by: Nick Boldt <nboldt@redhat.com>

* Fix undeclared var: INSTALL_PLAN_APPROVAL

Co-authored-by: Nick Boldt <nboldt@redhat.com>

* Update install script help output

* Update .rhdh/scripts/install-rhdh-catalog-source.sh

* Apply suggestions from code review

Co-authored-by: Nick Boldt <nboldt@redhat.com>

---------

Co-authored-by: Nick Boldt <nboldt@redhat.com>

* chore: RHIDP-855 rename the operator to append 'Operator' on it; relabel the CRD/Backstage instance as 'Red Hat Developer Hub' with a more detailed description too (#189)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Documentation for security mitigation (#182)

* Documemtation for security mitigation

* rename openshift-rhdh-operator to rhdh-operator for suggested namespace

* Update docs/admin.md

---------

Co-authored-by: Armel Soro <armel@rm3l.org>

* Add script and docs for air-gapped/restricted env setup (#183)

* feat: new script for restricted env setup - fetch dev hub images and related images from the index, and mirror to a cluster's internal registry
TODO: fix the skopeo copy step - not working :(

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Add script to deploy and expose mirror registry into the cluster

* 'skopeo copy' now working with deployed mirror registry

* Replace 'registry.redhat.io/rhdh/*' with 'quay.io/rhdh/*', as those images are not public yet?

* Add steps for deploying mirror registry in the same prepare-restricted-environment.sh script, using a 'use_existing_mirror_registry' option

Co-authored-by: Nick Boldt <nboldt@redhat.com>

* Delete previous deploy-mirror-registry.sh script

* Update .gitignore

* Move prepare-restricted-environment.sh to .rhdh/scripts

* Make helper mirror registry storage capacity configurable

This is to allow running it on CRC,
where storage might depend on CRC VM.

* Use right OCP major version for release image

* Change condition for replacing non-public CI images with quay.io

This script should work for customers installing GA version (1.1+) to their airgapped environment.
We also do the replacement only for rhdh images, and only if the image manifest does not exist, which would likely mean that the image is not public yet.

* Force-recreate the helper mirror registry Deployment

Generated registry password will change if we run the script twice. So we won't be able to login using the new password.

* Clean prepare-restricted-environment.sh script

* Add docs

* fixup! Add docs

* Update .rhdh/scripts/prepare-restricted-environment.sh

Co-authored-by: Jianrong Zhang <jianrongzhang89@gmail.com>

Co-authored-by: Nick Boldt <nboldt@redhat.com>

---------

Signed-off-by: Nick Boldt <nboldt@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Fix sonarlint vulnerabilities (initial) (#185)

* fix sonarlint issues (initial)

* increase limits

* Update config/manager/manager.yaml

---------

Co-authored-by: Armel Soro <armel@rm3l.org>

* Avoid hardcoded images (#187)

* remove hardcoded images

* fix image

* Update examples/janus-cr-with-app-configs.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* change lookup

* Update config/manager/default-config/db-statefulset.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update config/manager/default-config/deployment.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* change lookup

* change lookup

* Update config/manager/default-config/deployment.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* add generated files

* fix image

---------

Co-authored-by: Armel Soro <armel@rm3l.org>

* Port latest changes (automountServiceAccountToken and ephemeral storage limit) to downstream CSV for RHDH (#197)

This is an addendum commit to #185

* Fix service raw configuration (#203)

* remove hardcoded images

* fix image

* Update examples/janus-cr-with-app-configs.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* change lookup

* Update config/manager/default-config/db-statefulset.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update config/manager/default-config/deployment.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* change lookup

* change lookup

* Update config/manager/default-config/deployment.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* add generated files

* fix image

* fix service raw config

---------

Co-authored-by: Armel Soro <armel@rm3l.org>

* Set `VERSION` to `0.1.0-dev` in Makefile for `main` branch (#207)

As discussed in [1], it would make sense to use different `VERSION` on `main` and release branches.

[1] #200 (comment)

* Fix tags for images built for main and release branches (#208)

As discussed in [1], this would allow to run `make deploy` out of the box, as the image corresponding to the VERSION in Makefile would be present.

[1] #200 (comment)

* Replace operator API group janus-idp.io with rhdh.redhat.com (#201)

* Replace operator API group janus-idp.io with rhdh.redhat.com

* change to use module redhat-developer/red-hat-developer-hub-operator

* Remove files that were checked in by mistake

* Update examples/rhdh-cr.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update examples/rhdh-cr-with-app-configs.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update config/manifests/bases/backstage-operator.clusterserviceversion.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

---------

Co-authored-by: Armel Soro <armel@rm3l.org>

* Add warning note in install docs about OpenShift clusters with hosted control planes

* Fix diff computation for PR container builds

If a PR branch contained several commits but its HEAD had changes to some files not relevant for container build, the no image would be built completely for that PR

* Fix generated CSV (#212)

* Set `VERSION` to `0.2.0` in Makefile for `main` branch (#213)

It makes sense to align to the product version at this time:

```
upstream main ==  0.2.0
upstream 1.1.x branch == 0.1.0
downstream rhdh-1-rhel-9 branch == 1.2.0
downstream rhdh-1.1-rhel-9 branch == 1.1.0
```

* Fix typo (#214)

Signed-off-by: Moti Asayag <masayag@redhat.com>

* update dependencies (#215)

* update dependencies

Signed-off-by: Kim Tsao <ktsao@redhat.com>

* address review comments

Signed-off-by: Kim Tsao <ktsao@redhat.com>

---------

Signed-off-by: Kim Tsao <ktsao@redhat.com>

* [ci skip] chore: enable renovate for dockerfile and golang updates (#216)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore(deps): update actions/cache action to v4 (#220)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update docker/login-action action to v3 (#223)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update actions/github-script action to v7 (#222)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* bump dockerfiles per renovate bot PR #219 (#224)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* chore: enable digest pinning and major updates in dockerfiles; attempt to split go and docker into separate updates (different branch prefixes) (#225)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Update renovate.json - remove non-working code (#227)

* Update renovate.json - don't pin digests in dockerfile as it creates something that skopeo can't read (and likely breaks OSBS) (#230)

* chore(deps): pin dependencies (#228)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update github/codeql-action digest to 47b3d88 (#234)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* feat(seamless) chore: add `skipranges` and `replaces` logic TODOs to CSV (#231)

* feat(seamless) chore: add skipranges and replaces logic TODOs, which we can enable when 0.1 and 1.1 are live
alternatively, we could enable this sooner but then to install 1.2 you have to FIRST install 1.1, etc.

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* apply same change to config/manifests/bases/backstage-operator.clusterserviceversion.yaml

Signed-off-by: Nick Boldt <nboldt@redhat.com>

---------

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Add E2E tests using our examples against real clusters (#204)

* Add E2E tests against our examples on real clusters

- Do not error out when deleting a non-existing namespace

- Stream command output to the GinkgoWriter in real-time as well

This allows following what happens when calling potentially long-running commands

- Implement airgap test mode

- Ignore error when creating a namespace that already exists

- Allow to use existing mirror registry in airgap scenario

- Extract constants for test modes

- Add documentation

- Find an easier way to determine the IMG variable, using the Makefile

- Add more examples to README.md

- Add note about clusters with hosted control planes

- Support k3d clusters

- Support Minikube clusters

- Load image into local clusters using an archive instead

This allows this logic to be agnostic to the container engine used to
build the image. We rely on the container image to export the image to
an archive ('{podman,docker} image save').

- Run E2E test nightly on main and release branch

* Try running E2E tests on PRs by leveraging the already built operator image

* Revert "Try running E2E tests on PRs by leveraging the already built operator image"

This reverts commit fc87e04.

* Check if image exists locally before trying to export an archive

If not, try to pull it automatically.
This would avoid having to manually pull it.

* Update README.md

Co-authored-by: Gennady Azarenkov <gazarenkov@gmail.com>

* Ignore gosec warnings in test code

Those are not used in production

* Clarify in README that a connection to a cluster in the current kubeconfig is needed

* Increase timeout when waiting for controller to be up

On fresh clusters, 1 minute might be too short

* fixup! Clarify in README that a connection to a cluster in the current kubeconfig is needed

---------

Co-authored-by: Gennady Azarenkov <gazarenkov@gmail.com>

* chore(deps): pin actions/checkout action to b4ffde6 (#235)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update docker/setup-buildx-action digest to 0d103c3 (#239)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix: increase default size of the dynamic-plugins-root volume from 1Gi to 2Gi (#238)

* fix: increase default size of the dynamic-plugins-root volume from 1Gi to 2Gi

This applies the same fix done in the Helm Chart [1].

As depicted in [2], the init container might fail with insufficient
space error:
```
======= Installing dynamic plugin ./dynamic-plugins/dist/backstage-plugin-scaffolder-backend-module-github-dynamic
==> Grabbing package archive through `npm pack`
 Traceback (most recent call last):
  File "/opt/app-root/src/install-dynamic-plugins.py", line 304, in <module> main()
   File "/opt/app-root/src/install-dynamic-plugins.py", line 230, in main
    raise InstallException(f'Error while installing plugin \{ package } with \'npm pack\' : ' + completed.stderr.decode('utf-8')) __main__.InstallException: Error while installing plugin /opt/app-root/src/dynamic-plugins/dist/backstage-plugin-scaffolder-backend-module-github-dynamic with 'npm pack' : npm notice npm notice New major version of npm available! 9.8.1 -> 10.4.0 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.4.0> npm notice Run `npm install -g npm@10.4.0` to update! npm notice npm ERR! code ENOSPC npm ERR! syscall open npm ERR! path /dynamic-plugins-root/backstage-plugin-scaffolder-backend-module-github-dynamic-0.2.0-next.3.tgz npm ERR! errno -28 npm ERR! nospc ENOSPC: no space left on device, open '/dynamic-plugins-root/backstage-plugin-scaffolder-backend-module-github-dynamic-0.2.0-next.3.tgz' npm ERR! nospc There appears to be insufficient space on your system to finish. npm ERR! nospc Clear up some disk space and try again.
```

[1] redhat-developer/rhdh-chart#5
[2] https://issues.redhat.com/browse/RHIDP-1332

* Add test

* chore: RHIDP-1105 fix bundle annotations to be version agnostic; transform downstream (#244)

Signed-off-by: Nick Boldt <nboldt@redhat.com>

* Generate deployment manifest (#242)

* remove hardcoded images

* fix image

* Update examples/janus-cr-with-app-configs.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* change lookup

* Update config/manager/default-config/db-statefulset.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update config/manager/default-config/deployment.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* change lookup

* change lookup

* Update config/manager/default-config/deployment.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* add generated files

* fix image

* fix service raw config

* operator-script

* Update Makefile

Co-authored-by: Armel Soro <armel@rm3l.org>

* fix

* Apply suggestions from code review

---------

Co-authored-by: Armel Soro <armel@rm3l.org>

* chore: RHIDP-1105 switch annotations.yaml back to use fast channels; clean up comments (#246)

* chore: RHIDP-1105 switch annotations.yaml back to use fast channels

Signed-off-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>

* clean up comments

Signed-off-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>

---------

Signed-off-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>
Co-authored-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>

* chore(deps): update actions/cache digest to ab5e6d0 (#248)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update github/codeql-action digest to 8a470fd (#247)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Auto-push bundle manifests changes to PR branch if needed (#195)

* Make PR checks fail if bundle or manifests are not up-to-date

This is so that PR authors do not forget to regenerate those manifests.

* Update developer guide

* Save diff as patch file, so it can be downloaded and applied with Git

* Fix step names in PR Validation job

* Apply suggestions from code review

Co-authored-by: Jianrong Zhang <jianrzha@redhat.com>

* Do not error out if bundle manifests are outdated

Display warnings instead.
Also comment on the PR so that authors/reviewers are aware of that fact.

Co-authored-by: Gennady Azarenkov <gazarenkov@gmail.com>

* Update .github/workflows/pr.yaml

Co-authored-by: Nick Boldt <nboldt@redhat.com>

* Revert "Do not error out if bundle manifests are outdated"

This reverts commit ab2c12a.

* Auto-push any changes to the bundle manifests

This will alleviate the burden on contributors and maintainers.

* Run bundle diff checker in separate workflow triggered on 'pull_request_target' events

This is required to be able to write to fork PR branches

Similar to what we do already with the pull_request_target workflows, we also require manual authorization for unknown external forks, to prevent PWN requests

* Update PR template to think about eventually updating the rhdh-operator.csv.yaml file

* Update .github/workflows/pr-bundle-diff-checks.yaml

* Update docs/developer.md

Co-authored-by: Gennady Azarenkov <gazarenkov@redhat.com>

---------

Co-authored-by: Jianrong Zhang <jianrzha@redhat.com>
Co-authored-by: Gennady Azarenkov <gazarenkov@gmail.com>
Co-authored-by: Nick Boldt <nboldt@redhat.com>
Co-authored-by: Gennady Azarenkov <gazarenkov@redhat.com>

* chore(CI): Fix PR Bundle diff checker GH workflow

* chore(deps): pin dependencies (#249)

* chore(deps): pin dependencies

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Regenerate bundle manifests

Co-authored-by: renovate[bot] <renovate[bot]@users.noreply.github.com>

---------

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <renovate[bot]@users.noreply.github.com>

* fix(deps): update k8s.io/utils digest to e7106e6 (#232)

* fix(deps): update k8s.io/utils digest to e7106e6

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Replace deprecated usage of "k8s.io/utils/pointer" with "k8s.io/utils/ptr"

---------

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* chore(deps): update docker/build-push-action digest to af5a7ed (#250)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update k8s.io/utils digest to 4693a02 (#253)

* fix(deps): update k8s.io/utils digest to 4693a02

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Clean-up go.sum with 'go mod tidy'

---------

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* layered

* layered

* chore(deps): update actions/checkout digest to 9bb5618 (#255)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update actions/checkout digest to b4ffde6 (#256)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update github/codeql-action digest to 3ab4101 (#257)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Bump Ginkgo to v2.16.0 (#251)

* chore(deps): update docker/login-action digest to e92390c (#258)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update docker/build-push-action digest to 2cdde99 (#259)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update docker/setup-buildx-action digest to 2b51285 (#260)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update all non-major dependencies (#233)

* fix(deps): update all non-major dependencies

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix breaking changes from sigs.k8s.io/controller-runtime update

- `ctrl.Options#MetricsBindAddress` (TCP address that the controller should bind to for serving prometheus metrics) was deprecated and has been replaced with `metricsserver.Options#BindAddress` (in a `Metrics` struct) [1]
- `crl.Options#Port` (port that the webhook server serves at) was deprecated and has been replaced with `webhook.Options#Port` (in a `WebhookServer` field) [2]

[1] kubernetes-sigs/controller-runtime@e59161e#diff-d500fbd6a2aa620607ca5e2a7c3ac4f1a4c82309d1a549561e92abfcb18f2f0eL222-L225
[2] kubernetes-sigs/controller-runtime@e92eadb#diff-d500fbd6a2aa620607ca5e2a7c3ac4f1a4c82309d1a549561e92abfcb18f2f0eL282-L286

---------

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* fix(deps): update github.com/openshift/api digest to 4caef7f (#229)

* fix(deps): update github.com/openshift/api digest to 4caef7f

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Tidy up dependencies with 'go mod tidy'

---------

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* gomod

* gomod

* nextv2

* Regenerate bundle manifests

Co-authored-by: gazarenkov <gazarenkov@users.noreply.github.com>

* fix lint

* fix lint

* fix sonar issues

* fix minor sonar issues

* fix e2e tests

* fix e2e and add external db secret test

* small fixes

* small fixes

* merge

* Regenerate bundle manifests

Co-authored-by: gazarenkov <gazarenkov@users.noreply.github.com>

* Update examples/rhdh-cr-with-app-configs.yaml

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update Makefile

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update Makefile

---------

Signed-off-by: Nick Boldt <nboldt@redhat.com>
Signed-off-by: Moti Asayag <masayag@redhat.com>
Signed-off-by: Kim Tsao <ktsao@redhat.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>
Co-authored-by: Nick Boldt <nboldt@redhat.com>
Co-authored-by: Jianrong Zhang <jianrongzhang89@gmail.com>
Co-authored-by: Tomas Kral <tomas.kral@gmail.com>
Co-authored-by: Armel Soro <asoro@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
Co-authored-by: Moti Asayag <masayag@redhat.com>
Co-authored-by: Kim Tsao <84398375+kim-tsao@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: RHDH Build (rhdh-bot) <rhdh-bot@redhat.com>
Co-authored-by: Jianrong Zhang <jianrzha@redhat.com>
Co-authored-by: Gennady Azarenkov <gazarenkov@redhat.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <renovate[bot]@users.noreply.github.com>
Co-authored-by: gazarenkov <gazarenkov@users.noreply.github.com>
  • Loading branch information
15 people committed Apr 3, 2024
1 parent 496da58 commit 0a8f41f
Show file tree
Hide file tree
Showing 100 changed files with 5,473 additions and 1,952 deletions.
17 changes: 12 additions & 5 deletions Makefile
Expand Up @@ -131,6 +131,12 @@ test: manifests generate fmt vet envtest ## Run tests. We need LOCALBIN=$(LOCALB
mkdir -p $(LOCALBIN)/default-config && cp config/manager/$(CONF_DIR)/* $(LOCALBIN)/default-config
LOCALBIN=$(LOCALBIN) KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" go test $(PKGS) -coverprofile cover.out

.PHONY: integration-test
integration-test: ginkgo manifests generate fmt vet envtest ## Run integration_tests. We need LOCALBIN=$(LOCALBIN) to get correct default-config path
mkdir -p $(LOCALBIN)/default-config && cp config/manager/$(CONF_DIR)/* $(LOCALBIN)/default-config
LOCALBIN=$(LOCALBIN) KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" $(GINKGO) -v -r integration_tests


##@ Build

.PHONY: build
Expand Down Expand Up @@ -195,6 +201,12 @@ deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
$(KUSTOMIZE) build config/default | kubectl apply -f -

.PHONY: deployment-manifest
deployment-manifest: manifests kustomize ## Generate manifest to deploy operator.
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
$(KUSTOMIZE) build config/default > rhdh-operator-${VERSION}.yaml
@echo "Generated operator script rhdh-operator-${VERSION}.yaml"

.PHONY: undeploy
undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
$(KUSTOMIZE) build config/default | kubectl delete --ignore-not-found=$(ignore-not-found) -f -
Expand Down Expand Up @@ -402,9 +414,4 @@ show-img:
show-container-engine:
@echo -n $(CONTAINER_ENGINE)

.PHONY: deployment-manifest
deployment-manifest: manifests kustomize ## Generate manifest to deploy operator.
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
$(KUSTOMIZE) build config/default > rhdh-operator-${VERSION}.yaml
@echo "Generated operator script rhdh-operator-${VERSION}.yaml"

2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -62,7 +62,7 @@ Output:

## License

Copyright 2023 Red Hat Inc..
Copyright 2023 Red Hat Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
58 changes: 41 additions & 17 deletions api/v1alpha1/backstage_types.go
Expand Up @@ -16,27 +16,38 @@ package v1alpha1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/utils/ptr"
)

// Constants for status conditions
type BackstageConditionReason string

type BackstageConditionType string

const (
// TODO: RuntimeConditionRunning string = "RuntimeRunning"
ConditionDeployed string = "Deployed"
DeployOK string = "DeployOK"
DeployFailed string = "DeployFailed"
DeployInProgress string = "DeployInProgress"
BackstageConditionTypeDeployed BackstageConditionType = "Deployed"

BackstageConditionReasonDeployed BackstageConditionReason = "Deployed"
BackstageConditionReasonFailed BackstageConditionReason = "DeployFailed"
BackstageConditionReasonInProgress BackstageConditionReason = "DeployInProgress"
)

// BackstageSpec defines the desired state of Backstage
type BackstageSpec struct {
// Configuration for Backstage. Optional.
Application *Application `json:"application,omitempty"`

// Raw Runtime Objects configuration. For Advanced scenarios.
RawRuntimeConfig RuntimeConfig `json:"rawRuntimeConfig,omitempty"`
// Raw Runtime RuntimeObjects configuration. For Advanced scenarios.
RawRuntimeConfig *RuntimeConfig `json:"rawRuntimeConfig,omitempty"`

// Configuration for database access. Optional.
Database Database `json:"database,omitempty"`
Database *Database `json:"database,omitempty"`
}

type RuntimeConfig struct {
// Name of ConfigMap containing Backstage runtime objects configuration
BackstageConfigName string `json:"backstageConfig,omitempty"`
// Name of ConfigMap containing LocalDb (PostgreSQL) runtime objects configuration
LocalDbConfigName string `json:"localDbConfig,omitempty"`
}

type Database struct {
Expand Down Expand Up @@ -98,7 +109,7 @@ type Application struct {

// Image Pull Secrets to use in all containers (including Init Containers)
// +optional
ImagePullSecrets *[]string `json:"imagePullSecrets,omitempty"`
ImagePullSecrets []string `json:"imagePullSecrets,omitempty"`

// Route configuration. Used for OpenShift only.
Route *Route `json:"route,omitempty"`
Expand Down Expand Up @@ -177,13 +188,6 @@ type Env struct {
Value string `json:"value"`
}

type RuntimeConfig struct {
// Name of ConfigMap containing Backstage runtime objects configuration
BackstageConfigName string `json:"backstageConfig,omitempty"`
// Name of ConfigMap containing LocalDb (PostgreSQL) runtime objects configuration
LocalDbConfigName string `json:"localDbConfig,omitempty"`
}

// BackstageStatus defines the observed state of Backstage
type BackstageStatus struct {
// Conditions is the list of conditions describing the state of the runtime
Expand Down Expand Up @@ -268,3 +272,23 @@ type TLS struct {
func init() {
SchemeBuilder.Register(&Backstage{}, &BackstageList{})
}

// IsLocalDbEnabled returns true if Local database is configured and enabled
func (s *BackstageSpec) IsLocalDbEnabled() bool {
if s.Database == nil {
return true
}
return ptr.Deref(s.Database.EnableLocalDb, true)
}

// IsRouteEnabled returns value of Application.Route.Enabled if defined or true by default
func (s *BackstageSpec) IsRouteEnabled() bool {
if s.Application != nil && s.Application.Route != nil {
return ptr.Deref(s.Application.Route.Enabled, true)
}
return true
}

func (s *BackstageSpec) IsAuthSecretSpecified() bool {
return s.Database != nil && s.Database.AuthSecretName != ""
}
20 changes: 12 additions & 8 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

95 changes: 60 additions & 35 deletions bundle/manifests/backstage-default-config_v1_configmap.yaml
@@ -1,28 +1,33 @@
apiVersion: v1
data:
backend-auth-configmap.yaml: |
app-config.yaml: |-
apiVersion: v1
kind: ConfigMap
metadata:
name: # placeholder for '<cr-name>-backend-auth'
name: my-backstage-config-cm1 # placeholder for <bs>-default-appconfig
data:
"app-config.backend-auth.default.yaml": |
default.app-config.yaml: |
backend:
database:
connection:
password: ${POSTGRES_PASSWORD}
user: ${POSTGRES_USER}
auth:
keys:
# This is a default value, which you should change by providing your own app-config
- secret: "pl4s3Ch4ng3M3"
db-secret.yaml: |
db-secret.yaml: |-
apiVersion: v1
kind: Secret
metadata:
name: # placeholder for 'backstage-psql-secret-<cr-name>'
stringData:
"POSTGRES_PASSWORD": "rl4s3Fh4ng3M4" # default value, change to your own value
"POSTGRES_PORT": "5432"
"POSTGRES_USER": "postgres"
"POSTGRESQL_ADMIN_PASSWORD": "rl4s3Fh4ng3M4" # default value, change to your own value
"POSTGRES_HOST": "" # set to your Postgres DB host. If the local DB is deployed, set to 'backstage-psql-<cr-name>'
name: postgres-secrets # will be replaced
type: Opaque
#stringData:
# POSTGRES_PASSWORD:
# POSTGRES_PORT: "5432"
# POSTGRES_USER: postgres
# POSTGRESQL_ADMIN_PASSWORD: admin123
# POSTGRES_HOST: bs1-db-service #placeholder <crname>-db-service
db-service-hl.yaml: |
apiVersion: v1
kind: Service
Expand All @@ -44,7 +49,7 @@ data:
rhdh.redhat.com/app: backstage-psql-cr1 # placeholder for 'backstage-psql-<cr-name>'
ports:
- port: 5432
db-statefulset.yaml: |
db-statefulset.yaml: |-
apiVersion: apps/v1
kind: StatefulSet
metadata:
Expand All @@ -62,6 +67,10 @@ data:
rhdh.redhat.com/app: backstage-psql-cr1 # placeholder for 'backstage-psql-<cr-name>'
name: backstage-db-cr1 # placeholder for 'backstage-psql-<cr-name>'
spec:
# fsGroup does not work for Openshift
# AKS/EKS does not work w/o it
#securityContext:
# fsGroup: 26
automountServiceAccountToken: false
## https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
## The optional .spec.persistentVolumeClaimRetentionPolicy field controls if and how PVCs are deleted during the lifecycle of a StatefulSet.
Expand All @@ -77,13 +86,12 @@ data:
value: /var/lib/pgsql/data
- name: PGDATA
value: /var/lib/pgsql/data/userdata
envFrom:
- secretRef:
name: <POSTGRESQL_SECRET> # will be replaced with 'backstage-psql-secrets-<cr-name>'
# image will be replaced by the value of the `RELATED_IMAGE_postgresql` env var, if set
image: quay.io/fedora/postgresql-15:latest
image: quay.io/fedora/postgresql-15:latest # will be replaced with the actual image
imagePullPolicy: IfNotPresent
securityContext:
# runAsUser:26 does not work for Openshift but looks work for AKS/EKS
# runAsUser: 26
runAsGroup: 0
runAsNonRoot: true
allowPrivilegeEscalation: false
seccompProfile:
Expand Down Expand Up @@ -134,8 +142,6 @@ data:
- mountPath: /var/lib/pgsql/data
name: data
restartPolicy: Always
securityContext: {}
serviceAccount: default
serviceAccountName: default
volumes:
- emptyDir:
Expand All @@ -160,7 +166,7 @@ data:
apiVersion: apps/v1
kind: Deployment
metadata:
name: # placeholder for 'backstage-<cr-name>'
name: backstage # placeholder for 'backstage-<cr-name>'
spec:
replicas: 1
selector:
Expand All @@ -172,6 +178,11 @@ data:
rhdh.redhat.com/app: # placeholder for 'backstage-<cr-name>'
spec:
automountServiceAccountToken: false
# if securityContext not present in AKS/EKS, the error is like this:
#Error: EACCES: permission denied, open '/dynamic-plugins-root/backstage-plugin-scaffolder-backend-module-github-dynamic-0.2.2.tgz'
# fsGroup doesn not work for Openshift
#securityContext:
# fsGroup: 1001
volumes:
- ephemeral:
volumeClaimTemplate:
Expand All @@ -187,18 +198,19 @@ data:
defaultMode: 420
optional: true
secretName: dynamic-plugins-npmrc
initContainers:
- command:
- name: install-dynamic-plugins
command:
- ./install-dynamic-plugins.sh
- /dynamic-plugins-root
image: quay.io/janus-idp/backstage-showcase:latest # will be replaced with the actual image quay.io/janus-idp/backstage-showcase:next
imagePullPolicy: IfNotPresent
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
env:
- name: NPM_CONFIG_USERCONFIG
value: /opt/app-root/src/.npmrc.dynamic-plugins
# image will be replaced by the value of the `RELATED_IMAGE_backstage` env var, if set
image: quay.io/janus-idp/backstage-showcase:latest
imagePullPolicy: IfNotPresent
name: install-dynamic-plugins
volumeMounts:
- mountPath: /dynamic-plugins-root
name: dynamic-plugins-root
Expand All @@ -208,6 +220,9 @@ data:
subPath: .npmrc
workingDir: /opt/app-root/src
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 1000m
memory: 2.5Gi
Expand All @@ -220,6 +235,9 @@ data:
args:
- "--config"
- "dynamic-plugins-root/app-config.dynamic-plugins.yaml"
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
readinessProbe:
failureThreshold: 3
httpGet:
Expand All @@ -246,24 +264,22 @@ data:
env:
- name: APP_CONFIG_backend_listen_port
value: "7007"
envFrom:
- secretRef:
name: <POSTGRESQL_SECRET> # will be replaced with 'backstage-psql-secrets-<cr-name>'
# - secretRef:
# name: backstage-secrets
volumeMounts:
- mountPath: /opt/app-root/src/dynamic-plugins-root
name: dynamic-plugins-root
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 1000m
memory: 2.5Gi
ephemeral-storage: 5Gi
dynamic-plugins-configmap.yaml: |-
dynamic-plugins.yaml: |-
apiVersion: v1
kind: ConfigMap
metadata:
name: # placeholder for '<cr-name>-dynamic-plugins'
name: default-dynamic-plugins # must be the same as (deployment.yaml).spec.template.spec.volumes.name.dynamic-plugins-conf.configMap.name
data:
"dynamic-plugins.yaml": |
includes:
Expand All @@ -273,7 +289,7 @@ data:
apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: # placeholder for 'backstage-<cr-name>'
name: route # placeholder for 'backstage-<cr-name>'
spec:
port:
targetPort: http-backend
Expand All @@ -284,11 +300,20 @@ data:
to:
kind: Service
name: # placeholder for 'backstage-<cr-name>'
secret-envs.yaml: |
apiVersion: v1
kind: Secret
metadata:
name: backend-auth-secret
stringData:
# generated with the command below (from https://janus-idp.io/docs/auth/service-to-service-auth/#setup):
# node -p 'require("crypto").randomBytes(24).toString("base64")'
BACKEND_SECRET: "R2FxRVNrcmwzYzhhN3l0V1VRcnQ3L1pLT09WaVhDNUEK" # notsecret
service.yaml: |-
apiVersion: v1
kind: Service
metadata:
name: # placeholder for 'backstage-<cr-name>'
name: backstage # placeholder for 'backstage-<cr-name>'
spec:
type: ClusterIP
selector:
Expand Down

0 comments on commit 0a8f41f

Please sign in to comment.