enable ssl/tls for tekton results db connections #3109

avinal · 2024-01-22T08:23:13Z

Adds required YAML config for enabling db ssl connections
Uses public certificates for RDS available here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html#UsingWithRDS.SSL.RegionCertificates
Enables dbssl for staging environment
Includes Cherry-pick add config for enabling tekton results dbssl openshift-pipelines/pipeline-service#984

Signed-off-by: Avinal Kumar avinal@redhat.com

openshift-ci · 2024-01-22T08:23:19Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

enarha · 2024-03-18T10:02:25Z

Shouldn't you update the deploy.yaml files? This should only update the staging clusters right?

enarha · 2024-03-18T10:17:16Z

How this PR relates to #3436 ? Are they related? Is there an order to be merged?

avinal · 2024-03-18T12:20:29Z

Shouldn't you update the deploy.yaml files?

Ah I missed it, I will update.

This should only update the staging clusters right?

Yes

How this PR relates to #3436 ? Are they related? Is there an order to be merged?

Ideally they should be merged together so that there is no dependency on each other. Let us discuss this in daily.

gabemontero · 2024-03-18T12:43:59Z

Shouldn't you update the deploy.yaml files?

Ah I missed it, I will update.

This should only update the staging clusters right?

Yes

How this PR relates to #3436 ? Are they related? Is there an order to be merged?

Ideally they should be merged together so that there is no dependency on each other. Let us discuss this in daily.

yeah you need to pull the commit from #3436 into this PR

we also need to discuss how the cert is injected here

components/pipeline-service/base/certificates/tekton-results/tekton-results-db-ssl.yaml

gabemontero

/lgtm

gabemontero · 2024-04-01T13:52:25Z

components/monitoring/grafana/base/dashboards/pipeline-service/kustomization.yaml

@@ -2,4 +2,4 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
-  - https://github.com/openshift-pipelines/pipeline-service/operator/gitops/argocd/grafana/?ref=542db3fcc168c426d39dbed231a4230b101f8a2a
+  - https://github.com/openshift-pipelines/pipeline-service/operator/gitops/argocd/grafana/?ref=5e38cd44d8455e77def659ea116a684c5cfa4320


matches https://github.com/redhat-appstudio/infra-deployments/pull/3525/files

gabemontero · 2024-04-01T13:52:53Z

components/pipeline-service/development/kustomization.yaml

-  - https://github.com/openshift-pipelines/pipeline-service.git/developer/openshift/gitops/argocd/pipeline-service?ref=542db3fcc168c426d39dbed231a4230b101f8a2a
-  - https://github.com/openshift-pipelines/pipeline-service.git/developer/openshift/gitops/argocd/pipeline-service-storage?ref=542db3fcc168c426d39dbed231a4230b101f8a2a
+  - https://github.com/openshift-pipelines/pipeline-service.git/developer/openshift/gitops/argocd/pipeline-service?ref=5e38cd44d8455e77def659ea116a684c5cfa4320
+  - https://github.com/openshift-pipelines/pipeline-service.git/developer/openshift/gitops/argocd/pipeline-service-storage?ref=5e38cd44d8455e77def659ea116a684c5cfa4320


matches https://github.com/redhat-appstudio/infra-deployments/pull/3525/files

gabemontero · 2024-04-01T13:53:05Z

components/pipeline-service/staging/base/kustomization.yaml

@@ -8,11 +8,12 @@ commonAnnotations:
  argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true

 resources:
-  - https://github.com/openshift-pipelines/pipeline-service.git/operator/gitops/argocd/pipeline-service?ref=542db3fcc168c426d39dbed231a4230b101f8a2a
+  - https://github.com/openshift-pipelines/pipeline-service.git/operator/gitops/argocd/pipeline-service?ref=5e38cd44d8455e77def659ea116a684c5cfa4320


matches https://github.com/redhat-appstudio/infra-deployments/pull/3525/files

gabemontero · 2024-04-01T13:55:43Z

was about to ask @avinal if you ran https://github.com/redhat-appstudio/infra-deployments/blob/main/hack/generate-deploy-config.sh ... see you force-pushed something ..

gabemontero · 2024-04-01T13:56:58Z

/lgtm

avinal · 2024-04-01T13:57:25Z

Yes I ran the deploy.sh, you can see the changes in the respective deploy.yaml files. The kube lint was failing so I fixed that and pushed.

enarha · 2024-04-01T14:17:04Z

/lgtm

gabemontero · 2024-04-01T17:46:23Z

@avinal pipeline service is out of sync and degraded in the latest e2e run https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/redhat-appstudio_infra-deployments/3109/pull-ci-redhat-appstudio-infra-deployments-main-appstudio-e2e-tests/1774797750232158208

the api server deployment is not happy if you look at https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/redhat-appstudio_infra-deployments/3109/pull-ci-redhat-appstudio-infra-deployments-main-appstudio-e2e-tests/1774797750232158208/artifacts/appstudio-e2e-tests/gather-extra/artifacts/deployments.json :

            "status": {
                "conditions": [
                    {
                        "lastTransitionTime": "2024-04-01T14:05:51Z",
                        "lastUpdateTime": "2024-04-01T14:05:51Z",
                        "message": "Deployment does not have minimum availability.",
                        "reason": "MinimumReplicasUnavailable",
                        "status": "False",
                        "type": "Available"
                    },
                    {
                        "lastTransitionTime": "2024-04-01T14:15:52Z",
                        "lastUpdateTime": "2024-04-01T14:15:52Z",
                        "message": "ReplicaSet \"tekton-results-api-777f7bf5bd\" has timed out progressing.",
                        "reason": "ProgressDeadlineExceeded",
                        "status": "False",
                        "type": "Progressing"
                    }

for whatever reason I cannot find a watcher deployment.

nor do I see watcher pods listed at https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/redhat-appstudio_infra-deployments/3109/pull-ci-redhat-appstudio-infra-deployments-main-appstudio-e2e-tests/1774797750232158208/artifacts/appstudio-e2e-tests/gather-extra/artifacts/pods/

I suspect you will need to run an oc kustomize against the developer overlay and see what is happening wrt the missing watcher deployment

openshift-ci · 2024-04-02T06:03:36Z

New changes are detected. LGTM label has been removed.

openshift-ci · 2024-04-02T06:03:39Z

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: avinal, gabemontero

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

avinal · 2024-04-02T06:05:49Z

@gabemontero I figured out the issue. The certs generation was not added in infra-deployments, I was not aware of the test flow, so I missed it. Xin helped me understand it. Now I have added the cert generation, it is mostly copy-paste from the pipeline-service, but a quick look would be great.

- Adds required YAML config for enabling db ssl connections - Uses public certificates for RDS available here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html#UsingWithRDS.SSL.RegionCertificates - Enables dbssl for staging environment - Adds cert generation for development setup - Includes openshift-pipelines/pipeline-service#984 Signed-off-by: Avinal Kumar <avinal@redhat.com> rh-pre-commit.version: 2.2.0 rh-pre-commit.check-secrets: ENABLED

openshift-ci · 2024-04-03T12:42:16Z

@avinal: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/appstudio-e2e-tests	`0e3bab7`	link	true	`/test appstudio-e2e-tests`
ci/prow/appstudio-e2e-tests-ocp-4-14	`0e3bab7`	link	false	`/test appstudio-e2e-tests-ocp-4-14`
ci/prow/appstudio-hac-e2e-tests	`0e3bab7`	link	false	`/test appstudio-hac-e2e-tests`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

gabemontero · 2024-04-03T15:56:17Z

hack/secret-creator/create-plnsvc-secrets.sh

@@ -5,6 +5,7 @@ main() {
    create_namespace
    create_db_secret
    create_s3_secret
+    create_db_cert_secret_and_configmap


ok so the analogous items here like the db and s3 secrets are pulled from vault and external secrets for stage and prod @avinal , which is why it is OK that this script creates them for the developer overlay

but there is no analogous vault / external secret item for this public cert

so even though things are not right as I type, even if we got them right, this would only work for the developer overlay and not stage/prod

I think the solution @scoheb outlined in the email you sent, where it adds the public cert to the system root CA when building the image, is the way to go, at least short term, since we have a Dockerfile we can update in the results downstream fork.

Now, when we move to the openshift pipelines operator version of results, we'll have to see if that same approach works ...i.e. is there a Dockerfile somewhere in the CPaaS pipeline that would could do the analogous thing for.

From @scoheb email:

https://github.com/redhat-appstudio/managed-gitops/blob/main/Dockerfile#L49

https://github.com/redhat-appstudio/segment-bridge/blob/22e23754649e150239d5e86f40dac7f99456d8e6/data/ca-trust/2022-IT-Root-CA.pem#L2

/hold

while this script ^^ has progressed us along, I don't think that can be the final solution.

@enarha FYI

I realized that such a script could be moved to a Job like what @Roming22 did for the chains secret over in https://github.com/openshift-pipelines/pipeline-service/blob/main/operator/gitops/argocd/pipeline-service/openshift-pipelines/chains-secrets-config.yaml#L46-L111

Now, I also know @enarha is hoping to rework how we do the chains secret (though I don't remember for sure if that rework included removing this Job)

I am going through the mails and the threads. To summarize:

The RDS public cert can be added to the Dockerfile itself, this is how we are already doing

For development, we may want to use a Job, as we are doing in chains.

If that is all, we can proceed with the discussion.

gabemontero · 2024-04-03T17:08:14Z

the watcher and api server now come up with these changes, but the postgresql gitops application stays out of sync because of the cert related changes

I'll attach a screen shot

per discussion in email and slack, and with this solution not working for stage/prod, a pivot to changing the api server image to add the amazon cert to the root SA per pointer from @scoheb appears the be the path we need to take for the time being

/hold

openshift-ci bot added the do-not-merge/work-in-progress label Jan 22, 2024

avinal force-pushed the avinal/enable-dbssl-infra branch from 1d20dcb to 07def20 Compare March 18, 2024 06:54

avinal marked this pull request as ready for review March 18, 2024 06:55

openshift-ci bot removed the do-not-merge/work-in-progress label Mar 18, 2024

openshift-ci bot requested review from adambkaplan and enarha March 18, 2024 06:55

avinal mentioned this pull request Mar 18, 2024

pipeline-service update #3436

Merged

gabemontero reviewed Mar 18, 2024

View reviewed changes

components/pipeline-service/base/certificates/tekton-results/tekton-results-db-ssl.yaml Show resolved Hide resolved

avinal force-pushed the avinal/enable-dbssl-infra branch from 07def20 to 3cea633 Compare April 1, 2024 13:44

avinal mentioned this pull request Apr 1, 2024

pipeline-service update #3525

Closed

avinal force-pushed the avinal/enable-dbssl-infra branch from 3cea633 to 259f289 Compare April 1, 2024 13:51

gabemontero approved these changes Apr 1, 2024

View reviewed changes

openshift-ci bot assigned gabemontero Apr 1, 2024

openshift-ci bot added the lgtm label Apr 1, 2024

gabemontero added the approved label Apr 1, 2024

avinal force-pushed the avinal/enable-dbssl-infra branch from 259f289 to 5acc723 Compare April 1, 2024 13:55

openshift-ci bot removed the lgtm label Apr 1, 2024

openshift-ci bot added the lgtm label Apr 1, 2024

openshift-ci bot assigned enarha Apr 1, 2024

avinal force-pushed the avinal/enable-dbssl-infra branch from 5acc723 to 97b5194 Compare April 2, 2024 06:03

openshift-ci bot removed the lgtm label Apr 2, 2024

avinal force-pushed the avinal/enable-dbssl-infra branch 2 times, most recently from 1ffe89f to 7e502d7 Compare April 3, 2024 06:53

avinal force-pushed the avinal/enable-dbssl-infra branch from 7e502d7 to 0e3bab7 Compare April 3, 2024 09:38

gabemontero mentioned this pull request Apr 3, 2024

pipeline-service-exporter update openshift-pipelines/pipeline-service#973

Merged

gabemontero reviewed Apr 3, 2024

View reviewed changes

openshift-ci bot added the do-not-merge/hold label Apr 3, 2024

gabemontero mentioned this pull request Apr 3, 2024

Revert "add config for enabling tekton results dbssl" openshift-pipelines/pipeline-service#986

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enable ssl/tls for tekton results db connections #3109

enable ssl/tls for tekton results db connections #3109

avinal commented Jan 22, 2024 •

edited

openshift-ci bot commented Jan 22, 2024

enarha commented Mar 18, 2024

enarha commented Mar 18, 2024 •

edited

avinal commented Mar 18, 2024

gabemontero commented Mar 18, 2024

gabemontero left a comment

gabemontero Apr 1, 2024

gabemontero Apr 1, 2024

gabemontero Apr 1, 2024

gabemontero commented Apr 1, 2024

gabemontero commented Apr 1, 2024

avinal commented Apr 1, 2024

enarha commented Apr 1, 2024

gabemontero commented Apr 1, 2024

openshift-ci bot commented Apr 2, 2024

openshift-ci bot commented Apr 2, 2024

avinal commented Apr 2, 2024

openshift-ci bot commented Apr 3, 2024

gabemontero Apr 3, 2024

gabemontero Apr 3, 2024

gabemontero Apr 9, 2024

avinal Apr 9, 2024

gabemontero commented Apr 3, 2024 •

edited

enable ssl/tls for tekton results db connections #3109

Are you sure you want to change the base?

enable ssl/tls for tekton results db connections #3109

Conversation

avinal commented Jan 22, 2024 • edited

openshift-ci bot commented Jan 22, 2024

enarha commented Mar 18, 2024

enarha commented Mar 18, 2024 • edited

avinal commented Mar 18, 2024

gabemontero commented Mar 18, 2024

gabemontero left a comment

Choose a reason for hiding this comment

gabemontero Apr 1, 2024

Choose a reason for hiding this comment

gabemontero Apr 1, 2024

Choose a reason for hiding this comment

gabemontero Apr 1, 2024

Choose a reason for hiding this comment

gabemontero commented Apr 1, 2024

gabemontero commented Apr 1, 2024

avinal commented Apr 1, 2024

enarha commented Apr 1, 2024

gabemontero commented Apr 1, 2024

openshift-ci bot commented Apr 2, 2024

openshift-ci bot commented Apr 2, 2024

avinal commented Apr 2, 2024

openshift-ci bot commented Apr 3, 2024

gabemontero Apr 3, 2024

Choose a reason for hiding this comment

gabemontero Apr 3, 2024

Choose a reason for hiding this comment

gabemontero Apr 9, 2024

Choose a reason for hiding this comment

avinal Apr 9, 2024

Choose a reason for hiding this comment

gabemontero commented Apr 3, 2024 • edited

avinal commented Jan 22, 2024 •

edited

enarha commented Mar 18, 2024 •

edited

gabemontero commented Apr 3, 2024 •

edited