Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildah*: add BUILD_ARGS param #1023

Merged
merged 6 commits into from
May 21, 2024
Merged

Conversation

chmeliik
Copy link
Contributor

@chmeliik chmeliik commented May 16, 2024

STONEBLD-2462

Add a BUILD_ARGS array parameter to allow users to pass build args
directly, not just via BUILD_ARGS_FILE.

This provides greater flexibility. Those who just need static build args
can commit a file into their repo and use BUILD_ARGS_FILE. Those who
need dynamic values can create a custom task and incorporate its return
value into the BUILD_ARGS array.


Also:

  • generate-readme: handle array param defaults
  • buildah: re-generate README

Copy link
Member

@arewm arewm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we expose this parameter in the pipeline definitions that we produce?

task/buildah-remote/0.1/buildah-remote.yaml Outdated Show resolved Hide resolved
@chmeliik
Copy link
Contributor Author

Tested in my acmiel-tenant namespace with

oc create -f test-pipelinerun.yaml
test-pipelinerun.yaml (click me)
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  generateName: acmiel-test-
spec:
  pipelineSpec:
    tasks:
    - name: build-container
      params:
      - name: IMAGE
        value: quay.io/redhat-user-workloads/acmiel-tenant/build-service/build-service:test_buildah_build_args
      - name: DOCKERFILE
        value: https://gist.githubusercontent.com/chmeliik/920ee8dc43fd16927996f9f7c597fa56/raw/503729b236fd7ad1a2a97c5118e26016b46cd8af/Containerfile
      - name: CONTEXT
        value: .
      - name: BUILD_ARGS
        value:
          - "THIS_IS_A_LABEL_WITH_SOME_NEWLINES=Hello there!\nGeneral Kenobi."
          - |
            THIS_IS_A_WHOLE_PYTHON_SCRIPT=
            import this

            print()
            print("This was the Zen of Python, brought to you by a silly container build.")
          - |
            AND_THIS_IS_not_SHELL_INJECTION=
            for i in $(seq 1 10); do
              echo "Trying to come up with something funny, attempt $i"
            done
            echo "Failed"
      taskRef:
        params:
        - name: url
          value: https://github.com/chmeliik/build-definitions.git
        - name: revision
          value: build-args-param
        - name: pathInRepo
          value: task/buildah/0.1/buildah.yaml
        resolver: git
      workspaces:
      - name: source
        workspace: workspace

    workspaces:
    - name: workspace

  workspaces:
  - name: workspace
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
Relevant part of the logs (see that the build args got to the buildah build unscathed)
Fetch Dockerfile from https://gist.githubusercontent.com/chmeliik/920ee8dc43fd16927996f9f7c597fa56/raw/503729b236fd7ad1a2a97c5118e26016b46cd8af/Containerfile
Adding the entitlement to the build
STEP 1/9: FROM docker.io/library/python:3.12-alpine
Trying to pull docker.io/library/python:3.12-alpine...
Getting image source signatures
Copying blob sha256:c3cdf40b8bda8e4ca4be0f5fa7f1d128907271efcbc72cbfc7c8b0f939ec25ea
Copying blob sha256:60d2faee92e78fe7518f0ff1645cd7320bf6b140ff885fdec2a1ea1d878f0dca
Copying blob sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8
Copying blob sha256:b62713ed4827911d38bb5a9ac322efa0408b4bb135863b4b15c4bc383e59918b
Copying blob sha256:3a6cecfe70039fd21206783553d33ea4753700f031a2490428311619801d02f7
Copying config sha256:f44387b482817f41bdac1892c45711adaedb3a7dd381844cdc3f360e66314d7a
Writing manifest to image destination
STEP 2/9: ARG THIS_IS_A_LABEL_WITH_SOME_NEWLINES
STEP 3/9: ARG THIS_IS_A_WHOLE_PYTHON_SCRIPT
STEP 4/9: ARG AND_THIS_IS_not_SHELL_INJECTION
STEP 5/9: LABEL acmiel.test/foo=$THIS_IS_A_LABEL_WITH_SOME_NEWLINES
STEP 6/9: RUN python -c "$THIS_IS_A_WHOLE_PYTHON_SCRIPT"
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

This was the Zen of Python, brought to you by a silly container build.
STEP 7/9: RUN sh -c "$AND_THIS_IS_not_SHELL_INJECTION"
Trying to come up with something funny, attempt 1
Trying to come up with something funny, attempt 2
Trying to come up with something funny, attempt 3
Trying to come up with something funny, attempt 4
Trying to come up with something funny, attempt 5
Trying to come up with something funny, attempt 6
Trying to come up with something funny, attempt 7
Trying to come up with something funny, attempt 8
Trying to come up with something funny, attempt 9
Trying to come up with something funny, attempt 10
Failed
STEP 8/9: CMD ["echo", "Hello there!"]
STEP 9/9: LABEL "build-date"="2024-05-17T10:43:45" "architecture"="x86_64" "vcs-type"="git"
COMMIT quay.io/redhat-user-workloads/acmiel-tenant/build-service/build-service:test_buildah_build_args

Also

$ skopeo inspect --config docker://quay.io/redhat-user-workloads/acmiel-tenant/build-service/build-service:test_buildah_build_args | jq .config.Labels
{
  "acmiel.test/foo": "Hello there!\nGeneral Kenobi.",
  "architecture": "x86_64",
  "build-date": "2024-05-17T10:43:45",
  "io.buildah.version": "1.31.0",
  "vcs-type": "git"
}

@chmeliik
Copy link
Contributor Author

Should we expose this parameter in the pipeline definitions that we produce?

Added 👍

Also required fixing the generate-pipelines-readme script to handle arrays

@chmeliik chmeliik marked this pull request as ready for review May 17, 2024 11:44
Copy link
Contributor Author

@chmeliik chmeliik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildah[-remote|rhtap]: add BUILD_ARGS param

Dang it, forgot about buildah-oci-ta.

image

STONEBLD-2462

Add a BUILD_ARGS array parameter to allow users to pass build args
directly, not just via BUILD_ARGS_FILE.

This provides greater flexibility. Those who just need static build args
can commit a file into their repo and use BUILD_ARGS_FILE. Those who
need dynamic values can create a custom task and incorporate its return
value into the BUILD_ARGS array.

Also:
- quote a few unquoted array expansions which will otherwise break in
  hilarious ways
- improve the description of the BUILD_ARGS_FILE param

Signed-off-by: Adam Cmiel <acmiel@redhat.com>
Previously, given an array param with a default value, the script would
fail with

    Error: !!seq cannot be added to a !!str

Convert array params to string (the JSON representation of the array).

Also break up the huge line into multiple lines.

Signed-off-by: Adam Cmiel <acmiel@redhat.com>
If you're wondering why the READMEs were not regenerated for
buildah-remote and buildah-rhtap:
- buildah-remote has a manually modified README
- buildah-rhtap doesn't have one

Signed-off-by: Adam Cmiel <acmiel@redhat.com>
STONEBLD-2462

Exposes the BUILD_ARGS param of the buildah task.

Also improve the description of the build-args-file param.

Signed-off-by: Adam Cmiel <acmiel@redhat.com>
Also:
- fix wrong escape sequences in regex strings
- slightly improve param matching (previously, the script would only
  match param uses where the param is not part of a larger string)

Signed-off-by: Adam Cmiel <acmiel@redhat.com>
Signed-off-by: Adam Cmiel <acmiel@redhat.com>
@chmeliik chmeliik changed the title buildah[-remote|rhtap]: add BUILD_ARGS param buildah*: add BUILD_ARGS param May 17, 2024
Copy link

sonarcloud bot commented May 17, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@chmeliik
Copy link
Contributor Author

/retest

missed the failure

@chmeliik
Copy link
Contributor Author

/retest

@chmeliik
Copy link
Contributor Author

chmeliik commented May 21, 2024

We had too many Secrets linked to the appstudio-pipeline SA in the build-templates-e2e namespace, causing extremely large Pod requests that got rejected by the etcd server. The Applications (and, transitively, Secrets) in the namespace were cleaned up by @psturc 🙏

CI should have a better chance of working now.

@chmeliik
Copy link
Contributor Author

/retest

@chmeliik chmeliik added this pull request to the merge queue May 21, 2024
Merged via the queue into konflux-ci:main with commit 7dc8c7e May 21, 2024
6 checks passed
@chmeliik chmeliik deleted the build-args-param branch May 21, 2024 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants