Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core feature] Override task secret_requests using with_overrides #5261

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Mecoli1219
Copy link

@Mecoli1219 Mecoli1219 commented Apr 21, 2024

Tracking issue

#5085

Why are the changes needed?

with_overrides method should enable overriding secret_requests property. See Tracking issue for more information.

What changes were proposed in this pull request?

  • Adding override_security_context properties to TaskNodeOverride proto and OverrideSecurityContext to NodeSpec.
  • Implement overrides when TaskExecutionMetadata is created.

How was this patch tested?

Setup process

I ran flyte on my dev machine and tested my code with this script:

Testing Script
from flytekit import task, Secret, workflow
import flytekit
import typing


@task(secret_requests=[Secret("group1", key="key1")])
def foo(a: str) -> str:
    return a + flytekit.current_context().secrets.get("group2", "key2")


@task(secret_requests=[Secret("group2", key="key2")])
def foo2(a: str) -> str:
    return a + flytekit.current_context().secrets.get("group2", "key2")


@workflow(
    failure_policy=flytekit.WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE
)
def my_wf() -> typing.Tuple[str, str, str, str, str, str, str, str]:
    overrides1 = {
        "secret_requests": [Secret("group2", key="key2"), Secret("group3", key="key3")],
    }
    overrides2 = {
        "secret_requests": [Secret("group3", key="key3")],
    }
    overrides3 = {}
    str1 = foo(a="foo-fail-")
    str2 = foo2(a="foo2-success-")
    str3 = foo(a="foo-fail-").with_overrides()
    str4 = foo2(a="foo2-success-").with_overrides()
    str5 = foo(a="override1-success-").with_overrides(**overrides1)
    str6 = foo2(a="override2-fail-").with_overrides(**overrides2)
    str7 = foo(a="override3-fail-").with_overrides(**overrides3)
    str8 = foo2(a="override3-success-").with_overrides(**overrides3)

    return str1, str2, str3, str4, str5, str6, str7, str8
  1. Create the testing secret with these commands:
    kubectl create secret -n flytesnacks-development generic group1 --from-literal=key1=super-secret-value1
    kubectl create secret -n flytesnacks-development generic group2 --from-literal=key2=super-secret-value2
    kubectl create secret -n flytesnacks-development generic group3 --from-literal=key3=super-secret-value3
    
  2. Execute the script and the result should be like
    ${\color{red}Failed}$ => ${\color{green}Success}$ => ${\color{red}Failed}$ => ${\color{green}Success}$ => ${\color{green}Success}$ => ${\color{red}Failed}$ => ${\color{red}Failed}$ => ${\color{green}Success}$

Screenshots

image

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

flyteorg/flytekit#2365

Docs link

Copy link

welcome bot commented Apr 21, 2024

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Apr 21, 2024
@Mecoli1219 Mecoli1219 marked this pull request as draft April 21, 2024 09:34
Signed-off-by: Mecoli1219 <michaellai901026@gmail.com>
Signed-off-by: Mecoli1219 <michaellai901026@gmail.com>
Signed-off-by: Mecoli1219 <michaellai901026@gmail.com>
@Mecoli1219 Mecoli1219 changed the title [Core feature] Override task secret_requests using with_overrides [Core feature] Override task secret_requests using with_overrides May 7, 2024
@Mecoli1219 Mecoli1219 marked this pull request as ready for review May 7, 2024 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant