Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP --- Update how we are loading executors #723

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

fokion
Copy link
Contributor

@fokion fokion commented Sep 27, 2023

check them if we have them in the cache and read them only once

@fokion fokion force-pushed the feature/update-executor-cache branch 2 times, most recently from 0ea685c to f8f16f1 Compare September 27, 2023 22:50
…ing them from the cache properly

Signed-off-by: Fokion Sotiropoulos <fokion.s@gmail.com>
@fokion fokion force-pushed the feature/update-executor-cache branch from f8f16f1 to ff75472 Compare September 27, 2023 22:53
@ovh-cds
Copy link
Collaborator

ovh-cds commented Sep 28, 2023

CDS Report build-venom-a#105.0 ✘

  • Build
    • Build ✔
    • Unit Tests ✔
  • Tests
    • Acceptance Tests ✘

Copy link
Member

@yesnault yesnault left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand the purpose of the pull-request, but if the purpose is not to read the file several times, there is already a cache on it.
If the goal is to have a cache on the executor once read with the input variables, the cache map would have to contain the input vars also in the key.

The following tests are failed:

case testsuite / testcase-case	
testsuite with a user executor multilines / testA	
testsuite.26115637 / test 2	
testsuite with a user executor in custom dir / testfoobar custom	
testsuite with a user executor in custom dir which has multiple steps / testfoobar multisteps custom	
user_executor_http / call foobar_http_get_user	
Many user executor / mytc	
testsuite with a user executor.26115637 / Testy test part 1	
play-with-array / mytc	
play-with-array / using-var-from-arg

venom.go Outdated
}

v.RegisterExecutorUser(ux.Executor, ux)
ux.Input.AddAll(varsFromInput)
ux.Executor = executorName
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is no need to force assign here, ux.Executor already contains the value, as the name is already in the btes.
So, No need to have getExecutorName func.

venom.go Outdated
executorName, _ := getExecutorName(btes)
if len(executorName) == 0 {
fileName := filepath.Base(f)
executorName = strings.TrimSuffix(fileName, filepath.Ext(fileName))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of user executor is the attribute in the file. It should not be defined by the filename.

In the read, we can see that the user executor is defined in the file lib/customA.yml, but the executor name is executor.

@@ -44,6 +44,21 @@ func getVarFromPartialYML(ctx context.Context, btesIn []byte) (H, error) {
return partial.Vars, nil
}

func getExecutorName(btes []byte) (string, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed, see comment below.

@@ -229,12 +232,13 @@ func (v *Venom) RunUserExecutor(ctx context.Context, runner ExecutorRunner, tcIn
TestCaseInput: TestCaseInput{
Name: ux.Executor,
RawTestSteps: ux.RawTestSteps,
Vars: vrs,
Vars: inputOnlyVrs,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tc.Vars == tc.TestCaseInput.Vars.

I don't understand the goal of the variable inputOnlyVrs: Doing TestCaseInput.Vars = intputOnlyVrs, and then tc.Vars.AddAll(vrs) is the same as the previous code.

@@ -216,13 +215,19 @@ func (v *Venom) getUserExecutorFilesPath(vars map[string]string) (filePaths []st
}

func (v *Venom) registerUserExecutors(ctx context.Context, name string, vars map[string]string) error {
_, ok := v.executorsUser[name]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't do a cache here. The GetExecutorRunner is used in RunUserExecutor, so that vars can contains different value depending of the input.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will mark it as WIP as I need to check if I have migrated all the necessary changes for it to work properly.

You do need to check if you have read the executor or not as this is the point of the cache , we want to read that file only once and not multiple times ) .

@fokion fokion changed the title Update how we are loading executors WIP --- Update how we are loading executors Sep 29, 2023
…the executor from the file name if we dont define the executor field

Signed-off-by: Fokion Sotiropoulos <fokion.s@gmail.com>
@fokion fokion marked this pull request as draft October 21, 2023 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants