Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The need to specify a test suite in ${DYNAMICTEST(S)} is somewhat cumbersome #43

Open
nizwiz opened this issue Apr 22, 2021 · 9 comments

Comments

@nizwiz
Copy link

nizwiz commented Apr 22, 2021

Just imagine having to type this:

robot --variable "DYNAMICTESTS:MyDataDrivenTestSuiteWithParticularlyLongName.0101|MyDataDrivenTestSuiteWithParticularlyLongName.0102|MyDataDrivenTestSuiteWithParticularlyLongName.0103" MyDataDrivenTestSuiteWithParticularlyLongName.robot

Yeah you can copy'n'paste, but in a real world example you would want to copy'n'paste the test names as well, and having to insert the suite name in between test names is cumbersome (and in the frequent case that one specifies a single robot file on the command line, it's also redundant).

I think it would be great if the test suite part were optional.

I'd be willing to contribute a pull request.

@Snooz82
Copy link
Owner

Snooz82 commented Apr 22, 2021

Hi @nizwiz

Can you tell me a bit more about the use case?
Because DYNAMICTESTS where introduced for Pabot and for automatic selection by code.

is has not been designed to be used by humans.

The suite can only be skipped if exactly one suite is used.

@Snooz82
Copy link
Owner

Snooz82 commented Apr 22, 2021

With use case i mean in which situation are you using that and why can you not use tags or other things.

@nizwiz
Copy link
Author

nizwiz commented Apr 22, 2021

Well, the use case is pretty much as described in section "Selection of test cases by name" in Readme.rst.
I am currently looking to automate a bunch of very repetitive, parametrized test cases my colleagues are currently running in a semi-automatic fashion (using sophisticated Excel sheets with a lot of formulae). After full automation, they still want to be able to run selected test cases without having to execute the whole suite (which is rather lengthy). And with "selection" they really mean "a more or less random bunch" - without any categorization that could be made a priori. So tagging is really not applicable (basically, you'd have to have an individual tag for each line of the csv file, which - while technically possible - I'd rather not do because it blows up the report.html; BTW, I've actually tried this approach out, because specifiying tags seemed easier than specifying test cases via ${DYNAMICTESTS}).

@Snooz82
Copy link
Owner

Snooz82 commented Apr 22, 2021

Hm....

Let me think about this.
Maybe there should be another variable that can specify a "simple pattern" or glob?

but i think a better approach for you would be a Variablefile!

define a yaml variable file with the test case names as a list and hand it over to robot.

that should make this ugly command line better.

@Snooz82
Copy link
Owner

Snooz82 commented Apr 22, 2021

DYNAMICTESTS:
  - MyDataDrivenTestSuiteWithParticularlyLongName.0101
  - MyDataDrivenTestSuiteWithParticularlyLongName.0102
  - MyDataDrivenTestSuiteWithParticularlyLongName.0103

@nizwiz
Copy link
Author

nizwiz commented Apr 22, 2021

Oh wait, I'm actually also using pabot with --testlevelsplit (because each test case spends a considerable amount of time just waiting for a result to be computed). I've double-checked and, in deed, when using pabot together with DYNAMICTESTS, there doesn't seem to be any parallelization. I guess I'll have to use tags anyhow, then.
@Snooz82 You can close this ticket if you like.
But maybe the aforementioned section in Readme.tst should contain a note pointing out that DYNAMICTESTS cannot be used in conjunction with pabot. I for one really wasn't aware of the fact that DYNAMICTESTS is used internally to make pabot work.

@Snooz82
Copy link
Owner

Snooz82 commented Apr 22, 2021

Good point!

@Snooz82
Copy link
Owner

Snooz82 commented Apr 22, 2021

I will let it open as a maker for me to do that addition to docs!

thanks for your contribution!!!

@nizwiz
Copy link
Author

nizwiz commented Apr 22, 2021

Well, thank you for this excellent RF extension! It's really indispensable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants