-
Notifications
You must be signed in to change notification settings - Fork 101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
stbt run: Allow passing arguments to test case functions #290
base: main
Are you sure you want to change the base?
Conversation
This is a counterpart to passing arguments to the old-style test case files as `sys.argv`, but for new-style test case functions. It's also a lot nicer because now you don't need to implement argument parsing in the test-case itself, you just get the arguments passed through.
Includes the arguments that the test case takes and the docstring if available.
Note: I chose a single JSON argument to allow passing types through to the functions rather than passing everything through as a string. |
My initial reaction to this was that (a) it's a homegrown/non-standard way of doing remote procedure calls from the command line, and (b) yet another divergence from existing test runners like nose or pytest. Re. (a) I suppose the problem is that there simply isn't a good way of passing typed parameters from the command line or exec(3). In looking for existing solutions to this problem I came across systemd/dbus busctl tool, which looks like:
e.g.:
I'm not suggesting we use that. If anything it's a counter-example to show how much better it is to use json. I suppose the alternative would be to allow Re. (b) after further thought I don't care -- neither nose nor pytest support passing arguments to test functions at all, so it's not like we're being incompatible with them. Nose only supports passing configuration via a config file which you then read explicitly in your test function. I don't really have a point here, just rambling thoughts. |
I don't think this is true. We could use py.test's fixtures to pass the arguments in. |
I meant that you can't pass arguments at run-time from the command line. Though I suppose you could write a pytest plugin that used the fixtures mechanism to provide arguments from the command line? |
An alternative would be to encode the expected types in the test scripts themselves and do the conversion based on that. For example you could parse:
or
or even use PEP-484 stub files and know what types we are expecting and do conversion as appropriate. |
Yeah but then how do you specify the dict's contents on the command line? The |
I guess you would then require JSON. e.g from bash:
or
I agree, but the question is: is it good enough to go in? |
Yeah go ahead. I certainly haven't come up with anything better in the last 6 weeks. |
It would be nice to also pass in positional arguments specified on the command line, even if we don't do any type conversions and pass them as strings:
At least this would be consistent with the existing command-line interface. To keep backwards compatibility with functions that don't take arguments but that do look in |
If I recall correctly, one of the motivations for allowing arguments as JSON is to make it easy to integrate |
This is a counterpart to passing arguments to the old-style test case files as
sys.argv
, but for new-style test case functions.It's also a lot nicer because now you don't need to implement argument parsing in the test-case itself, you just get the arguments passed through.
stbt run
can now take pass arguments to test case functions with the--kwargs
option.Example: with this test case in
tests/channel_test.py
:you can pass the channel number 3 with:
Note: the
--kwargs
option takes a JSON formatted string and is passed as an argument after the test case name. This means that you can also pass the--kwargs
option on thestbt batch
command line for each test case:An argument could be made that we shouldn't override
sys.argv
in this case, or possibly produce an error if you attempt to pass additional arguments to test case functions. This would be a breaking change, but you could argue that it would be good to do it as it would allow an opportunity in the future to even more automatic argument parsing. Probably a bad idea for now though.