Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests #59

Open
8 tasks
cloudhead opened this issue Jan 15, 2020 · 3 comments
Open
8 tasks

Tests #59

cloudhead opened this issue Jan 15, 2020 · 3 comments

Comments

@cloudhead
Copy link
Owner

cloudhead commented Jan 15, 2020

  • parser.rs: the parser should be property tested with something like quickcheck
  • cmd.rs: CommandLine should be tested with ascii and utf8
  • resources.rs: Gif export should be tested
  • session.rs: inputs (mouse, keyboard) should be fuzz-tested to ensure nothing can crash the program.
  • session.rs: selection/* commands should be tested in headless mode
  • session.rs: all commands that take user input should be tested with unexpected inputs to ensure there is no division by zero for example
  • lib.rs: rx's entry point should be tested with different inputs
  • Test that everything that modifies a view sets the [modified] flag on it
@cloudhead
Copy link
Owner Author

@pop if anything above looks interesting, let me know and I can provide more detail. Would be great to have some help in making rx more robust.

@cloudhead cloudhead pinned this issue Jan 15, 2020
@pop
Copy link
Contributor

pop commented Jan 18, 2020

If I had to pick one, fuzz-testing the commands that take user input seems interesting. i.e.,

session.rs: all commands that take user input should be tested with unexpected inputs to ensure there is no division by zero for example

That said I have not dug too far into any of the topics so if there is something else you'd like to pass off to a contributor I'm open to one of the other topics.

I'm not entirely sure how to do headless testing but I'm sure once I get started writing the headless fuzz-tests would be pretty straight forward.

@cloudhead
Copy link
Owner Author

Great, I also think the fuzz testing will be the biggest ROI at this point, but it's also possibly more work than the others.

A good place to start looking is the execution module and the current tests in tests/, as there's already the ability to run rx in headless mode and just feed it inputs from a script. There's two levels at which the fuzz testing could be useful:

  1. Create random Command objects with random parameters and feed them into the Session and try to make it crash.
  2. Send random keyboard/mouse inputs directly to the session.

Both might be useful, but 2. is less likely to trigger complex behavior, since most inputs won't be valid commands.

Another thing that might be helpful alongside this is to add another input which is the current session mode, so run the fuzz tests in Visual, Normal and Command mode; I think this will increase coverage quite a bit.

Finally there's the question of whether to turn the renderer and platform layer off or not. I think to start out it would make sense, ie. to use a dummy renderer (trait Renderer) and context manager (platform). This way the fuzz tests can run 100x faster and test many more inputs.

These are my initial thoughts.

@cloudhead cloudhead assigned cloudhead and unassigned cloudhead Mar 12, 2020
@cloudhead cloudhead unpinned this issue Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants