Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal to add integration tests #23

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

biggerfisch
Copy link
Collaborator

Overview

IMHO, one of the trickier things about working on shell projects like this is ensuring one doesn't make an accidental edge case break. To that end, I've started some integration tests. The test cases aren't nearly complete and the commits could probably do with a good squash at the end, but I wanted to start a discussion as to whether this was wanted by anyone aside from me and if this was a valid starting point.

Goal

My eventual goal with this would be to achieve near full coverage of zshrc.sh as well as the ability to automate these tests somewhat with github actions etc. This would also serve the conversation in #21 about finding a minimum zsh version as it would be possible to bisect zsh versions with these tests.

Strategy

These tests are intended to each create a git repo in a certain precise state, run the git_super_status command and validate the produced output, theme colors and all.

Advantages

  • Complete output validation in a textual form allows for precise diffs and avoids user-specific configuration interference
  • Test environments are easy to understand
  • A large measure of change-security is introduced

Disadvantages/risks

  • Tests can be tedious to initially write, as color codes often need to be manually re-inserted
  • Test lines are themselves not easy to read
  • It's possible these tests are more bound by zsh version than anticipated.
  • Outside environment impacts must be manually avoided - this means that they might slip in anyways by accident.

Any & all thoughts on this approach are appreciated, thank you!

Without this, git wants to manage this repo instead of one for test
purposes
Adds a test structure that creates a test repo and runs against it,
comparing the output to expected. Also adds two initial tests to prove
the point.
This organization makes it easier to add new tests and to avoid
re-implementing the setup/cleanup loop around each case
Not fully on par with most testing frameworks, but something is better
than nothing!
- Default author info isn't always enough
@biggerfisch biggerfisch self-assigned this May 24, 2022
@thomasrebele
Copy link

Sorry for the late reply. I had a quick look and it seems a good approach. Great idea to use a marker to prevent deleting the wrong directory.

@biggerfisch biggerfisch marked this pull request as ready for review July 8, 2022 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants