You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a pull request is created towards the main branch from a feature branch, tests are triggered automatically by GitHub actions. These tests target the code in the feature branch. Tests are run iteratively for Python 3.9, 3.10, and 3.11 to ensure that the new changes do not break anything and the software's behavior is consistent across the supported Python versions. Tests are currently being run only for Linux environment. These tests are run by GitHub on a remote Linux machine somewhere.
This is specified in the automated tests workflow file .github/workflows/run-tests.yml
Test the new changes on Windows and MacOS in addition to testing new changes on Linux. This will help cover the following scenario.
Dependencies/libraries needed by aeolis such as numpy, matplotlib, etc., may behave differently on different operating systems or contain operating system (OS) specific bugs. Testing for multiple OS will help catch such errors in advance.
Proposed changes
Specify a matrix of operating systems containing Windows, MacOS, and Linux in .github/workflows/run-tests.yml
Currently tests take approximately 3.5 minutes to run. By specifying the tests to also run on Windows and MacOS for 3 Python versions, tests will take approximately 5.5-6 minutes to run based on an experiment in the fork.
This extra time taken by the tests to run does not impact the user and developers working with aeolis on their local machines. This extra time is spent only on GitHub's end when a pull request is created. As a reviewer, you will have to wait longer to see if the pull requests passes all the tests.
The text was updated successfully, but these errors were encountered:
@Sierd, @manuGil do you think this tradeoff between extra time spent to run tests on multiple operating systems (OS) and the added value of catching potential bugs across different OS is reasonable?
@Sierd, @manuGil do you think this tradeoff between extra time spent to run tests on multiple operating systems (OS) and the added value of catching potential bugs across different OS is reasonable?
I think it is, but only for releases. I suggest to create a new action to test installing different OS's. This should be triggered as one of the last steps before making a new release. For example just before the action to push the code to PyPI, and maybe it is a job we want to add to the same workflow file.
Current behavior
When a pull request is created towards the
main
branch from a feature branch, tests are triggered automatically by GitHub actions. These tests target the code in the feature branch. Tests are run iteratively for Python 3.9, 3.10, and 3.11 to ensure that the new changes do not break anything and the software's behavior is consistent across the supported Python versions. Tests are currently being run only for Linux environment. These tests are run by GitHub on a remote Linux machine somewhere.This is specified in the automated tests workflow file
.github/workflows/run-tests.yml
Desired behavior
Test the new changes on Windows and MacOS in addition to testing new changes on Linux. This will help cover the following scenario.
Proposed changes
Specify a matrix of operating systems containing Windows, MacOS, and Linux in
.github/workflows/run-tests.yml
Consequences of the proposed changes
The text was updated successfully, but these errors were encountered: