Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the current branch's commit as the tag for Agent's integration tests #324

Open
pablochacin opened this issue Aug 26, 2023 · 3 comments
Labels
ci/cd internal improvement improves code quality or organization

Comments

@pablochacin
Copy link
Collaborator

#322 introduces integration tests that use TestContainers to spawn a container with the disruptor agent using the latest tag for the image.

This introduces a significant risk of not testing the correct version of the agent when making local tests. If the agent forgets to build the image locally it will be testing the image from the main branch. It is also possible that an older version of the image is present locally and be tested instead of the current modified version.

To safely address this issue it would be desirable to change how the agent's image is tagged and how it is referenced in the integration tests.

Notice that the policy for tagging the agent images must be synchronized with the disruptor.

In brief, on each release, the disruptor will reference the agent's image with a tag that matches the same release version. When build from a development branch or main, it will reference latest. This is implemented in the version package.

In order to ensure integration tests use the current development version this policy must be changed.

@roobre
Copy link
Collaborator

roobre commented Aug 28, 2023

A similar approach I've seen working in other projects is to use a ci-only image name/tag, that guarantees such image cannot be pulled. For example, something like ci.local/grafana/xk6-disruptor.

Something like this might require adding some knobs to customize the agent image in runtime, though.

@pablochacin pablochacin changed the title Use current branch's commit as tag for Agent's integration tests Use the current branch's commit as the tag for Agent's integration tests Aug 28, 2023
@pablochacin
Copy link
Collaborator Author

pablochacin commented Aug 28, 2023

A similar approach I've seen working in other projects is to use a ci-only image name/tag

I think this approach works in contexts where the image is only pulled from the CI tests. For instance, the agent tests would work.

However, if we add any test that includes the disruptor, then the problem mentioned in the issue would likely happen.

A more robust solution is needed. I'm thinking about modifying the logic for defining the agent's image tag to support customization via an environment variable.

@roobre
Copy link
Collaborator

roobre commented Aug 28, 2023

then the problem mentioned in the issue would likely happen.

You're right, I was thinking in CI scenarios where we can assume the image has not been built before.

Customizing image and/or tag through an env var makes a lot of sense to me 👍🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/cd internal improvement improves code quality or organization
Projects
None yet
Development

No branches or pull requests

2 participants