Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inject tag or value to capture whether the build is the first or second build of the experiment. #539

Open
clayburn opened this issue Dec 13, 2023 · 3 comments

Comments

@clayburn
Copy link
Member

When analyzing the results of an experiment, it can be easy to get lost in a build scan and not be sure whether it was the first or second build of the experiment. Often, the best way to check is by assuming from the performance of the build itself or navigating back to the summary view. It would be more concrete to capture this metadata as a tag or custom value so that it would be clear.

@erichaagdev
Copy link
Member

A note on the design while I'm thinking about it: Using a custom value will show up as a difference in the build scan comparison, but a tag will not.

@ribafish
Copy link
Member

A note on the design while I'm thinking about it: Using a custom value will show up as a difference in the build scan comparison, but a tag will not.

While that is true, I would find it easier to work with in the comparison if it's tag, since it shows up at the top with the rest of the build header in task inputs comparison for example. That way you're sure which build is which on that page without needing to jump to the custom values view.

@erichaagdev
Copy link
Member

I was also advocating for a tag when I wrote that. A custom value means there will always be a difference between the builds, which could be misleading.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants