Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the dry-run mode and make it only available in the playground #996

Open
fmallet opened this issue Jul 25, 2023 · 0 comments
Open

Comments

@fmallet
Copy link

fmallet commented Jul 25, 2023

WHY

Our Usability Testing Sessions have helped identify certain misleading aspects of the dry-run mode:

  • During the onboarding process, users anticipate that Reviewpad will operate on their Pull Request as soon as the reviewpad.yml configuration file is incorporated into it.
  • After merging a reviewpad.yml configuration file, there's often an implicit expectation that users can immediately modify and test a new configuration within a PR.
  • Currently, Reviewpad only considers the reviewpad.yml configuration file found in the main branch. When this configuration file is added or updated as part of a PR, a dry-run report comment is generated. However, this report is frequently misunderstood by our users.
    Despite these issues, we initially introduced the dry-run mode as a security measure, particularly for open source projects. We wanted to prevent PR authors from modifying Reviewpad's default behavior.

WHAT

To make the process more intuitive, we suggest a priority system for selecting the reviewpad.yml configuration file when Reviewpad analyzes a PR:

  1. If a reviewpad.yml configuration file is part of the PR, Reviewpad will use it.
  2. If the PR doesn't include a reviewpad.yml file, Reviewpad will use the one located in the main branch, if available.

For projects with heightened security concerns, we propose a new optional global property, fixed-config-branch: main. This property will compel Reviewpad to always use the reviewpad.yml configuration file located in the main branch, ensuring the integrity of Reviewpad's behavior.

@fmallet fmallet added enhancement New feature or request onboarding and removed enhancement New feature or request labels Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant