Attention, contributors: See #15439 (comment).

Is your feature request related to a problem? Please describe.

PRs usually sit open for too long (related: #15029), partially due to nobody coming to review earlier.

Describe the solution you'd like

We could try CodeRabbit, which is an AI code review bot: it uses ChatGPT and perhaps other similar LLM tools to provide code review comments. Additionally:

• You can have a discussion with it.
• Tt's capable of learning the habits of a project.
• When there's an issue linked, it can verify whether the PR addresses the issue.
• It can summarize the changes of a PR.
• Reviews are done either automatically or on-demand (requested with a comment).
• It's free for open-source projects.

For a while now we at Lombiq tested it on a couple of repositories. It's a mixed bag, with perhaps 50-50 useful and useless comments, but the experience varies greatly PR by PR. Some examples:

• Useful comment.
• Mixed bag for the whole PR.
• Mostly useful comments
• No actionable comments despite enough code changes

So, I'm not entirely convinced, but we could try if it's useful for us and helps with reviews, or it's more like noise.

• Set it up first just from the CodeRabbit UI so it can be tested immediately.
• Once we figured out how to configure it, move the config to a file. See https://github.com/Lombiq/Open-Source-Orchard-Core-Extensions/pull/662/files for a good starting point.
• Document it for contributors that you'll get AI reviews, and take them the same as human reviews. See Get our pull request game in order #15029 for contribution docs too.

This will throw off our community metrics a bit, since the time to first reply for PRs will be down to a few minutes, so we won't be able to track the time to human reviews (and the useful remaining metric will be time to close). Also, if as a reviewer you scan for PRs to review by looking at the number of comments (as an indicator that somebody else reviewed already) then that won't apply anymore. Using the "No reviews" filter can be appropriate, but a "review" here is a changes requested/approved flag, what some people don't use during review (but they just comment).

Describe alternatives you've considered

Perhaps there are other such tools that we can check.