Should we set up prow?

[justinsb]

Prow is the tool used by most kubernetes projects to do CI/CD. Prow predates github actions, and many features of prow are now available in "core" github - though not necessarily implemented exactly the same way.

If we wanted prow support in kptdev, we would (likely) have to host and run our own instance.

The question is whether that is something we should prioritize.

The main features of prow that are likely relevant to us:

• Running CI jobs (presubmits and postsubmits)
• "Helper" commands like /lgtm and /approve
• The merge queue

[justinsb]

Self-commenting to separate my opinion from the "problem statement" :-)

• CI: Github Actions seems to work well for running CI jobs, at least so far.
• Merge Queue: We don't seem to need a merge queue with our current PR volume, and github has merge queue features available/in progress.
• Helper commands: Would be useful, because the github UI for approving PRs is not very efficient (e.g. no keyboard shortcuts AFAICT). But there is some progress here on the keyboard shortcuts, and there are browser extensions available also. My 2c is that these aren't enough to justify maintaining a prow installation.

I can investigate alternatives that could enable /approve and similar commands without requiring maintaining a full prow installation, in the hopes that these are eventually replaced by e.g. better github keyboard shortcut support.

[johnbelamaric]

We do have Prow up and running for Nephio. I can ask if we can piggyback on that...

[johnbelamaric]

@radoslawc @gvbalaji @s3wong WDYT?

[mikebz]

I find Prow UX distracting from the problem at hand. Outside of merge queue the rest requires memorizing IRC like commands which almost always needs a lookup for people who are in in the prow land all day long. I like buttons and I like web pages that allow me to configure the system - all of which Github provides.

[johnbelamaric]

I see your point regarding not having to remember the commands (I still look up the less used ones sometimes). But I think whatever tool we use should be the one that works best for the people who spend the most time with it - we should optimize for the frequent users. Web pages and buttons are often great for the casual user and a pain for the people that do it every day.

More importantly, last I checked, using the GitHub UI requires granting too many privileges. We have finer grained control with prow. Maybe that has improved - and it's not really needed right now for this project. I wouldn't think there is any urgency to change at this point (though I do find prow better on a day-to-day basis).

[mikebz]

It's good to have fine permissions, I am curious though what permissions we really need - there is a group of core maintainers who should be able to control what goes into the main branch (wether it's approval required or only those people can merge) and then there are casual or external users. Do we have other things like makings sure people don't run up the bill with executing CI tests too frequently?

[justinsb]

Totally agree with the idea of solving the problems we actually have. We haven't seen any real issues (yet) with Github Actions as our CI.

The only problem I have right now is that approving PRs is a bit of a pain, because the workflow there is really not very efficient compared to an /approve comment. I would say that it's not a big problem, I'd like to make it better, but if that's the biggest problem we have I can't think it's worth installing our own instance of prow.

[justinsb]

Also agree that we should probably be converging on the github flow, so e.g. /approve should be a shortcut for approving the PR in the normal github way.

[sdowell]

🪙 🪙

• I view tide as being more than a merge queue. More features can be seen here. A valuable day to day feature, IMO, is that tide always rebases against the latest HEAD and re-runs tests before submitting a PR. I'm not aware of similar functionality with GH actions, and this helps avoid common scenarios that break code/CI.
• lgtm/approve features are nice to have
• Integrating with GCP features/services is generally easier with prow, assuming the build cluster is hosted on GKE. For example e2e test jobs that run against GKE clusters or build jobs that publish to GCS/AR can easily authenticate using workload identity. This may or may not be applicable for the porch/kpt use case.
• AFAIK k8s prow is available to both kubernetes and CNCF projects, so presumably kpt could use k8s prow. Kpt may still want it's own build cluster, but that is orders of magnitude simpler to onboard/maintain compared to a prow instance.

All that being said, if there aren't any major issues with the existing setup - perhaps there isn't much motivation to change it.

[justinsb]

tide as being more than a merge queue.

Yeah, tide merge queue is great, but I think github's is superficially at least pretty similar: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue

I suspect we would want to try that before setting up prow.

Integrating with GCP features/services ... can easily authenticate using workload identity

I believe we can do this using github actions also: https://cloud.google.com/blog/products/identity-security/enabling-keyless-authentication-from-github-actions

lgtm/approve features are nice to have

Agreed, though I am going to see if I can live without them!

AFAIK k8s prow is available to both kubernetes and CNCF projects, so presumably kpt could use k8s prow. Kpt may still want it's own build cluster, but that is orders of magnitude simpler to onboard/maintain compared to a prow instance.

You made me double check, I think this issue is about trying to keep k8s prow for k8s itself and not for k8s-related CNCF projects (though we could always make our case I guess):
kubernetes/test-infra#12863

All that being said, if there aren't any major issues with the existing setup - perhaps there isn't much motivation to change it.

Agreed, I think that's looking like the likely decision for now (I think), but this discussion can document our reasoning so we can re-evaluate as things evolve.