-
|
We've seen a spike in activity with the release of Windows Package Manager 1.0. We have stopped the automated "merge" for PRs. Windows Package Manager team administrators will begin manually reviewing submissions to reduce the number of duplicate submissions, and manifests with sub-optimal metadata. This discussion is intended to provide an open forum to discuss how we should move forward with moderation. Please keep the discussion positive and constructive. The need for moderation was highlighted in #14621. We appreciate everyone's feedback and suggestions. The goal is to continue to grow a healthy community catalog of packages for the Windows Package Manager. If your suggestion is off-topic you should create another discussion topic, or if you have another feature in mind, feel free to create that new feature. |
Beta Was this translation helpful? Give feedback.
Replies: 12 comments 38 replies
-
|
Many of the most prolific contributors for the Windows Package Manager community repository have put countless hours of their own effort attempting to make this catalog the best it can be. I am mentioning their GitHub aliases below to identify those individuals as candidates for becoming moderators. We have not finalized the process for how they will be selected, These individuals have contributed to the repository for an extended period of time and have authored or corrected many manifests. In addition many of them have guided and coached new users of the community. @ItzLevvie I would like to also take the opportunity to mention that I don't believe any one individual can be an expert in all of the edge cases with installers and third party software. This is going to be a team effort. We will all learn together, and we will all likely make mistakes. We are human and each of us should be treated with respect and courtesy. The Windows Package Manager team is looking at several options for how moderation can and should work. There are possibly many correct answers here, not just one. Let's try to keep the philosophical debates to a minimum, and focus on solutions to the problem at hand. We will work together to select the best option to move forward in the short term, as well as other options over the long term. Our thinking and our solutions will evolve over time. |
Beta Was this translation helpful? Give feedback.
-
|
We need a more automatic system and/or more people that approve PRs, i'm getting tired of my PR's staying up for 3+ days at a time even when my PR's are the only ones with |
Beta Was this translation helpful? Give feedback.
-
|
If the bot could add some kind of “windows-package-manager-moderators” as a reviewer when it hits Azure-Pipeline-Passed that would be great, as I could see it from my GitHub inbox. I just started a new job and don’t have more than an hour or so a day to go through WinGet, so I can only hit the first page or two. (I need to up my GitHub filtering game though, that’s on me) |
Beta Was this translation helpful? Give feedback.
-
|
If you search |
Beta Was this translation helpful? Give feedback.
-
|
I always use https://github.com/microsoft/winget-pkgs/pulls?q=is%3Apr+is%3Aopen+draft%3Afalse+-label%3AAuthor-Not-Authorized+-label%3ABlocking-Issue+-label%3AError-Hash-Mismatch+-label%3AModerator-Approved+-label%3ANo-Recent-Activity+-label%3AProject-File+-label%3APullRequest-Error+-label%3AURL-Validation-Error+-label%3AValidation-Installation-Error+-label%3AValidation-Metadata+-label%3AValidation-Unattended-Failed or |
Beta Was this translation helpful? Give feedback.
-
|
We're working through the state diagram for labeling. The bot has some tricky limitations. Would you like us to start adding a "Needs-Moderation" label? We can do that when Azure-Pipelines-Passed, and no other labels are present. The exception that would remain is when a review requiring feedback from the author is posted, we add the "Needs-Author-Feedback" label, and when the author comments or pushes a change, we remove that label and add "Needs-Attention". It may be possible to add "Needs-Moderation" when we remove the "Needs-Author-Feedback" if "Azure-Pipeline-Passed" and no other blocking labels are present. |
Beta Was this translation helpful? Give feedback.
-
|
I think the process in #100 is first and foremost critical: The best case for WinGet to be trustworthy is that the official developers are involved in the maintenance process. If the developers treat WinGet as an official release channel, we should be able to be confident that a manifest is trustworthy. This definitely requires that the developer is able to define a group of users that is permitted to modify their org's folder, and once that's done, it should be safe to auto-merge PRs from that team to the repo. Deference to how to manage different versions and releases and such can be pretty much entirely left up to them at that point, they know their project best. The above represents both the best case for upstream developers and the least work for Microsoft/the WinGet team. Beyond that, I still think there's a likelihood volunteers should be defined as maintainers of given packages added by the community. It should be possible to take maintainership away from someone who isn't keeping up a given project, but having "too many cooks in the kitchen" on a given project can be really problematic. I imagine the goal here is that someone can write a script that includes |
Beta Was this translation helpful? Give feedback.
-
|
The feature for "verified publishers" is nearly complete. We're still working on the proper "business process" to validate those publishers. A few known publishers have already been identified for testing. We are also discussing the policy options for packages that appear to have been abandoned or have not been updated for some period of time. We have not yet landed on a policy for packages to be "maintained" by community members who are not directly affiliated with the actual publisher/ISV. We would likely treat these "maintainers" a bit differently than the official publishers. Feedback and suggestions are welcome here. |
Beta Was this translation helpful? Give feedback.
-
|
I agree community maintainers should be treated differently than verified publishers. Verified publishers can probably be more or less allowed to self-manage the packages in their org and skip review. If good package policy is laid out (see below), we can expect verified publishers to largely follow it, but I think it's safe to defer to their judgement entirely on their packages, because the alternative is just that publishers tell people to go straight to their website, and not support WinGet. Community maintainers should still be subject to review, but I think the goal of community maintainers is to provide a single-point-of-contact for the package. If I published/am maintaining the AdobeReader package, and someone else tries to submit a PR for it, ideally the moderators should ask me to look at it, and my opinion, as the regular maintainer, should be deferred to over a drive-by PR. That being said... opinions are something the WinGet team needs to attempt to factor out. Moderation isn't just about having a team, it's about having a consistent set of guidelines for that team to implement. For example:
|
Beta Was this translation helpful? Give feedback.
-
|
(Some thoughts about these.)
There should be very few scenarios where this is intentionally the case, since when you update a manifest you have to go and either get the new URL or choose explicitly to copy paste the info from the old manifest. (Notwithstanding the issues with wingetcreate and multiple arch support right now). Hopefully nobody would do that, but one could easily have the pipeline check the SHA256 hash to ensure that the previous version didn't use the exact same hash.
Release channel support is being tracked for the client at microsoft/winget-cli#147, but currently there's no real standard besides putting the manifest at a different PackageIdentifier, e.g.
I'd prefer that there wasn't, and that if you can pass SmartScreen and have publicly available licensing information that allows it to be added to the repo, it can be. The moderators would have to spend a lot of time deciding whether or not something is notable if they haven't heard of it before, and then we'd have the issues section turn into a fight about what notable means. |
Beta Was this translation helpful? Give feedback.
-
|
There are a couple vectors of abuse here that will ultimately need to be addressed. I think it will eventually grow to the level of security that CA's employ when countersigning SSL certs. Products like Browsers, Password Managers and GPG/SSH clients all play a key role in secret keeping. Getting a rouge product past moderation, even if only for a short while, could compromise many secrets. Beyond locking down folders like Chrome, and Putty, there should be a list of abstainers who wish to keep their product off of Once a cryptographic sighing protocol is in place, I would see the main job of maintainers as simply verifying the signatures, and ensuring no collisions in the existing products or abstainers list. Even once everything is locked down, there will be attacks by submitting something like "Chrom" browser instead of "Chrome" browser. Or simply submitting a package that is already on an abstainers list. Ideally I think submissions should come with the same signature as the submitted installer. If CodeSigning is used, then the submitter would need to sign some proof of ownership assertion then include it in a signed CAT file. So long as the same cert is used to sign the installer and the proof CAT, then ownership is proven. Alternatively GPG could be used with the Github GPG signed release / tag, or a signature housed in the same path as the installer. So long as the submitter produced a signature with the same key as the other GPG signatures, then proof is known. And as a final option, perhaps a TXT field in the DNS entry for the installers domain. All of these would provide cryptographic proof of ownership, the ability to abstain from inclusion, and moderation to avoid impostures. If #100 is used, I think it should be expanded to include some cryptographic proof for it to be introduced into a folder. Then include that process in the main MODERATION.md file. I'm not seeing it used yet Thx to all for this valuable discussion |
Beta Was this translation helpful? Give feedback.
-
Remember that some apps have not been updated for a really long time, but they still function properly, and clients still depend on them. But a notability policy would be garbage, especially for small developers. |
Beta Was this translation helpful? Give feedback.
-
|
@jedieaston has been granted "moderator" status at least temporarily to validate this process. We have configured the system such that once a PR has been labeled with "Azure-Pipeline-Completed" and "Validation-Completed" his "Approve" on the PR will result in another label "Moderator-Approved" being added to trigger the merge. Once we have validated this functionality, the other candidates can opt-in to being moderators to help through this period in addition to the Windows Package Manager administrators. |
Beta Was this translation helpful? Give feedback.
-
|
@ItzLevvie, @oxygen-dioxide, and @OfficialEsco There is zero pressure here. If you would like to volunteer to help us moderate PRs, we would greatly appreciate your support. If you prefer not to take on the additional challenges that come with being a moderator, we completely understand. |
Beta Was this translation helpful? Give feedback.
-
|
@ItzLevvie you should also now be able to "Approve" PRs. |
Beta Was this translation helpful? Give feedback.
-
|
Ofcourse i would love to moderate, definitively because of the EU timezone |
Beta Was this translation helpful? Give feedback.
-
|
@OfficialEsco you should also now be able to "Approve" PRs. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, Just had a small question for which I consider discussion to be too big. For example if I test some PRs and write my own simple approve (which does not auto merge), what should I do? shall I ping a moderator or just stay quite without disturbing. Because I think keeping myself not annoying and helpful to open source keeps me happy. 😁 |
Beta Was this translation helpful? Give feedback.
-
|
It won't get approved any faster since we still have to look over it, if its just a update or if we know you do quality control it will get approved without testing. (Depends on the software) |
Beta Was this translation helpful? Give feedback.
-
|
We will continue to work together to determine the criteria for becoming a moderator in a transparent manner. Some of the considerations are:
We would also like to have some criteria for retaining the moderator status.
|
Beta Was this translation helpful? Give feedback.
-
|
In terms of publishers, many orgs went through a rigorous process for validation on the Microsoft Store. It would be great if validation here could be linked, so that orgs don't have to go through the process all over again (if they already have a presence on the MS Store, and the contributor can be identified as associated with, or submitting on behalf of, that org, e.g. the main dev on the package's official GitHub repo). |
Beta Was this translation helpful? Give feedback.
-
|
@Jaifroid the current Microsoft Store source is experimental. We're still working through the issues related to making sure we can meet all of the requirements for packages in the store, and providing the best possible end to end experience for customers. There are some subtle differences in terms of what versions of a package could be available in the community repository and the "single" version of a package available from the store. We will continue working towards "verified publishers" in the community repository, and that should help quite a bit by eliminating the need for moderation for manifests submitted by "verified publishers". That feature is actively under development. |
Beta Was this translation helpful? Give feedback.
-
|
Might it be useful to use a tool like https://danger.systems to enforce certain rules on PRs? I'm definitely no expert on this repo nor Danger, I just think automating as many rules and checks as possible will save a lot of time and reduce the chance of human error, and that a tool like Danger could possibly help you to do so. I'm sure you have already thought of this but thought I'd point it out just in case. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
@denelon There appears to be a bigger backlog than usual on reviewing packages. While I'm sure this is temporary, and I know the volunteer moderators do a tremendous job, is there any advance on a more sustainable system? For example, could devs who have regularly and successfully submitted x number of updates to a specific package or packages, be allowed to merge future updates to the same package(s)? |
Beta Was this translation helpful? Give feedback.
-
|
The backlog was caused due to some unexpected downtime in @msftbot (#27041). I like this idea though, if we continue having a huge backlog even with 7 moderators something like this might help. |
Beta Was this translation helpful? Give feedback.
-
|
Can't we add something like this in the CODEOWNERS file?
|
Beta Was this translation helpful? Give feedback.
-
|
@Jaifroid we're discussing additional improvements to help reduce the workload on moderators. We've got a few changes lined up to help. The "verified developer" feature will add a .package file with GitHub owners who are authorized to make changes to their manifests. We're also discussing bringing "metadata-only" back for changes that don't require a full validation run. @vedantmgoyal2009 we had talked about CODEOWNERS in the past, and were concerned about the churn and complexity of maintaining a single file with the number of contributors we have. It just wouldn't scale. We're also looking to extend the functionality implemented in the .package file into other areas so we can continue to improve and inform the system about possible automated updates and other configurations to help with future validation. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, all for the updates. Sounds positive. And big thanks to the moderators who do an amazing job given the amount of traffic through the repository. Glad the bot was fixed. |
Beta Was this translation helpful? Give feedback.
-
|
@OfficialEsco share your list of Administrator/Maintainer needed PRs in a new Issue. I'll ask the engineering team to work through the list. |
Beta Was this translation helpful? Give feedback.
-
|
Curiously we should have been at around 100 PRs before the msftbot shutdown which i do not recall being in a long time, but a maintainer should keep a watch on these labels https://github.com/microsoft/winget-pkgs/pulls?page=1&q=is%3Apr+is%3Aopen+-label%3ABlocking-Issue+-label%3ADependencies+-label%3AValidation-Unattended-Failed We should not have to remind the engineers what PRs to look at when GitHub already got a filtering system the engineers can use. |
Beta Was this translation helpful? Give feedback.
-
|
@denelon, I have 42 PRs open for Edge to change metadata which are hitting validation errors: https://github.com/microsoft/winget-pkgs/pulls?q=is%3Aopen+is%3Apr+author%3Ajedieaston+archived%3Afalse+Edge+. They probably just need to manually be merged, as they won't install on a up-to-date version of Windows 10 (since you can't downgrade Edge). But they work on older builds. |
Beta Was this translation helpful? Give feedback.
-
|
I'll take a look at the documented processes we're following, and make adjustments as necessary. I think the intention was to have the labels be "self-explanatory" enough to be actionable by the author. That clearly isn't working well enough. I'll ask them to make comments on PRs with labels other than "Blocking-Issue" and "Validation-Unattended-Failed". And again thanks to the moderators for all of the support here. |
Beta Was this translation helpful? Give feedback.
-
|
@denelon I don't know whether to continue this discussion or start a new one, but it seems to me there is considerable lag in approvals lately on winget. It used to be much much faster to get a package approved here on winget than for the MS Store, but now I find my packages are usually approved within hours on the MS Store, while here they take days at best, sometimes a week. Moderators do a brilliant job, so this is not in any way a complaint, but it seems to me that a more automated process for regular/verified contributors is long overdue. There was talk of a verified contributor programme above, and I suggested a holistic approach that would allow linking developers uploading manifests here to verified accounts on the MS Store (not only for packages in the Store, but also for pure winget packages). The link would not be able to rely on the GitHub user id of course, but there needs to be a way of linking a verified presence on the MS Store with a GitHub ID here, or an independent verification process failing that. Otherwise the task for moderators is just too overwhelming, I imagine. The only other solution is for MS to employ some people to do moderation. It's too much to rely entirely on volunteers for what is ultimately a pretty mechanical, repetitive, never-ending process. |
Beta Was this translation helpful? Give feedback.
-
|
Based on the contents of this discussion, and the 1.0 release of winget without any sort of developer verification feature has continued to confirm to me that Microsoft simply doesn't value security here. I am still stunned in 2022 that winget is a thing that Microsoft shipped. |
Beta Was this translation helpful? Give feedback.
-
I've said this for a while, we do have one MS employee atm, also IMO wingetbot should be able to validate 99% of the PRs by it self as it does catch the ARP data from the installation. The difference between a few months ago and now is that both me and Levvie is burnt out from approving PRs every day for 1+ year, and submitting and managing packages for 2 years, AND its been a increase in daily PRs. |
Beta Was this translation helpful? Give feedback.
-
|
@OfficialEsco If you are doing work for Microsoft to maintain this repo, Microsoft should be paying you. tbh. People should generally not work for trillion dollar companies for free. Packages from community members should see significant moderator scrutiny, and trusted/verified developers should be able to push updates basically instantaneously/automatically. Microsoft should either be giving this project the engineering and moderation support it needs considering the vast scale of the project, and the fact that it's installed on every Windows PC by default, or it should shut it down. |
Beta Was this translation helpful? Give feedback.
-
|
@Jaifroid I've been out, and I'm just catching up on things. I couldn't agree more regarding the work of moderators. @OfficialEsco and @ItzLevvie have been moderating the bulk of manifests since the original automation proved insufficient. We've continued investing in additional automation and have started looking again at a reasonable subset of manifests that could be approved without human review. Namely those that are "obviously" (to code) simply an upgrade to an existing version of a previously approved package from the same publisher. We implemented the code behind the "verified developer" feature many months ago. The true challenge is the business process work, and it's likely to take us a bit longer to work all of that out. We're getting close to turning this feature on for some of our "first-party" packages. We've already identified a few other key third-party "verified developers" We still wouldn't trust any package submission because of the "verified developer" differently than any other in terms of static and dynamic analysis of the package and the metadata though. It would be more about "knowing" a package manifest PR was submitted by the approved GitHub account. In those cases, there would have been an initial human review prior to turning on the automated flow. As long as nothing requiring human review was triggered, the PR would get approved and merged. The goal would be to reduce the likelihood that the manifest is altered against the wishes of the "verified developer". I'm expecting to have something drafted after the holidays to discuss which field changes would fall into this "likely safe to automate" category so we can make sure to get some good peer review. My hope is that it would reduce the need for manual review down to new packages, and when new versions have other material changes. |
Beta Was this translation helpful? Give feedback.
-
This is my experience of how it seems to work on the MS Store. Seems to be based on a percentage of code change between the last installed package and the update. When I submit small code revisions to the MS Store, it seems to go through on an automated process within a couple of hours. Whenever I've made more substantial code revisions, the package gets delayed for (I assume) human review for a day or so. |
Beta Was this translation helpful? Give feedback.
-
|
What about a system like in many linux repositories where a someone can be the maintainer for that manifest and auto merge PR's after the pipelines come up green? Liking winget so far by the way and would be interested in putting my hat in the ring to help out in the manifest space as a community moderator. |
Beta Was this translation helpful? Give feedback.
-
|
We're working on that. For the technical implementation we added: The real challenge is the business process. We need a way for publishers to establish a formal relationship to verify their "ownership" and build an interface where they can manage which GitHub account(s) are authorized to make changes. |
Beta Was this translation helpful? Give feedback.
-
It looks like you're trying to reinvent the wheel! The Microsoft Store has this business process already in place. I know that, because it took quite a long time for our Open Source org to get verified on the MS Store, and we had to provide official documentation of the status of the org, including its registered charity status, etc. From our perspective, it would be ideal if you could re-use the registration already done for the MS Store rather than making us jump through hoops again... |
Beta Was this translation helpful? Give feedback.
-
|
@Jaifroid, we're not trying to reinvent the wheel. We're actually talking with the Partner Center team and another internal team who already vet/validate publishers. We just need some additional work around managing which GitHub identities are allowed to make changes. Organizations are likely to want either a list of approved accounts, and/or the ability to manage which accounts are authorized to make changes. |
Beta Was this translation helpful? Give feedback.
-
|
Any ETA for the changes to materialize? |
Beta Was this translation helpful? Give feedback.
-
|
No, there is no ETA. We're discussing with other teams to make the determination of which solution path is the best, and we will still need the other team to do work on their end to support managing the GitHub accounts authorized to make changes. |
Beta Was this translation helpful? Give feedback.
Many of the most prolific contributors for the Windows Package Manager community repository have put countless hours of their own effort attempting to make this catalog the best it can be. I am mentioning their GitHub aliases below to identify those individuals as candidates for becoming moderators. We have not finalized the process for how they will be selected, These individuals have contributed to the repository for an extended period of time and have authored or corrected many manifests. In addition many of them have guided and coached new users of the community.
@ItzLevvie
@jedieaston
@oxygen-dioxide
@OfficialEsco
I would like to also take the opportunity to mention that I don't beli…