[Spec] Auto generate manifests for newer versions

[vedantmgoyal9]

I've drafted a spec on how Winget automation could work based on what I’ve built so far, but the field names and other details may need a complete overhaul. Suggestions and feedback are more than welcome!

cc @denelon @stephengillie @Trenly @ImJoakim @ItzLevvie @jedieaston @KaranKad @mdanish-kh @OfficialEsco @quhxl @russellbanks @SpecterShell @PckgrBot

• Related issue: Auto generate manifests for newer versions #500

[wingetbot]

   

[Trenly]

How would software from RSS feeds be handled?

[vedantmgoyal9]

https://devblogs.microsoft.com/scripting/use-windows-powershell-to-parse-rss-feeds - like a normal XML

[stephengillie]

@wingetbot run

[Trenly]

You have detail in here about what the schema looks like, but there isn't anything describing the function of the application itself.

You mention that it runs on a set schedule, how often does it run? Is that something which is configurable per package? Is the application something where a package update can triggered by a user making an API call or will it only operate on the schedule?

Not all of the fields are described in the specification either.
What does the Skip field do? How and when should it be set? Is Reason required, or optional? Is there any length constraints? Any Regex validation? Are any of the fields in the specification enums?

Another consideration - how long should scripts be? How are they validated? Is there a maximum length? Maybe it makes more sense to have the script be a separate file in the same directory as the auto-update metadata, that way PSScriptAnalyzer can be run on it. Should the script be something like the format of a DSC resource where there is a Get, a Set, and a Test method (or something similar) where there is /a function specifying how to Get the URLS, a Set function for specifying how to call wingetcreate, and a Test method for checking the manifest is valid and performing any other needed updates before submission?

This is a good start, but I think it needs a lot more thought and detail around implementation.

[Trenly]

It would probably be better to have a separate directory structure specifically for auto-updates instead of including metadata in the manifests folder. While having the data right next to all of the manifests makes it easy for users to find, it greatly increases the complexity cost of a solution since whatever is built will have to recursively filter through all the directories to find the metadata files instead of filtering to only a known sub-tree

[Trenly]

In these cases, wouldn't it be better to have the ISV's integrate wingetcreate into their CI/CD pipeline? My worry would be that if an official automation is added, publishers would just do nothing and let the automation handle it instead of supporting their own applications.

While polling for a new version works at small scale, it can easily chew up a ton of bandwidth. There are currently ~8,000 package identifiers in the repository; If even 20% of them were polled, that would be 1,600 packages every few hours that need to be queried for updates, and that will only grow as the service grows.

If the manifest is simple enough that wingetcreate update --submit can handle it, then it is trivial for an ISV to integrate wingetcreate into their CI/CD process - turning the flow from a pull to a push

[Trenly]

If scripts are being added to the repository, the scripts will need to be validated somehow to ensure they are safe. There are security implications to running scripts in an automated environment - for example, what if the script tries to leak some process information that could contain secrets?

[Trenly]

Additionally, how are the endpoints that are being called validated for security? It wouldn't be good to have an azure agent talking to a URL that is MitM'ing all the traffic over to a threat actor, just to try and get information about Package Update.

Do domains need to be tested and whitelisted? Does there need to be some form of SmartScreen trust level check?

[Trenly]

Not just GitHub API Rate Limits, but what about other sites that have rate limits such as Mozilla? If they get hit with a few hundred requests quickly enough, they could throttle the connections or toss back 403 or 429 error

[Trenly]

What happens if a publisher removes their API or blocks the user agent or changes the response or the ISV changes the format of the file name? How do downstream breaking changes get surfaced to the repo here? It wouldn't be good to have bad PRs being submitted, but it also wouldn't be good to fail silently

[Trenly]

Isn't this already covered by the schema above, where any endpoint can be used?

[Trenly]

What is the UI / UX of the PRs that are submitted using the automation? Do they appear as normal PRs? Is there something linking them to the automation run? Are there any logs that are generated?

[Trenly]

What is the expected "correctness" of the automation? One thing that comes to mind is the request to be able to push changes onto wingetbot PRs since it isn't always accurate - this would likely have the same issue where if something was inaccurate the PR would have to be either merged or closed and a second PR would have to be opened to address it (as well as potentially a third to actually fix the script)

[stephengillie]

@wingetbot run

[stephengillie]

@wingetbot run

[Trenly]

Marking as needs changes based on my comments above, so that this doesn't get accidentally merged due to Levvie's approvals

[denelon]

We definitely need the schema for these files to be explicitly built and included for JSON schema validation. We're also going to need similar documentation to what we have for manifests so authors can look at these files. There are also going to be extra concerns regarding folks submitting new files and attempting to modify them.

[pl4nty]

Instead of JSON, could the manifests use YAML like app manifests? It would improve readability and simplify multi-line strings, but need an additional PowerShell module

[ItzLevvie]

@wingetbot run

[wingetbot]

Validation Pipeline Run WinGetSvc-Validation-63-236105-20250604-1