Windows VMs

[baszalmstra]

Let me start by saying thank you for this this initiative, it is fantastic! It is great to see that very complex and resource intensive builds can still be created using the resources provided here.

I am on a journey to try to improve the ML situation on Windows using conda-forge. While trying to build pytorch for windows I quickly ran into the large resource requirements for these builds. Ideally we would also be able to use these servers to create windows builds. It seems to me that the most straightforward way to achieve this is to use Windows VMs.

So my question is: I was wondering if it would be possible to build and use Windows images using this repository? I would be happy to contribute to make this happen.

[jaimergp]

Hey @baszalmstra, thanks for the question!

We haven't looked into this but I anticipate frictions like:

• Figure out Windows OS licensing. It's quite involved and we would need to figure out the billing.
• Figure out whether we are complying with the VC compilers redistribution clauses or whether we also need to pay for that.
• Figure out the contents of the VM image so it mirrors what we usually require in the Azure runners

That's from the top of my head. There might be more technical challenges when it comes to deploying it on OpenStack or things like that.

Tagging @aktech and @isuruf in case they have more details.

[aktech]

Agree with @jaimergp on what needs to be done for Windows.

Licensing is the main hurdle IMO, everything else technically is possible. I would suggest going with windows on cloud (like Azure). Cirun supports windows as well, with disk size customisation. All we need to do is:

• Add Azure credentials to cirun (in conda-forge account)
• Add an entry in .cirun.global.yml for Azure machine config: https://github.com/conda-forge/.cirun/blob/master/.cirun.global.yml
• Add an entry for access control for that machine to only allow pytorch feedstock: https://github.com/conda-forge/.cirun/blob/master/.access.yml
• Optionally (depending on what you need in the runner image), create a VM image and push it to the Azure account and use that for creating runners, we can just use the windows image from github: https://github.com/actions/runner-images

Example

• Here is an example of a workflow running on Azure VM via cirun.io:
  https://github.com/AktechLabs/github-actions-on-windows-azure-example/actions/runs/8601575417/job/23569159758
• Example cirun configuration: https://github.com/AktechLabs/github-actions-on-windows-azure-example/blob/main/.cirun.yml
• Docs: https://docs.cirun.io/os-platform/windows

[wolfv]

I think re-using the Github image would be the best option for us. Spinning it up on Google Cloud would be ideal, but we could arrange ourselves with another platform as well.

I am not sure about the process of using the Github image. Potentially we could also build something more lightweight with just Visual Studio community edition, git and patch.exe installed.

[aktech]

I think re-using the Github image would be the best option for us. Spinning it up on Google Cloud would be ideal, but we could arrange ourselves with another platform as well.

I would suggest going with Azure for now, for two reasons:

• Windows VM support on Azure is better than any other cloud provider.
• I believe the scripts here https://github.com/actions/runner-images/ work better with Azure as they upload to azure and are written in a way keeping azure in mind IIRC.

I am not sure about the process of using the Github image. Potentially we could also build something more lightweight with just Visual Studio community edition, git and patch.exe installed.

That's better if that's all you need in the image, then you don't need to worry about second point above.

[jaimergp]

With custom images, don't we have to worry about re-distributability of the artifacts? IOW, they'd need to be private, kept up to date and in sync with the Azure/GHA runner ones?

[aktech]

With custom images, don't we have to worry about re-distributability of the artifacts? IOW, they'd need to be private, kept up to date and in sync with the Azure/GHA runner ones?

They doesn't have to be private, the scripts can be public and you'll have a workflow to push them to an Azure account. Why do we need to keep them up to date with GHA runner ones? You'd only need to update the things that you "need" to keep updated in the CI, like say you need an update version of git or visual studio.