Windows port cmake build

[kdschlosser]

I am wanting to see if there would be interest in adding support to compile on windows using make and cmake. There is a windows binary of make that can be used to keep with how the rest of the ports are compiled. I am talking about a native windows build using MSVC and not mingw or wsl. It would require visual studio be installed or the Visual Studio build tools. I have written a python script that is able to locate and setup a proper build environment for Windows which can be used to help things out. cmake is available as part of Visual Studio so no problem there either.

This could be further expanded to compile for stm32, the rp2 and other MCU's as well. There are requirements that would need to be done to make it work. Thing like installing the arm toolkit for windows if compiling for stm32 or rp2 and the esp-idf toolkit if compiling for esp32. This is able to be done now that all of the various pieces have been made available.

arm toolkit
https://developer.arm.com/downloads/-/gnu-rm

esp-idf toolkit
https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/windows-setup.html

gnu make for windows
https://gnuwin32.sourceforge.net/packages/make.htm

sed for windows
https://gnuwin32.sourceforge.net/packages/sed.htm

That's to get the basic idea that everything is available.

Now make doe4sn't know anything about MSVC which is why cmake would be used. Make could check the OS and then launch cmake provided there is windows support in the current cmake files in the ports that use cmake and cmake files would need to be added to the ports that use make.

As a start off point it would be nice if the windows port was able to be compiled with a user c module and it should be able to be compiled without using msbuild. I don't know cmake or make for that matter makefile well enough to write those files. I do know msvc. If I could get together with someone that knows cmake we could get a build system going to support compiling natively on windows across all ports. After all Windows does have 72% market share for PC's and not having the ability to compile on Windows without needing to jump through hoops with cross compiling would be a nice addition to MicroPython. It would make it a lot easier for a lot of folks to be able to do that.

The tools are available to get it done....

Anyone that has interest in having this added speak up and let the devs know by responding to to post. If there is anyone that has knowledge that would help get it off the ground and you want to share here is a good place to start. If you want to lend a hand with the coding, make it known here...

I was going to open an issue for this and then I though it would be a better place to see if there was any interest in it and what the help situation looked like as well.

I do have everything compiling properly on Linux and also on macOS and the last piece is Windows.

[stinos]

I'd personally stay away from make for the msvc port, mainly because it's not super standard and we have already 2 ways to build working executables for Windows. As such I don't see any benefit of maintaining more build tool scripts, but I might be missing something there - what would be the key reasons why such make-based build would be beneficial (and/or easier than what we have now)? And/or for having make launch cmake?

Same question for cmake; I'm not sure if it brings much to the table?

Unless the end goal is to have a single cmake-based build for all PC/Mac based ports (i.e. unify the unix and windows ports into one cmake 'PC' port). While that makes it a bit harder and less convenient to 'just build' on windows I do think it would be pretty nice overall..

I have written a python script that is able to locate and setup a proper build environment for Windows which can be used to help things out.

Unless sticking to > VS2022 this is notoriously finicky with the various different versions of build tools and Windows SDK. If anything, this should imo be offloaded to an external tried&tested tool (or maybe that's what you did and your script just calls into that?). But then it would likely be easier to go for cmake instead because it should always work in that regard.

This could be further expanded to compile for stm32, the rp2 and other MCU's as well.

I'd think this would be a more interesting goal: 'one-click' builds on windows for other platforms. As in: it'll download and/or update the build tools for you unless you have them already then build. But that's more of a job for a basic script (Python or Powershell) than make/cmake.

[kdschlosser]

If the entire build system was unified into cmake it would make it vastly simpler and easier in every which way. There would be no need to have .mk files and then .cmake files that for all intents and purposes does the same thing. Most of the build system is already coded for cmake. it's the ports that aren't. pretty much everything else is.

[kdschlosser]

go look at the build system you will see that the majority of it is already coded for cmake.

[stinos]

Personally, my preferred compile system for all ports is docker containers

@andrewleech right, didn't you or Matt have a repository with docker configs or something like that?

Side note; I've used cmake a bit. I loath it.

I get what you mean, but on the other hand: I have used both, but not repeatedly nor enough to get to actually know them by heart, and then the key difference for me is: with cmake I at least get the idea that I know what I'm doing after a couple of minutes. Whereas with make I first have to decipher its archaic syntax :) As an actual argument for/against that doesn't really count though..

[andrewleech]

@andrewleech right, didn't you or Matt have a repository with docker configs or something like that?

Yep quite a few years ago I got a CI system building all boards and all ports, including windows and Mac Unix builds, all in docker containers on Gitlab. This was before GitHub actions existed, once that started being used here I stopped maintaining my repo. But the docker containers all still work fine to build all the ports.
I recently updated my Mac docker cross compiler for mpy-cross to add support for the Apple silicon / arm machine too.
When time permits I'd like to get a wiki page set to to describe how these can be used easily (without any changes to micropython codebase)

[andrewleech]

@kdschlosser I hear you clearly, however I've been trying to calmly share my experience with you of what does already work with micropython, without new build systems / practices needing to be written by an unknown cmake expert.

I already build user C module on Unix / windows port with make, cross compiled with mingw in a docker container.
In the past I've also used SDL and many other native windows libraries with the same mingw cross compiler on Linux build machines, it's a standard practice in the professional sw industry. I even sign mingw compiled windows applications with authenticode certificates as work (using Linux tools) that we then run and distribute on windows. This all works already. .

I'm well aware there are already duplicated build scripts throughout the codebase for make and cmake, you will see commits from me on many of them in the git history.
The cmake fragments were ported from the original makefile ones just to support esp and rp2 ports. Makefile is still considered the original and primary build system by the maintainers I speak to, cmake is begrudgingly updated to match when needed.

[stinos]

and what are the 2 working builds for Windows?

See https://github.com/micropython/micropython/blob/master/ports/windows/README.md: the msvc (msbuild as build system, so using the .vcxproj files) and mingw-w64 (make as build system, Makefile is very much like the one of all other ports).

If a user c module is not able to be compiled and a user is not able to freeze modules into it then it's incomplete/doesn't work properly.

Unless I'm missing something both support freezing and the mingw-w64 one does support user C modules. Although the msvc one also supports variants and other customizations so it's not like it's impossible to get external code into it. If you think user C modules are a worthy addition for the msvc port then please raise an issue for that.

Like Andrew says: if one of the reasons for wanting a cmake-based build is the lack of a feature in a windows port, then implementing then implementing that feature in the current build system seems like the sane thing to do, instead of implementing something completely new from scratch.

nor am I am to compile MicroPython to run on Windows.

Please raise an issue (or start a discussion) for this, because as far as I (and the CI builds) know everything is fine.

I CANNOT CROSS COMPILE BECAUSE OF USING SDL

That make little sense. SDL should work on all major platforms? And if not cross-compilation then with mingw-w64? And that micropython port does user C modules (again, as far a I'm aware, last time I tried it did) so should be a viable option?

If something is made to run on windows it should be compiled using MSVC because of the CLR

You probably mean CRT not CLR, but still that's not very accurate since other compilers (intel, gcc, ...) can use that.

If the entire build system was unified into cmake it would make it vastly simpler and easier in every which way

From the user's point of view, probably. But as far as maintaining it goes I'm not sure it's going to actually be simpler let alone easier if you take everything into account (custom build steps, the documentation, having to learn a new tool, ...). And in any case it would be a major effort (just guessing this is going to go towards multiple man-months for all ports probably).