[WIP] Preliminary support of meson builds. #488

bsmiles32 · 2017-11-25T02:01:33Z

In order to lower the maintenance burden of the build system (eg. keeping in sync both Makefile and Visual studio files) I've explored the possibility of migrating to the meson build system [1].
So, I've ported the original Makefile to meson with some minor arrangements to fit in the new paradigm.
What I'm looking for now is feedback from other developers to see if there is interest in that work and what can be done to ease the eventual transition to it. I've tested this on my machine (linux, x64) and I'd like to know if it works in other platforms (windows, bsd, mac), cpu (arm, x86, mips?, ppc?) and dev env (cross-compilation, msvc).

In order to test (assuming the use of the ninja backend):

cd <path_to_m64p-core>
mkdir build
cd build
meson ../
ninja

You can also inspect and change build options with

meson configure

Note that this build system relies on python3 which might not be available by default in windows (should we add it to win32-deps repo ?).

Please report any issue, feedback or comment.
When/If we get this repo into shape with meson, I'll also port other repo to it (should be more easy).

cc: @richard42

[1] http://mesonbuild.com/

Gillou68310 · 2017-11-25T11:24:09Z

Tried on windows with msvc but it fails, here's the log:

The Meson build system
Version: 0.43.0
Source dir: D:\Projects\GSI\mupen64plus\mupen64plus-core
Build dir: D:\Projects\GSI\mupen64plus\mupen64plus-core\build
Build type: native build
Project name: mupen64plus-core
Native C compiler: cl (msvc 18.00.31101)
Native C++ compiler: cl (msvc 18.00.31101)
Build machine cpu family: x86
Build machine cpu: x86
Compiler for C supports argument -ffast-math: NO
Compiler for C supports argument -fno-strict-aliasing: NO
Compiler for C supports argument -fvisibility=hidden: NO
Compiler for C supports argument -Wno-unused-function: NO
Compiler for C supports argument -Wno-unused-parameter: NO
Compiler for C++ supports argument -ffast-math: NO
Compiler for C++ supports argument -fno-strict-aliasing: NO
Compiler for C++ supports argument -fvisibility=hidden: NO
Compiler for C++ supports argument -Wno-unused-function: NO
Compiler for C++ supports argument -Wno-unused-parameter: NO
Compiler for C++ supports argument -fvisibility-inlines-hidden: NO
Found Pkg-config: NO

Meson encountered an error in file meson.build, line 192, column 0:
Pkg-config not found.

Also I'm not sure to understand, is this also going to generate a visual studio project file?

bsmiles32 · 2017-11-25T15:51:19Z

@Gillou68310 Thanks for giving a try :)

By default, meson uses the ninja backend, but you can also tell it to generate a vs project file using the --backend {vs,vs2010,vs2015,vs2017}
You might have to run this under the vs command prompt [1].

As for your error about missing pkg-config, I'll need to look a bit deeper into it. Indeed, meson rely on pkg-config to find dependencies. This works well in linux, but in windows, no so much. Two solution here: either we go the source route and use wrap files, or we go the prebuild binary way [2].

[1] http://mesonbuild.com/Using-with-Visual-Studio.html
[2] http://mesonbuild.com/Shipping-prebuilt-binaries-as-wraps.html

richard42 · 2017-11-25T16:20:25Z

My initial thought on this is that I don't like it. Makefiles and MSVC project files are "native" on linux/bsd/windows, so generally they're easiest to use for end users. Adding another system on top adds new things to break and additional dependencies, plus a new thing for developers to learn and maintain.

I think having separate project/make files for linux and windows is not as much of a hassle for developers as the fact that we have so many different modules, each with their own build system. So if a developer is working on a single module, like for instance RSP-HLE, and he or she adds a new file, this only requires changes in 2 places. But if a developer makes a general build system change (like, adding a new unix-like target system to the makefile) it requires changes to N files in N repositories, which is a bigger pain. Going to meson isn't necessarily going to address this issue.

Some projects do have success with other system like cmake, so I'm not saying that I would never accept this type of change. But it's going to be difficult, because at a minimum we need it to support all of the systems on which we currently build, and not introduce any additional complexity.

bsmiles32 · 2017-11-25T17:15:44Z

@richard42 Thanks for joining the discussion

meson can generate ninja files (equivalent to Makefiles), VS solutions (2010, 2015 and 2017) files and XCode files. So normally each developer should get something native on their plateform. Then adding/removing/moving a file is just a matter of editing the meson.build file (eg just as easy as what it is with our current Makefile, and much easier than editing manually a vs project+filter file).

Learning meson is not that hard. I've made this work in less than a week, working on it while commuting. The documentation is okay and in case of doubt there are many unit tests to learn from.

The point you raise about having many repo for each plugin is valid and I think here meson can help a little bit : if we have only 1 meson.build to edit per repo, that's half what we have to edit now (1 Makefile, 1vsproj). Also because meson.build files are "composable" (you can embed projects inside another) we might be able to switch to a single top repo with all core+ui+plugins as subprojects and factor some common build logic in the top project. This could maybe address the current maintenance issue.

Also some benefits I see with this transition :

many projects have reported a much better build time by switch to meson, especially on Windows.
better tracking of the compilation chain, meaning that when I make a change in a file or in a build option, meson will only rebuild what's really needed. I don't need to do a "make clean & make DEBUG=1 all" all the time. This speeds up my dev flow quite a bit.
dependency management looks good also. With the possibility to either use the system dependencies or some fallback (be it from source or some prebuilt objects).
Migration to other newer Visual Studio version will not require a significant effort anymore, just switch backend. And when I add a new directory, I don't need to care about some UniqueIdentifier in the filter file anymore.

Anyway, we don't have to switch to meson overnight. It can coexists (eventually in a separate branch if you don't wan't it in master yet) for some time until it has proven its value to other devs.

bentley · 2017-11-30T06:36:27Z

meson.build

+# Add math library portably (see meson FAQ)
+libm = cc.find_library('m', required : false)
+# Use minizip when available, use embedded version otherwise
+# XXX: use the dependency fallback mecanism ?


“mechanism”

bentley · 2017-11-30T06:36:53Z

meson.build

+# Apparently FreeBSD needs this
+if host_system == 'bsd'
+  cflags += ['-DIOAPI_NO_64']
+endif


bentley · 2017-11-30T06:38:36Z

meson.build

+    'src/osal/files_unix.c']
+  # On unix platforms we also need libdl
+  libdl = cc.find_library('dl')
+  extra_dep += libdl


As far as I can tell libdl is not required by POSIX, and the BSDs (and likely others) do not have it. This should be wrapped in “if linux”.

bentley · 2017-11-30T06:43:55Z

meson.build

+endif
+if with_dbg_timing
+  cflags += ['-DPROFILE']
+  extra_dep += cc.find_library('rt')


OpenBSD keeps librt entirely in libc, so there is no librt to find. It’s the odd one out in this regard, so this should be wrapped in “if not openbsd”.

bentley · 2017-11-30T06:45:12Z

meson.build

+  endif
+
+  # Select appropriate format
+  if 'linux bsd'.contains(host_system)


This does not match OpenBSD (“openbsd” does).

bentley · 2017-11-30T06:45:36Z

meson.build

+  elif host_system == 'windows'
+    nasm_fmt = 'win' + (host_cpu.endswith('64') ? '64' : '32')
+  else
+    error('Unsupported system')


A warning would be more friendly than an error here. Is nasm a hard dependency?

nasm is a hard dependency (provided that you enable dynarecs), so erroring is the right thing.

richard42 · 2017-12-01T03:56:27Z

There are some interesting potential advantages there. However I would suggest moving the meson files into a sibling directory of the other projects, ie /project/meson/, instead of the root, to avoid cluttering up the base folder.

bsmiles32 · 2017-12-03T20:17:11Z

Rebased against master. Tried to use the fallback parameter for dependencies. Tried to fix some concerns from @bentley.
@richard42 I'm not sure meson supports meson.build in non source root directory.

Moved some "external" files outside of src dir, but didn't update Makefile and VisualStudio. So build failures with them is expected.

bentley · 2017-12-04T06:02:52Z

meson.build

+  endif
+
+  # Select appropriate format
+  if 'linux bsd openbsd'.contains(host_system)


I doubt “bsd” picks up FreeBSD either, but I don’t use FreeBSD so I can’t verify. You probably want to bring acm@freebsd.org into this discussion.

I default now to ELF format so I can avoid this comparison

bentley · 2017-12-04T06:08:29Z

subprojects/minizip/meson.build

+# Disable 64-bit IO on bsd
+if host_machine.system() == 'bsd'
+    c_args += ['-DIOAPI_NO_64']
+endif


Same BSD detection problem as before. I would reword the comment to “non‐Linux systems don’t require nonstandard API for 64-bit I/O”, and switch the conditional to “if not Linux”.

bentley · 2017-12-04T07:39:53Z

meson.build

+endif
+
+# XXX: linking flags to export api symbols
+library('mupen64plus-core',


The library is called “libmupen64plus”, not “libmupen64plus-core”. Also, this should be shared_library(), not library() (see next comment).

Renamed to mupen64plus. However I kept library, as you can change this using the 'default_library' option.

bentley · 2017-12-04T07:47:36Z

meson.build

+    link_args : vflag,
+    link_depends : vscript,
+    dependencies : [libm, zlib, libpng, sdl, minizip, md5, xxhash, extra_dep],
+    version : meson.project_version(),


The version needs to be overrideable. This requires two things:

A version variable early in meson.build, libmupen64plus_version = meson.project_version()

This line to be changed to version : libmupen64plus_version,

You can test that you have it right by running “env LIBmupen64plus_VERSION=0.0 meson”. If it works, instead of libmupen64plus.so, libmupen64plus.so.2, libmupen64plus.so.2.5.0, you will get a single file libmupen64plus.so.0.0.

Did what you suggested, but couldn't override using env variables

bsmiles32 · 2017-12-16T16:53:06Z

Rebased against master, addressed some concerns from @bentley, and tried to update CI files, but it looks like meson documentation is not up-to-date (http://mesonbuild.com/Continuous-Integration.html). Will look into that later.
@Gillou68310 Can you try again building with msvc to see if the fallback subprojects work please ?

bsmiles32 · 2018-08-25T15:05:56Z

I think I've now reached parity with Makefile+VS2013 projects as both now build on Appveyor and TravisCI.
I also got the MXE cross compilation working on Travis.
However, I had to disable LTO with Clang and MXE. For the former I hit LLVMgold plugin error, which I think is due to using an old version of Ubuntu. For the latter, I get unresolved symbols on my own subprojects symbols (md5, minizip, oglft, ...) and some warning about LTO and "ar". So I disabled these for now.
Freetype library has not been yet properly imported as a meson subproject, so you either need to get it through pkg-config or use the prebuilt version in mupen64plus-win32-deps.
As a small bonus I added the generation of pkg-config file to ease consumption of this library by others. This has been asked recently #562.

I could still do some more cleanings and refinements here and there, but I feel like asking for some feedback first could be beneficial.

My porting to meson has been quite good overall even if a bit hard at times :

the meson documentation and test suite are very good resources to learn how to do things with meson
it has helped me finding some errors and bugs such as Update nasm path to use latest stable release 2.13.03 #583, Bugfixes #581 and we got a huge list of warnings now (maybe I should reduce the warning level for now to avoid compiler spam for casual user/contributor, and tackle in my own fork the warnings fixing)
the first difficulty was getting dependencies right. meson subprojects and wraps have been really helpful here and I'd like to go a little bit further so we can have all of our dependencies as subprojects (and even some as wrap). These would allow full flexibility regarding dependencies, eg. have them static or dynamic, have them properly configured (which features to enable/disable) ... without relying on the package manager which some system don't support (Windows, iOS?).
the second difficulty was getting things to build on Appveyor and TravisCI. Apart from the difficulty of getting the dependencies right, the main issue was circumventing the fact that these mostly provide old version of software (ninja, meson). So I need to fetch them from elsewhere which makes the build and configuration more complex (and longer) than it should be. Yet we could mitigate these downloads times by caching some of them, I think Appveyor and TravisCI support that.

So yeah, now is a good time to test this PR, so I can fix the remaining concerns.
cc : @richard42, @loganmc10 @Gillou68310 @bentley @fzurita

bsmiles32 · 2018-09-23T23:02:07Z

Added git-based freetype2 wrap (2.9.1)
updated with master

bsmiles32 · 2018-11-11T20:56:28Z

I've also had some success building using meson on my Raspberry Pi 3 (aarch64). There are still some stuff that are not supported (no_asm=1) and some dependencies still have some troubles getting compiled as subprojects from meson (~~libpng~~ and sdl2) but eventually will get there :)

update: PR pending for libpng (mesonbuild/libpng#9)

This reverts commit a2c3549. Causes build failure on Windows LINK : fatal error LNK1181: cannot open input file 'subprojects\\xxHash-0.6.5\\xxhash.lib' Will investigate later.

haydenmc · 2023-05-18T03:05:59Z

I'm messing with a little side project that consumes mupen64plus and just want to register my support for this change, though I know it's an old one! Being able to reference this as a meson subproject would greatly simplify my build.

Were there any significant blockers to getting this merged, or does it just need some help across the finish line?

richard42 · 2023-05-18T16:33:44Z

it's not just a little piece of work to get it across the finish line; it's more like we need a champion to take this on and commit to maintaining it in the future, because I'm happy enough just continuing with the MSVC and makefile build systems. I don't have interest in learning and maintaining an entirely different build system at this point.

bentley suggested changes Nov 30, 2017

View reviewed changes

bsmiles32 force-pushed the meson branch from 108ddca to 6046b60 Compare December 3, 2017 20:11

bentley suggested changes Dec 4, 2017

View reviewed changes

bsmiles32 force-pushed the meson branch from 6046b60 to 66df9a7 Compare December 16, 2017 16:39

bsmiles32 force-pushed the meson branch 19 times, most recently from 545c24b to 121c5a6 Compare December 17, 2017 18:06

bsmiles32 force-pushed the meson branch 6 times, most recently from 3473cd6 to 34c5a54 Compare August 25, 2018 13:59

Preliminary meson support.

4ffdfe4

bsmiles32 force-pushed the meson branch from 34c5a54 to 4ffdfe4 Compare August 27, 2018 18:54

bsmiles32 added 5 commits August 28, 2018 21:15

Merge branch 'master' into meson

02ec025

Merge branch 'master' into meson

1e4752b

Add git based freetype2 wrap

8179e48

Merge branch 'master' into meson

a55821e

Update appveyor script ?

aca01f9

bsmiles32 added 3 commits November 11, 2018 10:05

Add _CRT_SECURE_NO_WARNINGS define when compiling with msvc

39cc258

Merge branch 'master' into meson

5930c50

Use centricular port of freetype2

33110a3

bsmiles32 added 6 commits November 24, 2018 22:13

Merge branch 'master' into meson

ffc50f8

Update libpng wrap revision.

dec3c4a

Use freetype2 wrap from wrapDB.

46a5c28

Use xxhash wrap from wrapDB.

a2c3549

Add support for OpenCV >= 4.0.0

411218f

Revert "Use xxhash wrap from wrapDB."

5dfb9a1

This reverts commit a2c3549. Causes build failure on Windows LINK : fatal error LNK1181: cannot open input file 'subprojects\\xxHash-0.6.5\\xxhash.lib' Will investigate later.

Gillou68310 force-pushed the master branch from 9596180 to 0666982 Compare January 29, 2019 15:57

Mastergatto mentioned this pull request Mar 1, 2020

Improve glue code generation #729

Merged

bsmiles32 mentioned this pull request Dec 11, 2020

Cant build on mac #805

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Preliminary support of meson builds. #488

[WIP] Preliminary support of meson builds. #488

bsmiles32 commented Nov 25, 2017

Gillou68310 commented Nov 25, 2017

bsmiles32 commented Nov 25, 2017

richard42 commented Nov 25, 2017

bsmiles32 commented Nov 25, 2017

bentley Nov 30, 2017

bentley Nov 30, 2017

bentley Nov 30, 2017

bentley Nov 30, 2017

bentley Nov 30, 2017

bentley Nov 30, 2017

bsmiles32 Dec 3, 2017

richard42 commented Dec 1, 2017

bsmiles32 commented Dec 3, 2017 •

edited

bentley Dec 4, 2017

bsmiles32 Dec 16, 2017

bentley Dec 4, 2017

bsmiles32 Dec 16, 2017

bentley Dec 4, 2017

bsmiles32 Dec 16, 2017

bentley Dec 4, 2017

bsmiles32 Dec 16, 2017

bsmiles32 commented Dec 16, 2017

bsmiles32 commented Aug 25, 2018

bsmiles32 commented Sep 23, 2018

bsmiles32 commented Nov 11, 2018 •

edited

haydenmc commented May 18, 2023

richard42 commented May 18, 2023

[WIP] Preliminary support of meson builds. #488

Are you sure you want to change the base?

[WIP] Preliminary support of meson builds. #488

Conversation

bsmiles32 commented Nov 25, 2017

Gillou68310 commented Nov 25, 2017

bsmiles32 commented Nov 25, 2017

richard42 commented Nov 25, 2017

bsmiles32 commented Nov 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richard42 commented Dec 1, 2017

bsmiles32 commented Dec 3, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bsmiles32 commented Dec 16, 2017

bsmiles32 commented Aug 25, 2018

bsmiles32 commented Sep 23, 2018

bsmiles32 commented Nov 11, 2018 • edited

haydenmc commented May 18, 2023

richard42 commented May 18, 2023

bsmiles32 commented Dec 3, 2017 •

edited

bsmiles32 commented Nov 11, 2018 •

edited