[vcpkg-ci-meson] New test port #38658

dg0yt · 2024-05-09T12:45:47Z

Check canonical use of builtin dependency providers.

meson issues:

cmake dependency method return debug lib paths in release config.
Reason: CMake is run without CMAKE_BUILD_TYPE.
Proposed: Set CMAKE_BUILD_TYPE. Vcpkg port vcpkg-tool-meson.
Cherry-picked into [vcpkg-tool-meson] Fix warnings, cmake, llvm 18 #38796.
intl failing on windows-static triplets
Reason: meson not using iconv linking requirement unless asking for static intl.
Proposed: add iconv dep when linking fails. Meson.
Patch: dependencies-misc-diff.
intl failing on apple
Reason: meson not using CoreFoundation linking requirement.
Proposed: add foundation framework dep when linking fails. Meson.
Patch: dependencies-misc.diff.
~~Not using vcpkg cmake toolchain, ignoring vcpkg cmake wrappers~~
Ignoring $<CONFIG:...> elements.
Observed for the intl cmake method (vcpkg patch): Intl::Intl is not configured with Iconv::Iconv as expected.
Reason: Meson's cmake dependency parser cannot handle generator expression in interface link libraries (even not the typical generated $<LINK_ONLY:>): Meson not respecting config dependent generator expressions in INTERFACE_LINK_LIBRARIES from CMake dependencies. mesonbuild/meson#8232
Proposed: Extend generator expression support. Meson.
Patch: cmake-generator-expressions.patch.
Link check doesn't use WindowsApp.lib on uwp
Observed for iconv, despite WindowsApp.lib being in the crossfile's winlibs.
Reason: A link check is more bare-bones than generating build rules for executable target. The link check needs explicit link args.
Proposal: Add winlibs to link checks. Meson.
Patch: winlibs.patch.
CMake dependency provider not handling -framework FRAMEWORK link libraries.
Observed after adding -framework CoreFoundation in vcpkg wrapper for FindIntl.cmake.
Reason: -framework arguments are not recognized as linker options, and cannot be resolved as CMake targets.
Proposal: Treat them like -lLIB. Meson.
Patch: cmake-trace-framework.diff.
CMake dependency provider doesn't establish proper link lib order. Most linkers on unixish platforms need independent libs last.
Observed for intl/iconv/charset in x64-mingw-static, although without raising an error.
Reason: Meson sorts libs alphabetically, after losing order while forming a set.
Proposal: Remove lib list cleanup. (Maybe remove early duplicates). Meson.
Patch: cmake-no-sorted.diff.
CMake dependency provider doesn't establish proper link lib order. Most linkers on unixish platforms need independent libs last.
Even with sorting removed, the result will be wrong:
Reason: link libs are added as soon as reaching a target in the dependency digraph. This is not the required topological sort.
Proposal: Implement topological sort. Meson.
Patch: TBD (FTR I worked on this topic recently in pkgconf.)

dg0yt

dg0yt · 2024-05-13T18:43:06Z

ports/vcpkg-tool-meson/meson-intl.patch

+-            if self.static:
+-                if not self._add_sub_dependency(iconv_factory(env, self.for_machine, {'static': True})):
+            code = '''#include <libintl.h>\n\nint main() {\n    gettext("Hello world");\n}'''
+            if self.static or not self.clib_compiler.links(code, env, dependencies=[self], extra_args=self.clib_compiler.get_option_link_args(self.clib_compiler.get_options()))[0]:


Fixing meson intl detection on windows-static...
... but still not using WindowsApp.lib from c(pp)_winlibs.
Reference: mesonbuild/meson#7630 (comment) (from 2020!).

dg0yt · 2024-05-13T18:47:15Z

ports/vcpkg-tool-meson/vcpkg_configure_meson.cmake

@@ -356,7 +356,7 @@ function(vcpkg_configure_meson)
        set(meson_input_file_${buildname} "${CURRENT_BUILDTREES_DIR}/meson-${TARGET_TRIPLET}-${suffix_${buildname}}.log")
    endif()

-    vcpkg_list(APPEND arg_OPTIONS --backend ninja --wrap-mode nodownload -Dbuildtype=plain -Doptimization=plain)
+    vcpkg_list(APPEND arg_OPTIONS --backend ninja --wrap-mode nodownload -Doptimization=plain)


Original form was suggested in mesonbuild/meson#11257 (comment) (from Jan 2023), and added despite the config warning it generates.
Maybe the original issues was fixed with 1.0.1 (Feb 2023).

It suffices to only pass -Dbuildtype=plain, I think. I may have missed a / in that comment and forgot to reply?

The general options here are combined with per-build-type options which ATM control meson's -Ddebug.

Let's say the key goal is to use the pristine flags provided by vcpkg while keeping meson informed about the type of build.

I see now that there is also a build type custom which would be implicitly activated by -Doptimization=plain -Ddebug=true.
https://mesonbuild.com/Builtin-options.html#details-for-buildtype

dg0yt · 2024-05-13T19:00:18Z

(Sorry AT eli-schwartz, I pressed submit too early.)

dg0yt · 2024-05-15T07:16:03Z

Effect of *LINK_ONLY* patch, x64-mingw-static on linux host:

Guessed CMake target 'Intl::Intl'
CMake TARGET:
  -- name:      Intl::Intl
  -- type:      INTERFACE
  -- imported:  True
  -- properties: {
      'INTERFACE_INCLUDE_DIRECTORIES': ['/vcpkg/vcpkg/installed/x64-mingw-static/include']
      'INTERFACE_LINK_LIBRARIES': ['/vcpkg/vcpkg/installed/x64-mingw-static/debug/lib/libintl.a', '*LINK_ONLY*Iconv::Iconv']
     }
  -- tline: CMake TRACE: /vcpkg/vcpkg/downloads/tools/cmake-3.29.2-linux/cmake-3.29.2-linux-x86_64/share/cmake-3.29/Modules/FindIntl.cmake:178 add_library(['Intl::Intl', 'INTERFACE', 'IMPORTED'])
CMake TARGET:
  -- name:      Iconv::Iconv
  -- type:      INTERFACE
  -- imported:  True
  -- properties: {
      'INTERFACE_INCLUDE_DIRECTORIES': ['/vcpkg/vcpkg/installed/x64-mingw-static/include']
      'INTERFACE_LINK_LIBRARIES': ['/vcpkg/vcpkg/installed/x64-mingw-static/debug/lib/libiconv.a', 'Iconv::Charset']
     }
  -- tline: CMake TRACE: /vcpkg/vcpkg/downloads/tools/cmake-3.29.2-linux/cmake-3.29.2-linux-x86_64/share/cmake-3.29/Modules/FindIconv.cmake:178 add_library(['Iconv::Iconv', 'INTERFACE', 'IMPORTED'])
CMake TARGET:
  -- name:      Iconv::Charset
  -- type:      INTERFACE
  -- imported:  True
  -- properties: {
      'INTERFACE_LINK_LIBRARIES': ['/vcpkg/vcpkg/installed/x64-mingw-static/debug/lib/libcharset.a', '']
      'INTERFACE_INCLUDE_DIRECTORIES': ['']
     }
  -- tline: CMake TRACE: /vcpkg/vcpkg/installed/x64-mingw-static/share/iconv/vcpkg-cmake-wrapper.cmake:10 add_library(['Iconv::Charset', 'INTERFACE', 'IMPORTED'])
Include Dirs:         ['/vcpkg/vcpkg/installed/x64-mingw-static/include']
Compiler Options:     []
Libraries:            ['/vcpkg/vcpkg/installed/x64-mingw-static/debug/lib/libcharset.a', '/vcpkg/vcpkg/installed/x64-mingw-static/debug/lib/libiconv.a', '/vcpkg/vcpkg/installed/x64-mingw-static/debug/lib/libintl.a']

IMO the final list of libraries is in wrong order, but /usr/lib/ccache/x86_64-w64-mingw32-gcc doesn't complain when linking. 🤔

dg0yt · 2024-05-15T07:22:16Z

IMO the final list of libraries is in wrong order, but /usr/lib/ccache/x86_64-w64-mingw32-gcc doesn't complain when linking. 🤔

OMG, our version of meson sorts the libs alphabetically. Fixed in mesonbuild/meson@05f4e0d because it broke Apple frameworks, but plain static linking also needs to respect order.

Neumann-A · 2024-05-15T07:27:15Z

Yeah fun with meson o.O. The self proclaimed build system of the future..... just feel free to patch the hell out of it until it learns to behave :)

eli-schwartz · 2024-05-15T12:33:50Z

@dg0yt considering there is no complaint when linking, one may theorize that meson knows what it is doing and this isn't a problem after all.

I would elaborate on what exactly I mean by that (I had a lengthy comment prepared last night), but it seems the peanut gallery has found your PR and reduced my ability to care.

Instead I'll leave @Neumann-A with the following insight:

/usr/bin/ld: cannot find -lintl-NOTFOUND

Truly, we live in the glorious future of build systems, within which perfection has been reached and the One True Way has been discovered.

This message has been sponsored by CMAKE(tm), the ISO standard build system of the 21st century. Accept no substitutes.

Neumann-A · 2024-05-15T14:09:15Z

@dg0yt If you touch meson anyway you might want to directly update to 1.4.0 and investigate what issues appear with that version. I have another change in #37409 which allows to setup extra properties in meson but that probably has to wait until another port actually requires it.

don't know why eli-schwartz feels personally attacked by my previous comment but it seems like you have to carry the meson behavior discussion to the meson github. (Probably also makes it more discoverable for other people having a similar issues/questions)

yurybura · 2024-05-15T14:18:03Z

@dg0yt If you touch meson anyway you might want to directly update to 1.4.0 and investigate what issues appear with that version. I have another change in #37409 which allows to setup extra properties in meson but that probably has to wait until another port actually requires it.

I've tried to update meson to 1.4.0 in #37599. LLVM update is blocked by mesa, mesa build is blocked by meson...

dg0yt · 2024-05-15T19:54:59Z

I do have some sympathy for, and some disappointments from, both CMake and meson. And the same is true for vcpkg, maybe even worse.

These disappointments are no excuse for disrespectful communication.

eli-schwartz · 2024-05-16T01:23:14Z

I do have some sympathy for, and some disappointments from, both CMake and meson. And the same is true for vcpkg, maybe even worse.

Software is, indeed, complicated and occasionally frustrating. I'm not aware of anyone that ever achieved perfection.

Which does make it a bit annoying when I'm pinged on vcpkg <--> meson integration issues and then have to deal with people that to my knowledge have never once attempted to be cordial with the meson project, who show up exclusively in this issue to make... non-cordial jokes.

Perhaps it will be better going forth -- it seems he's blocked me on github and I can only assume intends to not come asking me for help with meson in the future.

Neumann-A · 2024-05-17T06:17:27Z

ports/vcpkg-tool-meson/meson.template.in

@@ -20,6 +20,7 @@ pkg-config= ['@PKGCONFIG@']
 [properties]
 cmake_toolchain_file  = '@SCRIPTS@/buildsystems/vcpkg.cmake'
 [cmake]
+CMAKE_BUILD_TYPE = '@buildconfig@'


Better check the toolchain meson generates. Could very well be that it is overriding this value.

This isn't added for fun. I didn't test with Meson 1.4 which has changes for CMAKE_BUILD_TYPE. But at the moment we need this. And ideally the proper configuration is validated by vcpkg-ci-meson.

I did not see any change in 1.4 regarding CMAKE_BUILD_TYPE in the generated toolchain nor the invocation log.

Would it make sense to split this and the changes to vcpkg_configure_meson suppressing the meson warnings out into a different PR?

The list in #38658 (comment) seems quite long and the stuff I mentioned seems to be trivial to merge while the other points might require a lot more iterations and time to solve.

Some splitting could be applied. But this is still in an experimental phase. Basically I'm studying mingw cross builds on linux in order to fix windows and uwp ;-) (How did I get here? Must have been gtk.)

FTR this is the coming change in meson: mesonbuild/meson#12947

Would it make sense to split this and the changes to vcpkg_configure_meson suppressing the meson warnings out into a different PR?

#38796

dg0yt · 2024-05-17T06:41:53Z

scripts/test_ports/vcpkg-ci-meson/project/meson.build

@@ -0,0 +1,23 @@
+project('vcpkg-ci-meson', 'c', version: '1.0')
+
+apple_frameworks = dependency('appleframeworks', modules: 'foundation', required: false)


Workaround.

ports/vcpkg-tool-meson/meson-intl.patch

For cmake, this should come from the dependency.

TARGET_NAME, LINK_ONLY, CONFIG

dg0yt · 2024-05-20T06:28:46Z

ports/vcpkg-tool-meson/winlibs.diff

+
+         cargs += self.get_compiler_check_args(mode)
+        if mode is CompileCheckMode.LINK:
+            # Add <lang>_winlibs, accounting for machhine file


Suggested change

+ # Add <lang>_winlibs, accounting for machhine file

+ # Add <lang>_winlibs, accounting for machine file

dg0yt · 2024-05-21T07:30:15Z

ports/vcpkg-tool-meson/dependencies-misc.diff

+-                if not self._add_sub_dependency(iconv_factory(env, self.for_machine, {'static': True})):
+            code = '''#include <libintl.h>\n\nint main() {\n    gettext("Hello world");\n}'''
+            if self.static or not self.clib_compiler.links(code, env, dependencies=[self])[0]:
+                if not self._add_sub_dependency(iconv_factory(env, self.for_machine, {'static': self.static})):


NB: system iconv on macOS is dynamic lib.

dg0yt · 2024-05-21T07:40:42Z

@eli-schwartz The original post is now updated to reflect sub-issues and proposed fixes, including naming the relevant patches. I understand that the patches need to be transfered to meson PRs. The entry barrier is creating proper unit tests which correspond to what is an end-to-end test in port vcpkg-ci-meson, for the static and uwp vcpkg triplets. I'm not very familiar with python and meson. And I have seen some meson PRs targeting CMake support with a conan background. There is some overlap.

BillyONeal · 2024-05-22T23:59:47Z

/azp run

azure-pipelines · 2024-05-23T00:00:08Z

Azure Pipelines successfully started running 1 pipeline(s).

Cherry-picked and extended from #38658. And a minor refactoring for easier management of patches. llvm 18 changes for #37599, squashed from #37599 (comment).

Neumann-A · 2024-05-23T05:25:18Z

@dg0yt Do you think you'll need more changes to vcpkg-tool-meson? I need to refactor it a bit so that I can get the command line flags out of it without actually calling meson. I need that so that the meson backend in python can be called with the arguments vcpkg would normally use to call meson directly.

Cherry-picked from #38658: Fixes errors like ~~~ CMake Error at /Users/vcpkg/Data/installed/x64-osx/share/ffmpeg/FindFFMPEG.cmake:70 (find_library): Could not find FFMPEG_DEPENDENCY_-F/Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/System/Library/Frameworks_RELEASE using the following names: -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/System/Library/Frameworks Call Stack (most recent call first): /Users/vcpkg/Data/installed/x64-osx/share/ffmpeg/FindFFMPEG.cmake:144 (append_dependencies) /Users/vcpkg/Data/installed/x64-osx/share/ffmpeg/vcpkg-cmake-wrapper.cmake:25 (_find_package) /Users/vcpkg/Data/work/1/s/scripts/buildsystems/vcpkg.cmake:813 (include) CMakeLists.txt:32 (find_package) ~~~

dg0yt · 2024-05-23T10:38:47Z

Do you think you'll need more changes to vcpkg-tool-meson?

@Neumann-A It should only need a cleanup and rebase onto master after integration of the split-out PRs. Then the PR will be ready for review. (I need to see if the current changes are accepted into vcpkg and into meson. Fixing the remaining lib order quirk is out of scope for this PR.)

dg0yt force-pushed the vcpkg-ci-meson branch 2 times, most recently from 2f54ad0 to 33207ce Compare May 9, 2024 20:03

JonLiu1993 self-assigned this May 10, 2024

JonLiu1993 added the category:new-port The issue is requesting a new library to be added; consider making a PR! label May 10, 2024

dg0yt force-pushed the vcpkg-ci-meson branch from 33207ce to f60faf5 Compare May 10, 2024 06:16

dg0yt commented May 13, 2024

View reviewed changes

dg0yt mentioned this pull request May 15, 2024

Meson not respecting config dependent generator expressions in INTERFACE_LINK_LIBRARIES from CMake dependencies. mesonbuild/meson#8232

Open

dg0yt force-pushed the vcpkg-ci-meson branch from 0cfb76a to 84901fd Compare May 15, 2024 07:09

Neumann-A reviewed May 17, 2024

View reviewed changes

dg0yt commented May 17, 2024

View reviewed changes

ports/vcpkg-tool-meson/meson-intl.patch Outdated Show resolved Hide resolved

yurybura mentioned this pull request May 17, 2024

[llvm] Update to 18.1.5 #37599

Draft

7 tasks

dg0yt added 4 commits May 17, 2024 20:46

[vcpkg-tool-meson] Fix warnings

2be62c1

[vcpkg-tool-meson] Control cmake build type

878eba0

[vcpkg-tool-meson] Refactor handling of patches

03c827c

[vcpkg-tool-meson] Backport llvm 18 changes

2287e6f

dg0yt mentioned this pull request May 17, 2024

[vcpkg-tool-meson] Fix warnings, cmake, llvm 18 #38796

Merged

dg0yt added 4 commits May 19, 2024 15:43

[vcpkg-ci-meson] New test port

0079b74

[vcpkg-ci-meson] Link intl with CoreFoundation

7018796

For cmake, this should come from the dependency.

[vcpkg-tool-meson] More generator expression support

62039b8

TARGET_NAME, LINK_ONLY, CONFIG

[vcpkg-tool-meson] Add link libs to dependency checks

196db05

versions

9a39007

dg0yt force-pushed the vcpkg-ci-meson branch from 43f130e to 9a39007 Compare May 19, 2024 16:35

dg0yt added 3 commits May 19, 2024 21:36

[vcpkg-tool-meson] Trim winlibs patch

230b313

[vcpkg-tool-meson] Fix imported cmake deps

f7e22bb

versions

c59ac4c

dg0yt commented May 20, 2024

View reviewed changes

dg0yt added 4 commits May 21, 2024 07:50

[vcpkg-tool-meson] intl:osx needs foundation framework

4477237

[vcpkg-tool-meson] Trace frameworks like libs

faf6b22

versions

1321d55

[gettext-libintl] Link CoreFoundation on apple

26b9a37

dg0yt commented May 21, 2024

View reviewed changes

dg0yt added 2 commits May 22, 2024 08:36

[ffmpeg] Handle -F from libs

e7e27f7

[vcpkg-tool-meson] Handle "-framework FRAMEWORK"

56ffd6d

This was referenced May 22, 2024

[ffmpeg] Handle -F from libs #38858

Merged

[gettext-libintl] Link CoreFoundation on apple #38859

Open

vicroms pushed a commit that referenced this pull request May 23, 2024

[vcpkg-tool-meson] Fix warnings, cmake, llvm 18 (#38796)

b4b8719

Cherry-picked and extended from #38658. And a minor refactoring for easier management of patches. llvm 18 changes for #37599, squashed from #37599 (comment).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vcpkg-ci-meson] New test port #38658

[vcpkg-ci-meson] New test port #38658

dg0yt commented May 9, 2024 •

edited

dg0yt left a comment

dg0yt May 13, 2024

dg0yt May 13, 2024

eli-schwartz May 13, 2024

dg0yt May 13, 2024

dg0yt May 15, 2024

dg0yt commented May 13, 2024

dg0yt commented May 15, 2024

dg0yt commented May 15, 2024

Neumann-A commented May 15, 2024 •

edited

eli-schwartz commented May 15, 2024

Neumann-A commented May 15, 2024

yurybura commented May 15, 2024

dg0yt commented May 15, 2024

eli-schwartz commented May 16, 2024

Neumann-A May 17, 2024

dg0yt May 17, 2024

Neumann-A May 17, 2024

Neumann-A May 17, 2024

dg0yt May 17, 2024

dg0yt May 17, 2024

dg0yt May 17, 2024

dg0yt May 17, 2024

dg0yt May 20, 2024

dg0yt May 21, 2024

dg0yt commented May 21, 2024 •

edited

BillyONeal commented May 22, 2024

azure-pipelines bot commented May 23, 2024

Neumann-A commented May 23, 2024

dg0yt commented May 23, 2024

		@@ -0,0 +1,23 @@
		project('vcpkg-ci-meson', 'c', version: '1.0')

		apple_frameworks = dependency('appleframeworks', modules: 'foundation', required: false)

	+ # Add <lang>_winlibs, accounting for machhine file
	+ # Add <lang>_winlibs, accounting for machine file

[vcpkg-ci-meson] New test port #38658

Are you sure you want to change the base?

[vcpkg-ci-meson] New test port #38658

Conversation

dg0yt commented May 9, 2024 • edited

dg0yt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dg0yt commented May 13, 2024

dg0yt commented May 15, 2024

dg0yt commented May 15, 2024

Neumann-A commented May 15, 2024 • edited

eli-schwartz commented May 15, 2024

Neumann-A commented May 15, 2024

yurybura commented May 15, 2024

dg0yt commented May 15, 2024

eli-schwartz commented May 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dg0yt commented May 21, 2024 • edited

BillyONeal commented May 22, 2024

azure-pipelines bot commented May 23, 2024

Neumann-A commented May 23, 2024

dg0yt commented May 23, 2024

dg0yt commented May 9, 2024 •

edited

Neumann-A commented May 15, 2024 •

edited

dg0yt commented May 21, 2024 •

edited