The Meson Build System

This is a partial implementation of #3001, but not everything.

This series allows meson builtins and project options to be specified in the cross and/or native files. This should make it easier to set up subprojects in a reliable way, as all of the options can be kept in a native/cross file. With file layering a projects can even ship preconfigured options, that the end user can overlay with their own cross/native files.

For mesa we'd like to couple this with the ability to set some builtin options (namely default_library) on a per-subproject basis to control wraps in a predictable way.

This is the second iteration of #9232.

I've done my best to better split the changes into smaller commits although while doing that I ended writing even more changes.

The changes can now be summarized:

Changes in the textual test log output

Test specific metadata was split to multiple lines to improve readability. Visual hints were added to group togehter all the lines related to a specifc test.

Textual test log (too) now respects the and options.

Changes in the test output in the console

The way test details (when being verbose or printing details for a failed test) get printed was updated. Most notable changes are:

• each line is prefixed with the relevant test number
• "headers" and indentation is used to group relevant lines together
• the style in which subtest results are listed changed

Results of individual subtests are displaed only in verbose mode or when printing details of a failed test.

The "live" output (when being verbose and using only one process) now follows the style of the normal verbose output more closely.

The "live" output is now automatically used when running only one test in verbose mode.

The statusline was updated to a multi line one. The status line is also disabled if the size of the terminal is not at least 80 characters wide and 24 rows tall.

The status line in quiet mode was simplified.

I made a recording where I try to demo the changes: https://asciinema.org/a/dE9DsiPzUXJcW5X0q0XyO4vQ9

I have some concers w.r.t these changes:

• Multiple status lines tend to "flicker" a bit when lots of lines are being outputted. Some people might find this annoying.
• There is currently no way to disable "live" output when using only one process (though this is true even withtout these changes). I thought about adding an option for that but too may options for the command is not that great either.
• The first iteration received feedback that the statusline is not nice on some windows terminal. I failed to reproduce this but then again my Windows skills are lacking. I tried these changes with the "command prompt for VS 2019" but didn't notice any issues there. I feel the statusline should just be disabled when using an incapable terminal (think meson test | cat) but without being able to reproduce the issue I don't quite know what to do.
• I've only tried this out in Sourcehut's and GitLab's CI systems. Other CI sytems (Travis, Azures, ...) have not been tested at all.
• Do other people like the change?

I'll mark this PR as draft to avoid premature merging until the above concerns get clarified and we can get a blessing from the relevant people. I'll mention @bonzini and @anarazel here by name since they joined the discussion during the first iteration.

This commit contains the following fixes:

1. When a shared library A does link_with: to static library B, the parts of B used by A will be added to A, and so we don't need to return B in A.get_dependencies() for targets that link to A. This already is the behaviour when a shared library A does link_whole: on B.

2. In situation (1), when generating a pkg-config file for A, we must also not add B to Libs.private for A. This already is the behaviour when a shared library A does link_whole: on B.

3. When a static library A does link_whole: to static library B, we must add the objects in B to A.

4. When a static library A does link_with: to static library B, and B is not installed (which makes it an internal static library), we must add the objects in B to A, otherwise nothing can use A.

5. In situation (4), when generating a pkg-config file for A, we must also not add B to Libs.private for A.

All these situations are tested by the unit test added in this commit.

Closes https://github.com/mesonbuild/meson/issues/3934 Closes https://github.com/mesonbuild/meson/issues/3937

This patch exploits the information residing in ltversion to set the -compatibility_version and -current_version flags that are passed to the linker on macOS.

it also fixes well known rpath issues on macOS with the dylibs generated by meson.

See #1445, #1446, #1451

Hi Jussi,

If you build a shared object with meson, the rpath get sets correctly (or so the docs say), but when I use find_library() and give it a search path, the rpath doesn't get set. This is causing me lot of pain.

I'm currently doing stuff like this:

thirdparty_root = '/usr/local/thirdparty-1.6.0'
thirdparty_lib = [thirdparty_root + '/lib/linux-x64-all',
    thirdparty_root + '/lib/linux-x64-gcc4.8'
]
thirdparty_rpath = thirdparty_lib[0] + ':' + thirdparty_lib[1]

thirdparty_dep = declare_dependency(
    include_directories: thirdparty_inc,
    dependencies: [
        find_library('lib1', dirs: thirdparty_lib),
        find_library('lib2', dirs: thirdparty_lib),
        find_library('lib3', dirs: thirdparty_lib)
    ]
)

Then doing this:

executable('foo', 'foo.cpp',
   ...,
    link_args: '-Wl,-rpath,@0@:@1@'.format(thirdparty_rpath, boost_rpath),
)

Am I doing it wrong?

Currently meson does not provide a way to specify a custom rpath, it removes rpath in install step even if it was specified via LDFLAGS. https://github.com/NixOS/nixpkgs/pull/28444#issuecomment-324033323

Please note: I'm aware about {build,install}_rpath, these are project or target options. I'm talking about something like cmake's CMAKE_INSTALL_RPATH which can be easily passed to cmake while configuring. Even autohell behaves fine with such use case. What I'm asking for is an option like --meson-install-rpath (like --prefix) which would allow users (project consumers, not developers) to build and install packages into a custom prefix without messing with meson.build .

This is needed when you want to install something into an arbitrary prefix to avoid overwriting system stuff, eg. you want to build something with new shiny features straight from git, but you don't want to break your OS.

Please consider adding such a feature.

PS: LD_RUN_PATH et al is not an option, the rest of the system should not know anything "prefixed" installs.

Added a new builtin-option called 'sharedlib-linkmodel' for being able to generate a partial linked object using a shared_library() target.

• when this option value is set to 'standard', the existing shared library linking method is used as is and is the default value.
• when this option value is set to 'partial', required compiler options for generating partial linked object will be selected and few of the 'standard' linking options are skipped. Currently the support is added for armcc, armclang and gnu compilers, a warning will be generated if this option is set to 'partial' while using any other compilers.

This has been requested multiple times in the mesa/xorg community, and apparently in the GStreamer community as well.

This allows a configuration summary to be printed at the end of the configuring phase.

For example:

sec1 = {'driver' : 'foobar', 'OS' : 'Linux', 'API' : 1.7}
...
sec2 = {'driver' : 'dive comp', 'OS' : 'Minix', 'API' : 1.1.2}
...
sec3 = {'with' : {'mesa' : true, 'gbm' : false}}

summary(
  {'Backend' : 'OpenGL'},
  'Server' : sec1,
  'Client' : sec2,
  'Misc' : sec3,
)

Which would print something like:

Configuration Summary:
  Backend = OpenGL
  Server:
    driver = foobar
    OS = Linux
    API = 1.7
  Client:
    driver = dive comp
    OS = Minix
    API = 1.1.2
  Misc:
    with = {'mesa : true, 'gbm' : false}

Fixes #757

This was made primarily so we can build librsvg with Meson, which now works: https://git.gnome.org/browse/librsvg/log/?h=wip/meson, and it actually required fewer hacks than I expected it to.

The integration can and should definitely be better, but I think the best way to get there is to have something that works right now and slowly improve Cargo till we have everything we need. Cargo is not going to magically grow API for giving greater control to build systems; we have to work towards it.

I have been talking to @alexcrichton about the things that Meson would need to integrate better with Cargo, and he has been very receptive and encouraging of our efforts, and in the need for adding features to Cargo that allow Meson to control what it does a bit better.

Here's what you can do right now:

'Extract' build outputs from Cargo.toml files in your source tree with the unstable-cargo module and mostly use them like you would any other target. For example:

cargo = import('unstable-cargo')
# Build the staticlib
librsvg_internals = cargo.static_library('rsvg_internals', toml : 'Cargo.toml')

librsvg = library('rsvg-' + rsvg_api_major, librsvg_sources,
                  dependencies : [librsvg_deps, librsvg_internals])

You can also specify the list of sources if you wish:

cargo.executable('mesonbin',
                 toml : 'Cargo.toml',
                 # This is optional, since ninja keeps track of dependencies on source files.
                 sources : ['src/main.rs'],
                 # Yes, installation works
                 install : true)

The sources list is optional because cargo will output a nice depfile for us and ninja can then keep track of when to rebuild the target.

Meson TODO:

• Cross-compilation
• Cargo tests and benchmarks
• cdylibs or rust-specific library types
• ninja dist does not support vendoring crate sources yet (needed for librsvg)

Cargo TODO:

• https://github.com/rust-lang/cargo/pull/4711
• CARGO_TARGET_DIR should be an argument --target-dir
• Options to set the output library/executable name (prefix/suffix); perhaps also in Cargo.toml
• In general, the ability to more accurately control build outputs from via cargo arguments
  • This is where our work comes in; something half-working is the best way to figure out what is needed to make it work well ;)
• Dry-run downloading of sources by cargo vendor so we can ship a list of files instead of the sources themselves (cc @alexlarsson)

Cargo long-term TODO:

• Output rustc invocations needed to build a crate so we can run everything ourselves
• Overall, Meson should be able to output build artifacts exactly where it wants, and with the rustc flags that we want.

Okay, I stopped thinking about it and started writing code. This is not complete, there's a fair number of things not done (listed below). Currently, this is just enough to build one trivial test case, but I think it touches all the places that need to be changed, even if there is more work to do. I'm not sure how much more time I'll be able to spend on this, but here's a starting point if someone else wants to take it up.

TODO:

• [x] handle a single static source
• [x] handle a single generated source
• [ ] handle a mixture of static and generated sources
• [x] work out the best way to handle python 2 support (if it's needed at all)
• [ ] work out the best way to handle Cython -> C++
• [ ] deal with the fact that filesystem layout matters (also a problem for Rust and Zig)

In ‘x64 Native Tools Command Prompt for VS 2017’, the cpu detected by meson is 'x86' ？

**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.16
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

D:\opt\glib-2.62.2\_build>meson .. --buildtype=release --prefix="D:\opt\glib\"
The Meson build system
Version: 0.52.0
Source dir: D:\opt\glib-2.62.2
Build dir: D:\opt\glib-2.62.2\_build
Build type: native build
Project name: glib
Project version: 2.62.2
C compiler for the host machine: cl (msvc 19.16.27034)
C linker for the host machine: link 19.16.27034
C++ compiler for the host machine: cl (msvc 19.16.27034)
C++ linker for the host machine: link 19.16.27034
Host machine cpu family: x86
Host machine cpu: x86

And I tried in vs2013 , 2015, got the same result.

Describe the bug When specifying the input argument or custom_target or generator with a directory path, it will fail with

ERROR: File <directory path> does not exist.

A build target may depends on a directory both theoretically and practically, and GNU Make does allow directories to serve as dependencies or build targets. In my use case, I would like to generate a file archive for each input directories. There should be a way to do that with Meson so as to cover these use cases.

This was originally posted in discussion in https://github.com/mesonbuild/meson/discussions/11230 . After searching through the documentation without finding a workaround, I guess this should be considered an issue.

To Reproduce

1. Construct the following directories and files:

srcdir
├── gen_zip.sh
├── meson.build
└── sample_hello.source
    ├── empty
    └── hello.txt

1 directory, 4 files

gen_zip.sh

#!/bin/sh

INPUT_DIR="$1"
OUTPUT="$(realpath "$2")"

cd "$INPUT_DIR" || exit 1
find . -mindepth 1 -maxdepth 1 -exec zip -r -y "$OUTPUT" "{}" \+

hello.txt

Hello world!

meson.build

project('my-project', 'c',
version: '0.1.0',
license: 'MIT',
default_options: [
  'buildtype=debugoptimized',
  'c_std=c11',
  'warning_level=3'
])

exe_gen_zip = find_program('./gen_zip.sh')

sample_hello = custom_target('sample_hello',
input: 'sample_hello.source',
output: 'sample_hello.zip',
command: [
  exe_gen_zip,
  '@INPUT@',
  '@OUTPUT@'
])

2. Run

   touch "srcdir/sample_hello.source/empty"
   chmod +x "srcdir/gen_zip.sh"
   

3. Run

   cd srcdir
   meson setup build.tmp
   

4. See error

   meson.build:12:0: ERROR: File sample_hello.source does not exist.
   

Expected behavior

Users should be able to use custom_target and generator.process with directories as input, and the above build steps should run without the file-not-found error.

system parameters

• Is this a cross build or just a plain native build (for the same computer)? This is a native build.
• what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.) NixOS
  • system: "x86_64-linux"
  • host os: Linux 5.15.80, NixOS, 22.11 (Raccoon), 22.11.20221130.596a8e8
  • multi-user?: yes
  • sandbox: yes
  • version: nix-env (Nix) 2.11.0
  • nixpkgs: /nix/store/qwgw2b19ga8dr7c6wfpd6p87bdjyrnha-source
• what Python version are you using: 3.10.8
• what meson --version: 0.63.1
• what ninja --version if it's a Ninja build: 1.11.1

Without get_dependency_gen_args() depfiles are not generated for cuda sources and can result in partial target rebuilds when source dependency changes occur in the project.

The nvcc preprocessor now supports -MD, -MDD, -MF, but still lacks implementation support for -MQ, instead providing only -MT. Supporting -MD and -MF overcomes previous limitations where nvcc required depfile generation in two commands and this was not possible with ninja. Using -MT is not great, but may be better than nothing for the moment. https://github.com/mesonbuild/meson/issues/1003#issuecomment-259103463 https://github.com/mesonbuild/meson/discussions/11227#discussioncomment-4553800

Describe the bug Commit 18bec0d3e31884dc493a03741b2ec9fef17047ee introduces a regression in my build setup with the way it sets PKG_CONFIG_LIBDIR. In this particular configuration I use msys2 pkg-config which expects Unix-style path separator (':') in env vars. It seems that aformentioned commit changes the path separator to windows-style (';') which is no longer recongized by pkg-config and dependency discovery fails. I'm not aware if there's any other pkg-config variant usable on Windows which uses ';' as a separator, msys seems lika a natural choice.

meson-log.txt before the commit:

env[PKG_CONFIG_LIBDIR]: c:\data2\moonlight-deps\windows-clang15\icu4c-71_1\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\cmph-2.0\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\openssl-1.1.1s\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\xxHash-0.8.1\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\zlib-1.2.13\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\zstd-1.5.2\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\flac-1.3.2\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\libogg-1.3.5\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\opus-1.3.1\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\opusfile-0.12\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\libsndfile-1.1.0\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\soxr-0.1.3\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\libvorbis-1.3.7\x64\lib\pkgconfig:c:\data2\moonlight-deps\windows-clang15\yaml-cpp-0.7.0\x64\share\pkgconfig

after the commit:

env[PKG_CONFIG_LIBDIR]: c:\data2\moonlight-deps\windows-clang15\icu4c-71_1\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\cmph-2.0\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\openssl-1.1.1s\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\xxHash-0.8.1\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\zlib-1.2.13\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\zstd-1.5.2\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\flac-1.3.2\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\libogg-1.3.5\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\opus-1.3.1\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\opusfile-0.12\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\libsndfile-1.1.0\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\soxr-0.1.3\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\libvorbis-1.3.7\x64\lib\pkgconfig;c:\data2\moonlight-deps\windows-clang15\yaml-cpp-0.7.0\x64\share\pkgconfig

This causes the discovery of libcrypto to fail:

moonlight-tools| Run-time dependency libcrypto found: NO (tried pkgconfig, system and cmake)

subprojects\moonlight-tools\src\meson.build:2:0: ERROR: Dependency "libcrypto" not found, tried pkgconfig, system and cmake

To Reproduce Please include your meson.build files, preferably as a minimal toy example showing the issue. You may need to create simple source code files (don't include private/proprietary code).

setup command:

 meson setup build -Dbuildtype=release -Dwrap_mode=forcefallback -Db_lto=false -Ddebug=True -Doptimization=3 -Db_ndebug=false -Denable_cuda=true --native-file tools/meson/windows.conf --native-file tools/meson/windows/x64.conf

windows.conf:

[constants]
deps_dir = 'c:\data2\moonlight-deps\windows-clang15'
boost_stem = 'boost_1_80_0'
cmph_stem = 'cmph-2.0'
icu4c_stem = 'icu4c-71_1'
openssl_stem = 'openssl-1.1.1s'
xxhash_stem = 'xxHash-0.8.1'
zlib_stem = 'zlib-1.2.13'
yaml_cpp_stem = 'yaml-cpp-0.7.0'
flac_stem = 'flac-1.3.2'
ogg_stem = 'libogg-1.3.5'
opus_stem = 'opus-1.3.1'
opusfile_stem = 'opusfile-0.12'
sndfile_stem = 'libsndfile-1.1.0'
soxr_stem = 'soxr-0.1.3'
vorbis_stem = 'libvorbis-1.3.7'
zstd_stem = 'zstd-1.5.2'
thinlto_cache_dir = 'c:\ThinLTO'

[properties]
clang_target = 'msvc'
platform = platform
deps_dir = deps_dir
boost_root = deps_dir / boost_stem
pkg_config_libdir = [
    deps_dir / icu4c_stem / platform / 'lib\pkgconfig',
    deps_dir / cmph_stem / platform / 'lib\pkgconfig',
    deps_dir / openssl_stem / platform / 'lib\pkgconfig',
    deps_dir / xxhash_stem / platform / 'lib\pkgconfig',
    deps_dir / zlib_stem / platform / 'lib\pkgconfig',
    deps_dir / zstd_stem / platform / 'lib\pkgconfig',
    deps_dir / flac_stem / platform / 'lib\pkgconfig',
    deps_dir / ogg_stem / platform / 'lib\pkgconfig',
    deps_dir / opus_stem / platform / 'lib\pkgconfig',
    deps_dir / opusfile_stem / platform / 'lib\pkgconfig',
    deps_dir / sndfile_stem / platform / 'lib\pkgconfig',
    deps_dir / soxr_stem / platform / 'lib\pkgconfig',
    deps_dir / vorbis_stem / platform / 'lib\pkgconfig',
    deps_dir / yaml_cpp_stem / platform / 'share\pkgconfig']
cuda_lib_dir = cuda_lib_dir
thinlto_cache_dir = thinlto_cache_dir

[built-in options]
c_link_args = ['-Wl,/lldltocache:' + thinlto_cache_dir]
cpp_link_args = c_link_args
cpp_args = [
    # XXX workaround: boost/type_traits/has_trivial_copy.hpp:34:4: error: builtin __has_trivial_copy is deprecated
    '-Wno-deprecated-builtins',
    # XXX workaround: https://github.com/llvm/llvm-project/issues/56300
    '-D_DISABLE_STRING_ANNOTATION',
    '-D_DISABLE_VECTOR_ANNOTATION']

[project options]
CUDA_dir = 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8'

[binaries]
pkgconfig = 'c:\msys64\usr\bin\pkg-config'
c = ['c:\msys64\mingw64\bin\ccache', 'clang', '--target=' + clang_target]
cpp = ['c:\msys64\mingw64\bin\ccache', 'clang++', '--target=' + clang_target]
c_ld = 'lld'
cpp_ld = 'lld'
strip = 'llvm-strip'
ar = 'llvm-ar'
nm = 'llvm-nm'
ranlib = 'llvm-ranlib'

x64.conf:

[constants]
platform = 'x64'
platform_bits = '64'
cuda_lib_dir = 'lib\x64'
clang_target = 'x86_64-pc-windows-msvc'

(minimized) meson.build:

project('foo', 'cpp',
  meson_version: '>= 0.56.0',
  default_options: [
    'cpp_std=c++20',
    'werror=true',
    'warning_level=2',
    'b_lto=true',
    'b_lto_mode=thin',
])

deps = [
  dependency('libcrypto', static: true, required: true, include_type: 'system'),
  dependency('libxxhash', static: true, required: true, include_type: 'system'),
]
...

Expected behavior I expect the build setup to succeed just like it did before commit 18bec0d3e31884dc493a03741b2ec9fef17047ee

system parameters

• native build
• Windows 10
• Python 3.9.1
• meson --version 0.62.99 (doesn't happen with 0.62.1, happens after 18bec0d3e31884dc493a03741b2ec9fef17047ee)
• ninja --version 1.10.2

clang-cl now supports /std:c++20. I'm not sure exactly when it was added, but given ClangClCPPCompiler does no version checking (at least from a cursory comparison with VisualStudioCPPCompiler), I guess that's not an issue. Anyway, it's certainly there in current (i.e. clang-cl 15.0.1 from VS 17.4.3).

Obviously preferable to use over c++latest as that enables a lot of C++23 stuff...

Here's a quick fix if you want it. Wasn't sure if it warrants a release snippet since it's not really a new feature, but happy to add it.

Thanks

This fixes a bug where meson fails to build on windows due to msys2 containing incorrect build paths. Most of the time Windows simply says that the directory doesn't exist, but in special cases (such as with BitLocker), an exception is thrown that blocks the build.

This catches the exception and converts it to a warning.

How these warnings look like in practice:

C:\envs\venv\Lib\site-packages\mesonbuild\compilers\mixins\gnu.py:497: UserWarning: Error processing directory [WinError -2144272384] This drive is locked by BitLocker Drive Encryption. You must unlock this drive from Control Panel: 'D:\a\_temp\msys64\ucrt64\x86_64-w64-mingw32\lib\x86_64-w64-mingw32\10.3.0'