1724391 - mach configure fails on pkg-config when PKG_CONFIG_PATH is set

Status: RESOLVED FIXED

(Developer Infrastructure :: Developer Environment Integration, defect)

Description

[Alexander]

Attachment: Complete-Mach-Configure-Output-OK.txt

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

./mach configure (fails)
./mach configure --disable-bootsrap NOT FAIL With patch:
hg diff -reverse -r 181a2776e265 build/moz.configure/toolchain.configure
it gets the correct cflags.

Actual results:

ERROR: Command /usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4' failed with exit status 1.
Error running mach: ['configure']

File "/home/magnolia/src/Nightly/python/mozbuild/mozbuild/build_commands.py", line 185, in configure
return driver.configure(
File "/home/magnolia/src/Nightly/python/mozbuild/mozbuild/controller/building.py", line 1528, in configure
status = self._run_command_in_objdir(
File "/home/magnolia/src/Nightly/python/mozbuild/mozbuild/base.py", line 845, in _run_command_in_objdir
return self.run_process(cwd=self.topobjdir, **args)
File "/home/magnolia/src/Nightly/python/mach/mach/mixin/process.py", line 174, in run_process
raise Exception

Expected results:

The failing Command "/usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4' " gives no error when I execute it in the same shell I started "./mach configure". The output is then:
/usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'
-I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/harfbuzz -I/usr/include/freetype2 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/fribidi -I/usr/include/uuid -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/libpng16 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/gio-unix-2.0 -I/usr/include/wayland -I/usr/include/libxkbcommon -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -I/usr/include/libdrm

The same command executed by mach with python3.8 causes the error.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::General' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Alexander]

In the error message is the bug number 1543241 mentioned.
ERROR: Command /usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4' failed with exit status 1.
Error running mach:

['configure']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file configure| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

I'm guessing that this is related to the recent sysroot work.

Comment 4

[Mike Hommey [:glandium]]

Can you attach obj-x86_64-pc-linux-gnu/config.log from a failed attempt?

Comment 5

[Alexander]

Attachment: obj-x86_64-pc-linux-gnu/config.log

I have attached the obj-x86_64-pc-linux-gnu/config.log

Comment 6

[Mike Hommey [:glandium]]

What distro are you on?
Can you provide the output from the following command:

PKG_CONFIG_DEBUG_SPEW=1 PKG_CONFIG_SYSROOT_DIR=$HOME/.mozbuild/sysroot-x86_64-linux-gnu PKG_CONFIG_LIBDIR=$HOME/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/pkgconfig:$HOME/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/pkgconfig:$HOME/.mozbuild/sysroot-x86_64-linux-gnu/usr/share/pkgconfig /usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'

Comment 7

[Alexander]

I use openSUSE Tumbleweed 20210805, KDE Plasma Version: 5.22.4, KDE Frameworks Version: 5.84.0, Qt Version: 5.15.2, Kernel Version: 5.13.8-3.g263a9fe-default (64-bit). I let the default prefix /usr/local unchanged in the firefox buildsystem to not mess up the System standard install path. In openSuse is the prefix /usr default. Not /usr/local like in many other Linux Distros.
Here the output of the command:
/usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'
-I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/gtk-3.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/pango-1.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib64/glib-2.0/include -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/harfbuzz -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/freetype2 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/libmount -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/blkid -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/fribidi -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/uuid -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/cairo -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/pixman-1 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/libpng16 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/gdk-pixbuf-2.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/gio-unix-2.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/wayland -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/libxkbcommon -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/atk-1.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/at-spi2-atk/2.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/dbus-1.0 -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib64/dbus-1.0/include -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/at-spi-2.0 -pthread -I/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/libdrm

No error on ./mach configure with this paths.
But I get a build error instead: The build error disappear when I use --no-bootstrap. Some header files are missing in $HOME/.mozbuild/sysroot dir, firefox-obj-dir or are in the wrong directory.
0:02.40 devtools/platform⏎
2 0:02.40 In file included from /home/magnolia/src/Nightly/accessible/atk/AtkSocketAccessible.cpp:7:⏎
3 0:02.40 In file included from /home/magnolia/src/Nightly/obj-x86_64-pc-linux-gnu/dist/system_wrappers/atk/atk.h:3:⏎
4 0:02.40 In file included from /home/magnolia/src/Nightly/other-licenses/atk-1.0/atk/atk.h:31:⏎
5 0:02.40 In file included from /home/magnolia/src/Nightly/other-licenses/atk-1.0/atk/atkobject.h:23:⏎
6 0:02.40 In file included from /home/magnolia/src/Nightly/obj-x86_64-pc-linux-gnu/dist/system_wrappers/glib-object.h:3:⏎
7 0:02.40 In file included from /home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0/glib-object.h:23:⏎
8 0:02.40 In file included from /home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0/gobject/gbinding.h:28:⏎
9 0:02.40 In file included from /home/magnolia/src/Nightly/obj-x86_64-pc-linux-gnu/dist/system_wrappers/glib.h:3:⏎
10 0:02.40 In file included from /home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0/glib.h:30:⏎
11 0:02.40 In file included from /home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0/glib/galloca.h:32:⏎
12 0:02.40 In file included from /home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0/glib/gtypes.h:32:⏎
13 0:02.40 /home/magnolia/src/Nightly/obj-x86_64-pc-linux-gnu/dist/system_wrappers/glibconfig.h:3:15: fatal error: 'glibconfig.h' file not found⏎
14 0:02.40 #include_next <glibconfig.h>⏎
15 0:02.40 ^~~~~~~~~~~~~~⏎
16 0:02.40 In file included from /home/magnolia/src/Nightly/browser/components/shell/nsGNOMEShellDBusHelper.cpp:8:⏎
17 0:02.40 In file included from /home/magnolia/src/Nightly/browser/components/shell/nsGNOMEShellSearchProvider.h:11:⏎
18 0:02.40 In file included from /home/magnolia/src/Nightly/obj-x86_64-pc-linux-gnu/dist/include/mozilla/DBusHelpers.h:10:⏎
19 0:02.40 In file included from /home/magnolia/src/Nightly/obj-x86_64-pc-linux-gnu/dist/system_wrappers/dbus/dbus.h:3:⏎
20 0:02.40 /home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/dbus-1.0/dbus/dbus.h:29:10: fatal error: 'dbus/dbus-arch-deps.h' file not found⏎
21 0:02.40 #include <dbus/dbus-arch-deps.h>⏎
22 0:02.40 ^~~~~~~~~~~~~~~~~~~~~~~⏎

Comment 8

[Mike Hommey [:glandium]]

(In reply to Alexander from comment #7)

I use openSUSE Tumbleweed 20210805, KDE Plasma Version: 5.22.4, KDE Frameworks Version: 5.84.0, Qt Version: 5.15.2, Kernel Version: 5.13.8-3.g263a9fe-default (64-bit). I let the default prefix /usr/local unchanged in the firefox buildsystem to not mess up the System standard install path. In openSuse is the prefix /usr default. Not /usr/local like in many other Linux Distros.
Here the output of the command:
/usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'

Did you pass PKG_CONFIG_DEBUG_SPEW=1 like in comment 6? Because the output should be much more verbose.

Comment 9

[Mike Hommey [:glandium]]

(In reply to Mike Hommey [:glandium] from comment #8)

(In reply to Alexander from comment #7)

I use openSUSE Tumbleweed 20210805, KDE Plasma Version: 5.22.4, KDE Frameworks Version: 5.84.0, Qt Version: 5.15.2, Kernel Version: 5.13.8-3.g263a9fe-default (64-bit). I let the default prefix /usr/local unchanged in the firefox buildsystem to not mess up the System standard install path. In openSuse is the prefix /usr default. Not /usr/local like in many other Linux Distros.
Here the output of the command:
/usr/bin/pkg-config --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'

Did you pass PKG_CONFIG_DEBUG_SPEW=1 like in comment 6? Because the output should be much more verbose.

Looks like pkg-config on tumbleweed doesn't output more than that... which is unfortunate because I also can't reproduce that output. I'm getting:

-I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/gtk-3.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/pango-1.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/glib-2.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/cairo -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/pixman-1 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/freetype2 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/libpng12 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/gdk-pixbuf-2.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/gio-unix-2.0/ -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/harfbuzz -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/atk-1.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/at-spi2-atk/2.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/at-spi-2.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/dbus-1.0 -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/dbus-1.0/include -pthread -I/home/glandium/.mozbuild/sysroot-x86_64-linux-gnu/usr/include/libdrm

which is what it's supposed to be. In your case, .../usr/lib/x86_64-linux-gnu ends up being .../usr/lib64 which is why the build fails, but I don't see why this would be happening.

Comment 10

[Mike Hommey [:glandium]]

Ok, pkg-config on tumbleweed is actually pkgconf, which doesn't do anything useful with PKG_CONFIG_DEBUG_SPEW=1. Can you try again with:

PKG_CONFIG_SYSROOT_DIR=$HOME/.mozbuild/sysroot-x86_64-linux-gnu PKG_CONFIG_LIBDIR=$HOME/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/pkgconfig:$HOME/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/pkgconfig:$HOME/.mozbuild/sysroot-x86_64-linux-gnu/usr/share/pkgconfig /usr/bin/pkg-config --debug --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'

Also, can you also paste the output from pkg-config --version?

Comment 11

[Alexander]

Here the requested info:
pkg-config --version⏎9 1.7.4⏎
echo $PKG_CONFIG_SYSROOT_DIR⏎
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu⏎
echo $PKG_CONFIG_LIBDIR⏎
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/pkgconfig:/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/pkgconfig:/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/share/pkgconfig⏎

magnolia@localhost:~> /usr/bin/pkg-config --debug --cflags 'gtk+-wayland-3.0 >= 3.14 xkbcommon >= 0.4.1 libdrm >= 2.4'⏎
Output to long as comment. I attach the pkg-config logfile and the new config.log.

Comment 12

[Alexander]

Attachment: pkg-config-verbose.log

Comment 13

[Alexander]

Attachment: config.log-build_error

Comment 14

[Mike Hommey [:glandium]]

Is PKG_CONFIG_PATH set in your environment?

Comment 15

[Alexander]

This is what I have in PKG_CONFIG_PATH:

magnolia@localhost:~/src/Nightly> echo $PKG_CONFIG_PATH
/usr/lib64/pkgconfig:/usr/share/pkgconfig:/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/pkgconfig:/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/pkgconfig:/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/share/pkgconfig

$PKG_CONFIG=/usr/bin/pkg-config

Comment 16

[Alexander]

My dirs looks like this:
magnolia@localhost:/> tree -d -L 2
/
|------/etc/pkgconfig
├── home
│   └── magnolia/.mozbuild
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── sbin -> usr/sbin
└── usr
├── include
├── lib
├── lib64/pkgconfig
├── local
├── sbin
├── share/pkgconfig
├── src
└── x86_64-suse-linux
magnolia@localhost:/usr/x86_64-suse-linux> tree -d -L 2
.
├── bin
├── include
└── lib
└── ldscripts -> ../../lib64/ldscripts

find / -type d -name pkgconfig
/etc/pkgconfig
/usr/share/pkgconfig
/usr/lib/pkgconfig
/usr/lib64/pkgconfig
/usr/local/lib64/pkgconfig
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/pkgconfig
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/share/pkgconfig
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/pkgconfig

find / -type d -name 'include'
/home/magnolia/.config/.mozbuild/node/include/node/openssl/archs/linux-x86_64/asm/crypto/include
/usr/include
/usr/local/include
/usr/x86_64-suse-linux/include
/usr/share/cmake/include
/usr/lib64/gtk-2.0/include
/usr/lib64/glib-2.0/include
/usr/lib64/dbus-1.0/include
/usr/lib64/clang/11.0.1/include
/usr/lib64/clang/12.0.1/include
/usr/lib64/gcc/x86_64-suse-linux/11/include
/home/magnolia/.cargo/git/checkouts/wasmtime-e29865abfb2cd3c1/3bbadf8/crates/c-api/include

Include in .mozbuild:
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/include
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/glib-2.0/include
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/dbus-1.0/include
/home/magnolia/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/7/include
/home/magnolia/.mozbuild/node/include/node/openssl/archs/linux-x86_64/asm/include

Comment 17

[Mike Hommey [:glandium]]

Attachment: Bug 1724391 - Reset PKG_CONFIG_PATH.

If it has a value set, that interferes with values we set to other
PKG_CONFIG_* variables for sysroots.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6309c533e51c
Reset PKG_CONFIG_PATH. r=firefox-build-system-reviewers,mhentges

Comment 19

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/6309c533e51c