1791275 - ../media/ffvpx/libavutil/cpu.c:46:10: fatal error: sys/sysctl.h: No such file or directory

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[tt_1]

Attachment: firefox-nightly-aarch64.log.gz

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

I downloaded latest sources from git and compiled them for aarch64

Actual results:

the build fails to compile with:

31:50.05 /usr/bin/aarch64-unknown-linux-gnu-gcc --sysroot /usr/aarch64-unknown-linux-gnu -std=gnu99 -o 8bd_ipred_prepare_tmpl.o -c -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox_build/dist/system_wrappers -include /var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 -DDAV1D_API= -DSTACK_ALIGNMENT=16 -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/media/libdav1d -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox_build/media/libdav1d -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/third_party/dav1d -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/third_party/dav1d/include -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/third_party/dav1d/include/dav1d -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/third_party/dav1d/src -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/third_party/dav1d/include/compat/gcc -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox_build/dist/include -I/usr/aarch64-unknown-linux-gnu/usr/include/nspr -I/usr/aarch64-unknown-linux-gnu/usr/include/nss -I/usr/aarch64-unknown-linux-gnu/usr/include/nspr -I/var/tmp/portage/portage/www-client/firefox-106.0/work/firefox_build/dist/include/nss -I/usr/aarch64-unknown-linux-gnu/usr/include/libpng16 -I/usr/aarch64-unknown-linux-gnu/usr/include -I/usr/aarch64-unknown-linux-gnu/usr/include -I/usr/aarch64-unknown-linux-gnu/usr/include/pixman-1 -I/usr/aarch64-unknown-linux-gnu/usr/include -include /var/tmp/portage/portage/www-client/firefox-106.0/work/firefox_build/mozilla-config.h -DMOZILLA_CLIENT -pipe -fomit-frame-pointer -fPIC -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -O2 -fno-omit-frame-pointer -funwind-tables -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wduplicated-cond -Wlogical-op -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wno-multistatement-macros -Wno-error=class-memaccess -Wformat -Wformat-security -Wformat-overflow=2 -Wno-psabi -fno-strict-aliasing -ffp-contract=off -MD -MP -MF .deps/8bd_ipred_prepare_tmpl.o.pp -fdiagnostics-color 8bd_ipred_prepare_tmpl.c
31:50.05 media/libdav1d/8bd_ipred_tmpl.o
31:50.06 /var/tmp/portage/portage/www-client/firefox-106.0/work/firefox-106.0/media/ffvpx/libavutil/cpu.c:46:10: fatal error: sys/sysctl.h: No such file or directory
31:50.07 46 | #include <sys/sysctl.h>
31:50.07 | ^~~~~~~~~~~~~~
31:50.07 compilation terminated.

Expected results:

I expect the compile to pass through smoothly

I will attach the compressed build log

Comment 1

[tt_1]

<sys/sysctl.h> is a header which was shipped with glibc before version glibc-2.32:

• The deprecated <sys/sysctl.h> header and the sysctl function have been
  removed. To support old binaries, the sysctl function continues to
  exist as a compatibility symbol (on those architectures which had it),
  but always fails with ENOSYS. This reflects the removal of the system
  call from all architectures, starting with Linux 5.5.

the include of this header is controlled via "HAVE_SYSCTL" in the file ../media/ffvpx/libavutil/cpu.c on line 46

if you grep for HAVE_SYSCTL inside of ../media/ffvpx/ , you're going to end up at various places:

config_unix64.asm:321:%define HAVE_SYSCTL 0
config_darwin64.h:315:#define HAVE_SYSCTL 1
config_win64.h:332:#define HAVE_SYSCTL 0
config_win64_aarch64.h:315:#define HAVE_SYSCTL 0
config_win32.asm:316:%define HAVE_SYSCTL 0
config_win64.asm:316:%define HAVE_SYSCTL 0
config_darwin64.asm:300:%define HAVE_SYSCTL 1
config_audio.h:315:#define HAVE_SYSCTL 0
config_android32.h:337:#define HAVE_SYSCTL 0
config_unix32.h:332:#define HAVE_SYSCTL 0
config_darwin_aarch64.h:337:#define HAVE_SYSCTL 1
config_android_x86_64.h:337:#define HAVE_SYSCTL 0
config_unix_aarch64.h:332:#define HAVE_SYSCTL 1
config_win32.h:332:#define HAVE_SYSCTL 0
config_android64.h:337:#define HAVE_SYSCTL 0
config_unix64.h:332:#define HAVE_SYSCTL 0
libavutil/cpu.c:41:#if HAVE_SYSCTL
libavutil/cpu.c:215:#elif HAVE_SYSCTL && defined(HW_NCPUONLINE)
libavutil/cpu.c:221:#elif HAVE_SYSCTL && defined(HW_NCPU)
config_generic.h:337:#define HAVE_SYSCTL 0

three unix configs report back correctly of not shipping the header, but one of them does:

config_unix_aarch64.h:332:#define HAVE_SYSCTL 1

which is why I believe this must fail, right?

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[James Lee]

I'm hitting this build failure as well.

I'm cross compiling firefox "nightly" from central for a aarch64-unknown-linux-gnu system with a sysroot based on gcc-12.2.0+binutils-2.39+glibc-2.36. I'm compiling bootstrap disabled, so all code is getting compiler locally. My sysroot does not contain /usr/include/sys/sysctl.h (and this is to be expected since it was removed from glibc in version 2.32).

This same setup was able to build firefox in the past.

I believe this build breakage to be a regression caused by https://phabricator.services.mozilla.com/D150972 (which was a fix for Bug1765480), and this issue to be basically a replay of this old bug: Bug1261414

I feel that the severity of this bug should be raised, since this is a build breaking issue. (I'm unsure why the firefox build machines haven't flagged this, maybe they have a glibc version older than 2.32?)

I'm going to see if I can get a patch into Phabricator that just defines HAVE_SYSCTL to 0 (same as the fix for Bug1261414 was).

Comment 4

[James Lee]

Attachment: Bug 1791275 - Don't build libav with sysctl on Unix AArch64 r=alwu

Comment 5

[tt_1]

yeah, my setup is essentially the same and I agree to your verdict 100%

config_unix_aarch64.h needs to #define HAVE_SYSCTL 0 instead of 1, I tested this locally and can confirm it fixes the compile

there is however a big warning in the first line of config_unix_aarch64.h , to not edit this file by hand as it is automatically configured.

This seems to indicate to me that the real fix will be somewhere else maybe? I just have no idea who created these header files in the first place - was this bug imported from upstream ffmpeg sources?

Comment 6

[James Lee]

I pushed a fix up to Phabricator and requested a review, so hopefully it can generate some momentum to get the fix merged :)

I share tt_1's concern that just changing the value might not be the "proper fix", however it definitely fixes building on my setup.

Also for reference, the accepted fix for Bug1261414 was changing the define, and there was also the "Automatically generated" warning at the top of that file ( https://hg.mozilla.org/integration/mozilla-inbound/file/081c52bae5e7c053685743f190398d73669f419d/media/libav/config_unix.h )

Comment 7

[James Lee]

Aside: I was curious, so I checked. The aarch64-linux-gnu sysroot that the Mozilla build machines use is based on Debian 10, which uses glibc-2.28. This explains why the regression wasn't detected by them, since their sysroot contains the sys/sysctl.h file.

$ tar tf ~/.mozbuild/toolchains/b88d9d3cbb82f2de-sysroot-aarch64-linux-gnu.tar.zst | grep sys/sysctl.h
sysroot-aarch64-linux-gnu/usr/include/aarch64-linux-gnu/sys/sysctl.h

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0efaf5a00aac
Don't build libav with sysctl on Unix AArch64 r=alwu,padenot

Comment 9

[tt_1]

If this sticks, can the fix please be backported to 106 beta branch?

Comment 10

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/0efaf5a00aac

Comment 11

[vcunat]

I'm wondering why this doesn't get fixed in 106 already. Sure, anyone can find the patch and apply it, but...