1542622 - >=firefox-66.0.2 fails to cross compile to aarch64 with error: [style 0.0.1] /usr/lib/gcc/aarch64-unknown-linux-gnu/8.3.0/include/g++-v8/cstddef:50:10: fatal error: 'stddef.h' file not found

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[tt_1]

Attachment: build.log.gz

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

This can be reproduced by cross-compiling firefox to aarch64.

Actual results:

the build crashes with various errors related to error: cast from pointer to smaller type 'uintptr_t' (aka 'unsigned int') loses information

Expected results:

the build should pass, as it does for firefox-60.6.1

Comment 1

[Mike Hommey [:glandium]]

Can you try current mozilla-central? We've recently setup aarch64 cross builds, and they work.

Comment 2

[tt_1]

Not sure, since my downstream is limited. So could you point out at which point the changes you mentioned are going to be merged into beta channel? Or if there's a set of commits, so that I can backport them.

Comment 3

[Mike Hommey [:glandium]]

Your specific error looks like it could be related to bug 1522614

Comment 4

[tt_1]

#1522614 seems to be about windows, nonetheless with 67.0b9 the pointer error is gone.

Now there is this new error:

16:34.20 [style 0.0.1] /usr/lib/gcc/aarch64-unknown-linux-gnu/8.3.0/include/g++-v8/cstddef:50:10: fatal error: 'stddef.h' file not found
16:34.30 [style 0.0.1] /usr/lib/gcc/aarch64-unknown-linux-gnu/8.3.0/include/g++-v8/cstddef:50:10: fatal error: 'stddef.h' file not found, err: true
16:35.29 [style 0.0.1] /usr/lib/gcc/aarch64-unknown-linux-gnu/8.3.0/include/g++-v8/cstddef:50:10: fatal error: 'stddef.h' file not found
16:35.29 [style 0.0.1] /usr/lib/gcc/aarch64-unknown-linux-gnu/8.3.0/include/g++-v8/cstddef:50:10: fatal error: 'stddef.h' file not found, err: true

the headers are present:

/usr/lib/gcc/aarch64-unknown-linux-gnu/8.3.0/include/stddef.h
/usr/lib/gcc/armv7a-unknown-linux-gnueabihf/8.3.0/include/stddef.h
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include/stddef.h

shall I open a new bug?

also still waiting for input from #1

Comment 5

[Nathan Froyd [:froydnj]]

Can you attach your $OBJDIR/config.status file? It looks like the clang bindgen is invoking is very confused about what set of things it should be looking for; it's certainly not getting invoked with --target, which is what I would expect.

I wonder if we could get bindgen to spit out what it thinks the clang search paths etc. are.

Comment 6

[tt_1]

Attachment: firefox-67.0_beta9-config.status

here's the config.status from firefox-67.0_beta9

Comment 7

[tt_1]

froydnj, do you think there's anything helpfull in that log?

Comment 8

[Nathan Froyd [:froydnj]]

It looks like clang's getting the right flags for everything. What does:

echo '#include <cstddef>' | /usr/lib64/llvm/8/bin/clang++ -x c++ -fno-sized-deallocation --target=aarch64-linux-gnu -E -o - - -v`

say? And, just for fun, what does:

echo '#include <cstddef>' | /usr/lib64/llvm/8/bin/clang++ -x c++ -fno-sized-deallocation --target=aarch64-unknown-linux-gnu -E -o - - -v`

say?

Comment 9

[tt_1]

Attachment: aarch64-linux-gnu

this is for --target=aarch64-linux-gnu

(I'm still on llvm/clang-7.1.0 though)

Comment 10

[tt_1]

Attachment: aarch64-unknown-linux-gnu

and this is for aarch64-unknown-linux-gnu

CHOST for gentoo (and fedora?) is aarch64-unknown-linux-gnu, I think aarch64-linux-gnu is a debianism?

Comment 11

[Nathan Froyd [:froydnj]]

Well, that'd be the problem then. We're passing --target=aarch64-linux-gnu to clang, which then can't find the GCC headers for aarch64-unknown-linux-gnu. I guess we need to be a little smarter somewhere along the line in what --target clang should receive, though I'm not totally sure what changed in the 67 timeframe...glandium, do you have any thoughts here?

Comment 12

[tt_1]

During configure, I explicitly set it to --target aarch64-unknown-linux-gnu, so might this be a case of --target not honored?

Comment 13

[Nathan Froyd [:froydnj]]

Sort of. We canonicalize --target:

https://searchfox.org/mozilla-central/source/build/moz.configure/init.configure#753-760

But it turns out that in some cases, the vendor part of the target triple really does matter, and we should be including it. This would be one of those cases.

You can make this line:

https://searchfox.org/mozilla-central/source/build/moz.configure/init.configure#760

be `toolchain = '%s-unknown-%s' % (cpu, os)

and that, will, I think solve your problem.

Comment 14

[tt_1]

Attachment: firefox-67.0-cross.patch

I tried to use the attached patch, but to no avail. Still the very same error about stddef.h not found from #4

Comment 15

[Mike Hommey [:glandium]]

What is BINDGEN_SYSTEM_FLAGS in config.status?

Comment 16

[Nathan Froyd [:froydnj]]

(In reply to Mike Hommey [:glandium] from comment #15)

What is BINDGEN_SYSTEM_FLAGS in config.status?

-x c++ -fno-sized-deallocation -DTRACING=1 -DIMPL_LIBXUL -DMOZILLA_INTERNAL_API -DRUST_BINDGEN -DOS_POSIX=1 -DOS_LINUX=1 --target=aarch64-linux-gnu -I/usr/aarch64-unknown-linux-gnu/usr/include/nspr -I/usr/include/nss -I/usr/aarch64-unknown-linux-gnu/tmp/portage/www-client/firefox-67.0_beta9/work/firefox-67.0/ff/dist/include/nss -I/usr/aarch64-unknown-linux-gnu/usr/include/pixman-1 -I/usr/aarch64-unknown-linux-gnu/tmp/portage/www-client/firefox-67.0_beta9/work/firefox-67.0/ff

If you're asking what they are with the patch from comment 14, I do not know; those are from the attached config.status. What about with the patch from comment 14?

Comment 17

[tt_1]

Attachment: build.log.gz

here is the full build log with the patch from #14

Comment 18

[tt_1]

Attachment: firefox-67.0-config.status

and here is the config.status file

I'm not sure wether there is any difference, but that --target is now aarch64-unknown-linux-gnu, and the patch does work as intended.

Comment 19

[Mike Hommey [:glandium]]

What happens if you try to build with clang rather than gcc?

Comment 20

[tt_1]

Using clang rather than gcc triggers another bug during configure, concerning system-libjpeg. So guess that needs to be adressed before :)

Comment 21

[tt_1]

I suspected the clang-sys crate to be the reason for this, but the code hasn't changed inbetween firefox v66 and v67

Comment 22

[tt_1]

Attachment: firefox-66.0-updated-bindgen-configure.patch

I think I found the problem, the attached patch will make firefox-66.0.5 fail for the same reason. I isolated it from a diff of the two build folders of firefox-66.0.5/build/ and firefox-67.0/build/

I'll try to revert that patch and do some runtime testing.

Comment 23

[tt_1]

I was able to reduce the patch, it's the last chunk that makes all the difference. Can you help me to find out more about the actuall problem here?

Comment 24

[tt_1]

Attachment: firefox-66.0-more-reduced-bindgen-configure.patch

Comment 25

[Mike Hommey [:glandium]]

What happens if you apply the patch from bug 1526857?

Comment 26

[tt_1]

this makes me hit #1557030 I believe:

0:22.10 checking for rustfmt... not found
0:23.38 checking for clang for bindgen... /usr/lib/llvm/7/bin/clang++
0:23.41 checking for libclang for bindgen... not found
0:23.41 ERROR: Could not find libclang to generate rust bindings for C/C++. Please install the necessary packages, run mach bootstrap, or use --with-libclang-path to give the path containing it.
0:23.45 *** Fix above errors and then restart with
0:23.45 "./mach build"
0:23.45 gmake: *** [client.mk:115: configure] Error 1

passing --with-libclang-path=/usr/lib/llvm/7/lib64

0:06.00 checking for cbindgen... /usr/bin/cbindgen
0:06.00 checking for rustfmt... not found
0:06.05 checking for clang for bindgen... /usr/lib/llvm/7/bin/clang++
0:06.05 checking for libclang for bindgen... /usr/lib/llvm/7/lib64/libclang.so
0:06.09 checking that libclang is new enough... yes
0:06.09 Traceback (most recent call last):
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/configure.py", line 132, in <module>
0:06.09 sys.exit(main(sys.argv))
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/configure.py", line 38, in main
0:06.09 sandbox.run(os.path.join(os.path.dirname(file), 'moz.configure'))
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/configure/init.py", line 481, in run
0:06.09 func(*args)
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/configure/init.py", line 525, in _value_for
0:06.09 return self._value_for_depends(obj)
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/util.py", line 947, in method_call
0:06.09 cache[args] = self.func(instance, *args)
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/configure/init.py", line 534, in _value_for_depends
0:06.09 value = obj.result()
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/util.py", line 947, in method_call
0:06.09 cache[args] = self.func(instance, *args)
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/configure/init.py", line 151, in result
0:06.09 return self._func(*resolved_args)
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/python/mozbuild/mozbuild/configure/init.py", line 1097, in wrapped
0:06.09 return new_func(*args, **kwargs)
0:06.09 File "/usr/armv7a-unknown-linux-gnueabihf/tmp/portage/www-client/firefox-68.0_beta11/work/firefox-68.0/build/moz.configure/bindgen.configure", line 278, in basic_bindgen_cflags
0:06.09 return args + (clang_flags or [])
0:06.09 TypeError: can only concatenate list (not "unicode") to list
0:06.12 *** Fix above errors and then restart with
0:06.12 "./mach build"

Comment 27

[Mike Hommey [:glandium]]

Please attach config.log after trying to build again under the same conditions as for comment 26, but with the MOZ_CONFIGURE_TRACE environment variable set to 1.

Comment 28

[tt_1]

this was solved by exporting sysroot through BINDGEN_CFLAGS