1440013 - 'too many sections' error in MinGW x64

Status: RESOLVED FIXED

(Core :: MFBT, defect)

Description

[Tom Ritter [:tjr]]

Try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3b674bc9b065d9fba44ae7c84e192ed1211837df&selectedJob=163485267

This happened before I disabled the crash reporter and after, so that's not involved.

There's this break early on:

>  /builds/worker/workspace/build/src/obj-firefox/_virtualenv/bin/python /builds/worker/workspace/build/src/config/expandlibs_exec.py --uselist --  /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/mingw32/bin/x86_64-w64-mingw32-g++ -mwindows -shared -Wl,--gc-sections -Wl,--out-implib > -Wl,libcrashinjectdll.a -o crashinjectdll.dll crashinjectdll.o ./module.res -lpthread -Wl,--build-id -static /builds/worker/workspace/build/src/build/win32/crashinjectdll/crashinjectdll.def        -luuid -lgdi32 -lwinmm -lwsock32 -luserenv -lsecur32
>  Executing: ../../../../sccache2/sccache ../../../../mingw32/bin/x86_64-w64-mingw32-g++ -mwindows -shared -Wl,--gc-sections -Wl,--out-implib -Wl,libcrashinjectdll.a -o crashinjectdll.dll /builds/worker/workspace/build/src/obj-firefox/build/win32/crashinjectdll/tmpAKUgnN.list module.res -lpthread -Wl,--build-id -static ../../../../build> /win32/crashinjectdll/crashinjectdll.def -luuid -lgdi32 -lwinmm -lwsock32 -luserenv -lsecur32
>  /builds/worker/workspace/build/src/obj-firefox/build/win32/crashinjectdll/tmpAKUgnN.list:
>      INPUT("crashinjectdll.o")
>  crashinjectdll.o: In function `__throw_bad_exception':
>  /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/throw_gcc.h:53: undefined reference to `__imp_mozalloc_abort'
>  crashinjectdll.o: In function `__throw_bad_alloc':
>  /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/throw_gcc.h:59: undefined reference to `__imp_mozalloc_abort'
>  crashinjectdll.o: In function `__throw_bad_cast':
>  /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/throw_gcc.h:65: undefined reference to `__imp_mozalloc_abort'
>  crashinjectdll.o: In function `__throw_bad_typeid':
>  /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/throw_gcc.h:71: undefined reference to `__imp_mozalloc_abort'
>  crashinjectdll.o: In function `__throw_bad_function_call':
>  /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/throw_gcc.h:78: undefined reference to `__imp_mozalloc_abort'
>  crashinjectdll.o:/builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/throw_gcc.h:84: more undefined references to `__imp_mozalloc_abort' follow
>  collect2: error: ld returned 1 exit status
>  /builds/worker/workspace/build/src/config/rules.mk:699: recipe for target 'crashinjectdll.dll' failed
>  make[4]: *** [crashinjectdll.dll] Error 1


But later on I see the following:

> /builds/worker/workspace/build/src/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/6.4.0/../../../../x86_64-w64-mingw32/bin/as: Unified_cpp_js_src13.o: too many sections (39186)
> {standard input}: Assembler messages:
> {standard input}: Fatal error: can't write 72 bytes to section .text$_Z7fprintfP6_iobufPKcz of Unified_cpp_js_src13.o because: 'File too big'
> /builds/worker/workspace/build/src/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/6.4.0/../../../../x86_64-w64-mingw32/bin/as: Unified_cpp_js_src13.o: too many sections (39186)
> {standard input}: Fatal error: can't close Unified_cpp_js_src13.o: File too big
> /builds/worker/workspace/build/src/config/rules.mk:1047: recipe for target 'Unified_cpp_js_src13.o' failed
> make[4]: *** [Unified_cpp_js_src13.o] Error 1
>
> /builds/worker/workspace/build/src/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/6.4.0/../../../../x86_64-w64-mingw32/bin/as: Unified_cpp_js_src15.o: too many sections (34086)
> {standard input}: Assembler messages:
> {standard input}: Fatal error: can't write 72 bytes to section .text$_Z7fprintfP6_iobufPKcz of Unified_cpp_js_src15.o because: 'File too big'
> /builds/worker/workspace/build/src/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/6.4.0/../../../../x86_64-w64-mingw32/bin/as: Unified_cpp_js_src15.o: too many sections (34086)
> {standard input}: Fatal error: can't close Unified_cpp_js_src15.o: File too big
> /builds/worker/workspace/build/src/config/rules.mk:1047: recipe for target 'Unified_cpp_js_src15.o' failed
> make[4]: *** [Unified_cpp_js_src15.o] Error 1

Comment 1

[(Away)]

Offhand I don't know about the mozalloc_abort thing, glandium might know.

The later error looks like a combination of -ffunction-sections, large unified files, and I'm guessing a 32k section limit? Does gcc on linux builds not have this limit? Maybe we could unify less or drop -ffunction-sections. (And this looks like it's building JS code rather than crashinject so it might be cleaner to open a separate bug.)

Comment 2

[Tom Ritter [:tjr]]

Yea, I'm going to put the mozalloc thing in another bug.

Comment 3

[Tom Ritter [:tjr]]

Try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=11bab636539db05cccbf68b5f60b61c9c6fcec8a

Comment 4

[Tom Ritter [:tjr]]

Attachment: Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

Review commit: https://reviewboard.mozilla.org/r/225878/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/225878/

Comment 5

[Mike Hommey [:glandium]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

https://reviewboard.mozilla.org/r/225878/#review233024

::: build/autoconf/compiler-opts.m4:128
(Diff revision 1)
>      if test -z "$DEVELOPER_OPTIONS"; then
>          CFLAGS="$CFLAGS -ffunction-sections -fdata-sections"
>          CXXFLAGS="$CXXFLAGS -ffunction-sections -fdata-sections"
> +
> +	# For MinGW, we need big-obj otherwise we create too many sections in Unified builds
> +        if test "$OS_ARCH" != "$HOST_OS_ARCH"; then

That really seems like the wrong test here. Specifically, this would probably break android builds on mac.

Comment 6

[Tom Ritter [:tjr]]

Try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=0c8c97fdc6c86c7599207cc9be26c54fc5326d77

Comment 7

[Tom Ritter [:tjr]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/225878/diff/1-2/

Comment 8

[Mike Hommey [:glandium]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

https://reviewboard.mozilla.org/r/225878/#review234952

::: build/autoconf/compiler-opts.m4:128
(Diff revision 2)
>      if test -z "$DEVELOPER_OPTIONS"; then
>          CFLAGS="$CFLAGS -ffunction-sections -fdata-sections"
>          CXXFLAGS="$CXXFLAGS -ffunction-sections -fdata-sections"
> +
> +        # For MinGW, we need big-obj otherwise we create too many sections in Unified builds
> +        if test "${HOST_OS_ARCH}" = "Linux" -a "${OS_ARCH}" = "WINNT"; then

It seems to me the host os doesn't matter. As a matter of fact, if you were to "natively" build with mingw (on windows), you'd still need the flag, but this doesn't set it. IOW, you only need to check the target OS (since you're already in a "compiler is gcc" branch)

Comment 9

[Tom Ritter [:tjr]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/225878/diff/2-3/

Comment 10

[Mike Hommey [:glandium]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

https://reviewboard.mozilla.org/r/225878/#review236024

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f5c601a73945
For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors r=glandium

Comment 12

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/f5c601a73945

Comment 13

[Tom Ritter [:tjr]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

[Approval Request Comment]

This is one of several MinGW Build patches I'd like to land in esr60 for Tor. It will prevent them from carrying their own patches for the lifetime of esr60 and will enable us to keep the MinGW build functioning and know if/when/how it was broken by new commits into esr60.

This commit only affects the MinGW build configuration, so it is low-risk.

Comment 14

[Julien Cristau [:jcristau]]

Comment on attachment 8956940 [details]
Bug 1440013 For MinGW build, pass -Wa,-mbig-obj to solve 'too many sections' errors

mingw build fix, approved for 60.1esr

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/b20aeb0746e7