1634344 - Perma build bustage make[4]: *** [libxul.so] Error 1 when Gecko 77 merges to Beta on 2020-05-04

Status: VERIFIED FIXED

(Toolkit :: Telemetry, defect, P1)

Description

[Andreea Pavel [:apavel]]

Central as Beta simulation

Treeherder link: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=0e3276a4b01bab1c31fb852dfcaa7d92fd837b13&selectedTaskRun=OGo7hUz-Suuq79sTJOnIgw-0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=300192053&repo=try&lineNumber=53482

[task 2020-04-30T11:29:47.232Z] 11:29:47 INFO - toolkit/library/build/libxul.so
[task 2020-04-30T11:29:47.232Z] 11:29:47 INFO - rm -f libxul.so
[task 2020-04-30T11:29:47.232Z] 11:29:47 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:04.852Z] 11:30:04 INFO - make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:04.855Z] 11:30:04 INFO - /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang++ -std=gnu++17 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wunused-function -Wunused-variable -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -fno-aligned-new -fsanitize=thread -fcrash-diagnostics-dir=/builds/worker/artifacts -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -gline-tables-only -fomit-frame-pointer -funwind-tables -fPIC -shared -Wl,--gc-sections -Wl,-h,libxul.so -o libxul.so /builds/worker/workspace/obj-build/toolkit/library/build/libxul_so.list -lpthread -fsanitize=thread -rdynamic -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,-z,nocopyreloc -Wl,-Bsymbolic-functions -Wl,--build-id=sha1 -fstack-protector-strong -Wl,-rpath-link,/builds/worker/workspace/obj-build/dist/bin -Wl,-rpath-link,/usr/local/lib ../../../security/nss/lib/crmf/crmf_crmf/libcrmf.a ../../../js/src/build/libjs_static.a /builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/libgkrust.a ../../../config/external/nspr/pr/libnspr4.so ../../../config/external/nspr/libc/libplc4.so ../../../config/external/nspr/ds/libplds4.so ../../../config/external/lgpllibs/liblgpllibs.so ../../../security/nss/lib/nss/nss_nss3/libnss3.so ../../../security/nss/lib/util/util_nssutil3/libnssutil3.so ../../../security/nss/lib/smime/smime_smime3/libsmime3.so ../../../config/external/sqlite/libmozsqlite3.so ../../../security/nss/lib/ssl/ssl_ssl3/libssl3.so ../../../widget/gtk/mozgtk/stub/libmozgtk_stub.so ../../../widget/gtk/mozwayland/libmozwayland.so -Wl,--version-script,symverscript -lrt -lm -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lpthread -ldl -lc -lfreetype -lfontconfig -ldbus-glib-1 -ldbus-1 -lgobject-2.0 -lglib-2.0 -latk-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgio-2.0 -lxcb-shm -lpangoft2-1.0 -lXt -lgthread-2.0 -Wl,--version-script,/builds/worker/checkouts/gecko/build/unix/stdc++compat/hide_std.ld
[task 2020-04-30T11:30:04.855Z] 11:30:04 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/library/gtest'
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang++ -std=gnu++17 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wunused-function -Wunused-variable -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-unknown-warning-option -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -fno-aligned-new -fsanitize=thread -fcrash-diagnostics-dir=/builds/worker/artifacts -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -gline-tables-only -fomit-frame-pointer -funwind-tables -fPIC -shared -Wl,--gc-sections -Wl,-h,libxul.so -o libxul.so /builds/worker/workspace/obj-build/toolkit/library/gtest/libxul_so.list -lpthread -fsanitize=thread -rdynamic -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,-z,nocopyreloc -Wl,-Bsymbolic-functions -Wl,--build-id=sha1 -fstack-protector-strong -Wl,-rpath-link,/builds/worker/workspace/obj-build/dist/bin -Wl,-rpath-link,/usr/local/lib ../../../security/nss/lib/crmf/crmf_crmf/libcrmf.a ../../../js/src/build/libjs_static.a /builds/worker/workspace/obj-build/x86_64-unknown-linux-gnu/release/libgkrust_gtest.a ../../../config/external/nspr/pr/libnspr4.so ../../../config/external/nspr/libc/libplc4.so ../../../config/external/nspr/ds/libplds4.so ../../../config/external/lgpllibs/liblgpllibs.so ../../../security/nss/lib/nss/nss_nss3/libnss3.so ../../../security/nss/lib/util/util_nssutil3/libnssutil3.so ../../../security/nss/lib/smime/smime_smime3/libsmime3.so ../../../config/external/sqlite/libmozsqlite3.so ../../../security/nss/lib/ssl/ssl_ssl3/libssl3.so ../../../widget/gtk/mozgtk/stub/libmozgtk_stub.so ../../../widget/gtk/mozwayland/libmozwayland.so -Wl,--version-script,symverscript -lrt -lm -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lpthread -ldl -lc -lfreetype -lfontconfig -ldbus-glib-1 -ldbus-1 -lgobject-2.0 -lglib-2.0 -latk-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgio-2.0 -lxcb-shm -lpangoft2-1.0 -lXt -lgthread-2.0 -Wl,--version-script,/builds/worker/checkouts/gecko/build/unix/stdc++compat/hide_std.ld
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - /builds/worker/fetches/binutils/bin/ld: ../../components/glean/gtest/Unified_cpp_glean_gtest0.o: in function FOG_FogInitDoesntCrash_Test::TestBody()': [task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - /builds/worker/checkouts/gecko/toolkit/components/glean/gtest/TestFogInit.cpp:23: undefined reference tofog_init'
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - /builds/worker/fetches/binutils/bin/ld: libxul.so: hidden symbol `fog_init' isn't defined
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - /builds/worker/fetches/binutils/bin/ld: final link failed: bad value
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
[task 2020-04-30T11:30:06.677Z] 11:30:06 INFO - /builds/worker/checkouts/gecko/config/rules.mk:608: recipe for target 'libxul.so' failed
[task 2020-04-30T11:30:06.677Z] 11:30:06 ERROR - make[4]: *** [libxul.so] Error 1
[task 2020-04-30T11:30:06.678Z] 11:30:06 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/library/gtest'
[task 2020-04-30T11:30:06.678Z] 11:30:06 INFO - /builds/worker/checkouts/gecko/config/recurse.mk:74: recipe for target 'toolkit/library/gtest/target' failed
[task 2020-04-30T11:30:06.678Z] 11:30:06 ERROR - make[3]: *** [toolkit/library/gtest/target] Error 2
[task 2020-04-30T11:30:06.678Z] 11:30:06 INFO - make[3]: *** Waiting for unfinished jobs....
[task 2020-04-30T11:30:07.119Z] 11:30:07 INFO - make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:07.119Z] 11:30:07 INFO - /builds/worker/workspace/obj-build/_virtualenvs/init_py3/bin/python -m mozbuild.action.check_binary --target libxul.so
[task 2020-04-30T11:30:07.119Z] 11:30:07 INFO - TEST-PASS | check_stdcxx | libxul.so
[task 2020-04-30T11:30:07.119Z] 11:30:07 INFO - TEST-PASS | check_libgcc | libxul.so
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - TEST-PASS | check_glibc | libxul.so
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - TEST-PASS | check_textrel | libxul.so
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - TEST-PASS | check_pt_load | libxul.so
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - chmod +x libxul.so
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - make[4]: Entering directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - ../../../config/nsinstall -R -m 644 'libxul.so' '../../../dist/bin'
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - make[4]: Leaving directory '/builds/worker/workspace/obj-build/toolkit/library/build'
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - /builds/worker/checkouts/gecko/config/recurse.mk:32: recipe for target 'compile' failed
[task 2020-04-30T11:30:07.120Z] 11:30:07 ERROR - make[2]: *** [compile] Error 2
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - /builds/worker/checkouts/gecko/config/rules.mk:394: recipe for target 'default' failed
[task 2020-04-30T11:30:07.120Z] 11:30:07 ERROR - make[1]: *** [default] Error 2
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - client.mk:125: recipe for target 'build' failed
[task 2020-04-30T11:30:07.120Z] 11:30:07 INFO - make: *** [build] Error 2
[task 2020-04-30T11:30:07.161Z] 11:30:07 INFO - 318 compiler warnings present.
[task 2020-04-30T11:30:07.301Z] 11:30:07 INFO - Notification center failed: Install notify-send (usually part of the libnotify package) to get a notification when the build finishes.
[task 2020-04-30T11:30:07.356Z] 11:30:07 ERROR - Return code: 2
[task 2020-04-30T11:30:07.356Z] 11:30:07 WARNING - setting return code to 2
[task 2020-04-30T11:30:07.356Z] 11:30:07 FATAL - 'mach build -v' did not run successfully. Please check log for errors.
[task 2020-04-30T11:30:07.356Z] 11:30:07 FATAL - Running post_fatal callback...
[task 2020-04-30T11:30:07.356Z] 11:30:07 FATAL - Exiting -1

Chris, please take a look.

Comment 1

[Jan-Erik Rediger [:janerik]]

Uh, the test is not gated on nightly-only. I should have caught that in review.

Comment 2

[Chris H-C :chutten]

Criminy, I should've caught that in dev.

Comment 3

[Chris H-C :chutten]

Attachment: Bug 1634344 - Don't run FOG gtests without Glean r?janerik!

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f14aa9bab390
Don't run FOG gtests without Glean r=janerik

Comment 5

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/f14aa9bab390

Comment 6

[Cosmin Sabou [:CosminS]]

Fix verified in today's beta-sim: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cusercancel%2Crunnable&author=opoprus%40mozilla.com&tochange=117f7bba1b4b8e0022dee8c69d5643e1e1a2a474&fromchange=0523f1a1202f25b722c54dc086c3e6e6d0b37996&searchStr=build