Closed Bug 1484073 Opened 6 years ago Closed 6 years ago

Perma-failing tier2 make[1]: *** [automation/package-tests] Error 2

Categories

(Firefox Build System :: General, defect, P5)

defect

Tracking

(firefox64 fixed)

RESOLVED FIXED
mozilla64
Tracking Status
firefox64 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: marco)

References

(Blocks 1 open bug)

Details

(Keywords: in-triage, intermittent-failure, Whiteboard: [stockwell unknown])

Attachments

(1 file)

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=194267582&repo=mozilla-central

https://queue.taskcluster.net/v1/task/bTFhecMUQqyNdFDpA5KyoA/runs/0/artifacts/public/logs/live_backing.log

[task 2018-08-16T10:31:56.084Z] 10:31:56     INFO -  package-tests> make[2]: *** [package-tests-gtest] Error 1
[task 2018-08-16T10:31:56.084Z] 10:31:56     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:31:56.084Z] 10:31:56     INFO -  package-tests> make[2]: *** Waiting for unfinished jobs....
[task 2018-08-16T10:31:56.280Z] 10:31:56     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:31:56.281Z] 10:31:56     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  raptor '/builds/worker/artifacts/target.raptor.tests.tar.gz'
[task 2018-08-16T10:31:56.281Z] 10:31:56     INFO -  package-tests> Wrote 924 files in 6987259 bytes to target.raptor.tests.tar.gz in 5.52s
[task 2018-08-16T10:31:56.281Z] 10:31:56     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:31:56.778Z] 10:31:56     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:31:56.778Z] 10:31:56     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  talos '/builds/worker/artifacts/target.talos.tests.tar.gz'
[task 2018-08-16T10:31:56.778Z] 10:31:56     INFO -  package-tests> Wrote 1371 files in 15609520 bytes to target.talos.tests.tar.gz in 6.02s
[task 2018-08-16T10:31:56.778Z] 10:31:56     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:32:06.713Z] 10:32:06     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:32:06.713Z] 10:32:06     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  cppunittest '/builds/worker/artifacts/target.cppunittest.tests.tar.gz'
[task 2018-08-16T10:32:06.713Z] 10:32:06     INFO -  package-tests> Wrote 64 files in 142439118 bytes to target.cppunittest.tests.tar.gz in 15.99s
[task 2018-08-16T10:32:06.714Z] 10:32:06     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:32:10.486Z] 10:32:10     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:32:10.486Z] 10:32:10     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  xpcshell '/builds/worker/artifacts/target.xpcshell.tests.tar.gz'
[task 2018-08-16T10:32:10.487Z] 10:32:10     INFO -  package-tests> Wrote 5868 files in 11000486 bytes to target.xpcshell.tests.tar.gz in 19.73s
[task 2018-08-16T10:32:10.487Z] 10:32:10     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:32:59.579Z] 10:32:59     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:32:59.579Z] 10:32:59     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  common '/builds/worker/artifacts/target.common.tests.tar.gz'
[task 2018-08-16T10:32:59.579Z] 10:32:59     INFO -  package-tests> Wrote 45023 files in 40610305 bytes to target.common.tests.tar.gz in 68.79s
[task 2018-08-16T10:32:59.579Z] 10:32:59     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:17.582Z] 10:33:17     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:17.582Z] 10:33:17     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  reftest '/builds/worker/artifacts/target.reftest.tests.tar.gz'
[task 2018-08-16T10:33:17.582Z] 10:33:17     INFO -  package-tests> Wrote 30477 files in 47757980 bytes to target.reftest.tests.tar.gz in 86.83s
[task 2018-08-16T10:33:17.582Z] 10:33:17     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:17.630Z] 10:33:17     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:17.630Z] 10:33:17     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  mochitest '/builds/worker/artifacts/target.mochitest.tests.tar.gz'
[task 2018-08-16T10:33:17.630Z] 10:33:17     INFO -  package-tests> Wrote 30165 files in 58348578 bytes to target.mochitest.tests.tar.gz in 86.86s
[task 2018-08-16T10:33:17.630Z] 10:33:17     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  web-platform '/builds/worker/artifacts/target.web-platform.tests.tar.gz'
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  package-tests> Wrote 66998 files in 53753804 bytes to target.web-platform.tests.tar.gz in 111.15s
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  package-tests> make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox'
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  /builds/worker/workspace/build/src/build/moz-automation.mk:89: recipe for target 'automation/package-tests' failed
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  make[1]: *** [automation/package-tests] Error 2
[task 2018-08-16T10:33:41.945Z] 10:33:41     INFO -  make[1]: *** Waiting for unfinished jobs....
[task 2018-08-16T10:35:40.657Z] 10:35:40     INFO -  package> make[5]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/browser/installer'
[task 2018-08-16T10:35:40.657Z] 10:35:40     INFO -  package> OMNIJAR_NAME=omni.ja \
[task 2018-08-16T10:35:40.658Z] 10:35:40     INFO -  package> NO_PKG_FILES="core bsdecho js js-config jscpucfg nsinstall viewer TestGtkEmbed elf-dynstr-gc mangle* maptsv* mfc* msdump* msmap* nm2tsv* nsinstall* res/samples res/throbber shlibsign* certutil* pk12util* BadCertServer* OCSPStaplingServer* SymantecSanctionsServer* GenerateOCSPResponse* chrome/chrome.rdf chrome/app-chrome.manifest chrome/overlayinfo components/compreg.dat components/xpti.dat content_unit_tests necko_unit_tests *.dSYM " \
[task 2018-08-16T10:35:40.665Z] 10:35:40     INFO -  package> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python /builds/worker/workspace/build/src/toolkit/mozapps/installer/packager.py -DPKG_LOCALE_MANIFEST=/builds/worker/workspace/build/src/obj-firefox/browser/installer/locale-manifest.in -DMOZ_APP_NAME=firefox -DPREF_DIR=defaults/preferences -DMOZ_DEBUG=1 -DMOZ_GTK=1 -DMOZ_GTK3=1 -DJAREXT= -DMOZ_ENABLE_SKIA_PDF=1 -DMOZ_CHILD_PROCESS_NAME=plugin-container -DNECKO_WIFI -DDLL_PREFIX=lib -DDLL_SUFFIX=.so -DBIN_SUFFIX= -DDIR_MACOS= -DDIR_RESOURCES= -DBINPATH='bin' -DRESPATH='bin' -DLPROJ_ROOT=en -DENABLE_MARIONETTE=1 -DA11Y_LOG=1 -DACCESSIBILITY=1 -DATK_MAJOR_VERSION=2 -DATK_MINOR_VERSION=4 -DATK_REV_VERSION=0 -DBROWSER_CHROME_URL=chrome://browser/content/browser.xul -DBROWSER_CHROME_URL_QUOTED='"chrome://browser/content/browser.xul"' -DBUILD_CTYPES=1 -DCROSS_COMPILE='' -DEARLY_BETA_OR_EARLIER=1 -DENABLE_INTL_API=1 -DENABLE_SYSTEM_EXTENSION_DIRS=1 -DENABLE_TESTS=1 -DEXPOSE_INTL_API=1 -DFIREFOX_VERSION=63.0a1 -DFORCE_PR_LOG=1 -DFUNCPROTO=15 -DGDK_VERSION_MAX_ALLOWED=GDK_VERSION_3_4 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_32 -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_26 -DGL_PROVIDER_GLX=1 -DGTEST_HAS_RTTI=0 -DHAVE_64BIT_BUILD=1 -DHAVE_ALLOCA_H=1 -DHAVE_BYTESWAP_H=1 -DHAVE_CLOCK_MONOTONIC=1 -DHAVE_CPUID_H=1 -DHAVE_DIRENT_H=1 -DHAVE_DLADDR=1 -DHAVE_DLOPEN=1 -DHAVE_FONTCONFIG_FCFREETYPE_H=1 -DHAVE_FT_BITMAP_SIZE_Y_PPEM=1 -DHAVE_FT_GLYPHSLOT_EMBOLDEN=1 -DHAVE_FT_LOAD_SFNT_TABLE=1 -DHAVE_GETOPT_H=1 -DHAVE_GMTIME_R=1 -DHAVE_INTTYPES_H=1 -DHAVE_LCHOWN=1 -DHAVE_LINUX_IF_ADDR_H=1 -DHAVE_LINUX_PERF_EVENT_H=1 -DHAVE_LINUX_QUOTA_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -DHAVE_LOCALECONV=1 -DHAVE_LOCALTIME_R=1 -DHAVE_LSTAT64=1 -DHAVE_MALLINFO=1 -DHAVE_MALLOC_H=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MEMALIGN=1 -DHAVE_MEMMEM=1 -DHAVE_NETINET_IN_H=1 -DHAVE_NL_TYPES_H=1 -DHAVE_POSIX_FADVISE=1 -DHAVE_POSIX_FALLOCATE=1 -DHAVE_POSIX_MEMALIGN=1 -DHAVE_PTHREAD_H=1 -DHAVE_RES_NINIT=1 -DHAVE_SETPRIORITY=1 -DHAVE_STAT64=1 -DHAVE_STDINT_H=1 -DHAVE_STRERROR=1 -DHAVE_STRNDUP=1 -DHAVE_SYSCALL=1 -DHAVE_SYS_QUEUE_H=1 -DHAVE_SYS_QUOTA_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_THREAD_TLS_KEYWORD=1 -DHAVE_TRUNCATE64=1 -DHAVE_UNISTD_H=1 -DHAVE_VALLOC=1 -DHAVE_VA_COPY=1 -DHAVE_VA_LIST_AS_ARRAY=1 -DHAVE_VISIBILITY_ATTRIBUTE=1 -DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE=1 -DHAVE__UNWIND_BACKTRACE=1 -DHAVE___CXA_DEMANGLE=1 -DJS_64BIT=1 -DJS_BUILD_BINAST=1 -DJS_CODEGEN_X64=1 -DJS_DEFAULT_JITREPORT_GRANULARITY=3 -DJS_GC_ZEAL=1 -DJS_JITSPEW=1 -DJS_PUNBOX64=1 -DJS_TRACE_LOGGING=1 -DMALLOC_H='<malloc.h>' -DMALLOC_USABLE_SIZE_CONST_PTR='' -DMOZILLA_OFFICIAL=1 -DMOZILLA_UAVERSION='"63.0"' -DMOZILLA_VERSION='"63.0a1"' -DMOZILLA_VERSION_U=63.0a1 -DMOZ_ACCESSIBILITY_ATK=1 -DMOZ_ALLOW_LEGACY_EXTENSIONS=1 -DMOZ_APP_UA_NAME='""' -DMOZ_APP_UA_VERSION='"63.0a1"' -DMOZ_AV1=1 -DMOZ_BUILD_APP=browser -DMOZ_BUILD_WEBRENDER=1 -DMOZ_BUNDLED_FONTS=1 -DMOZ_CODE_COVERAGE=1 -DMOZ_CRASHREPORTER=1 -DMOZ_DATA_REPORTING=1 -DMOZ_DEBUG=1 -DMOZ_DEBUG_RUST=1 -DMOZ_DEMANGLE_SYMBOLS=1 -DMOZ_DISTRIBUTION_ID='"org.mozilla"' -DMOZ_DLL_SUFFIX='".so"' -DMOZ_DUMP_PAINTING=1 -DMOZ_ENABLE_DBUS=1 -DMOZ_ENABLE_GCONF=1 -DMOZ_ENABLE_SIGNMAR=1 -DMOZ_ENABLE_SKIA=1 -DMOZ_ENABLE_SKIA_PDF=1 -DMOZ_ENABLE_SKIA_PDF_SFNTLY=1 -DMOZ_ENABLE_XREMOTE=1 -DMOZ_FEEDS=1 -DMOZ_FFMPEG=1 -DMOZ_FFVPX=1 -DMOZ_FMP4=1 -DMOZ_GECKO_PROFILER=1 -DMOZ_GLUE_IN_PROGRAM=1 -DMOZ_INSTRUMENT_EVENT_LOOP=1 -DMOZ_LIBAV_FFT=1 -DMOZ_LOGGING=1 -DMOZ_MACBUNDLE_ID=org.mozilla.nightlydebug -DMOZ_MEMORY=1 -DMOZ_PEERCONNECTION=1 -DMOZ_PHOENIX=1 -DMOZ_PLACES=1 -DMOZ_PULSEAUDIO=1 -DMOZ_RAW=1 -DMOZ_REFLOW_PERF=1 -DMOZ_REFLOW_PERF_DSP=1 -DMOZ_REPLACE_MALLOC=1 -DMOZ_RUST_SIMD=1 -DMOZ_SAMPLE_TYPE_FLOAT32=1 -DMOZ_SCTP=1 -DMOZ_SERVICES_HEALTHREPORT=1 -DMOZ_SRTP=1 -DMOZ_STATIC_JS=1 -DMOZ_TELEMETRY_ON_BY_DEFAULT=1 -DMOZ_TELEMETRY_REPORTING=1 -DMOZ_TREE_CAIRO=1 -DMOZ_TREE_PIXMAN=1 -DMOZ_UPDATER=1 -DMOZ_UPDATE_CHANNEL=default -DMOZ_USER_DIR='".mozilla"' -DMOZ_VERIFY_MAR_SIGNATURE=1 -DMOZ_VORBIS=1 -DMOZ_VPX_NO_MEM_REPORTING=1 -DMOZ_WEBM_ENCODER=1 -DMOZ_WEBRTC=1 -DMOZ_WEBRTC_ASSERT_ALWAYS=1 -DMOZ_WEBRTC_SIGNALING=1 -DMOZ_WEBSPEECH=1 -DMOZ_WEBSPEECH_TEST_BACKEND=1 -DMOZ_WIDGET_GTK=3 -DMOZ_X11=1 -DMOZ_XUL=1 -DNIGHTLY_BUILD=1 -DNO_NSPR_10_SUPPORT=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DSTATIC_JS_API=1 -DSTDC_HEADERS=1 -DTARGET_XPCOM_ABI='"x86_64-gcc3"' -DUSE_SKIA=1 -DUSE_SKIA_GPU=1 -DU_STATIC_IMPLEMENTATION=1 -DU_USING_ICU_NAMESPACE=0 -DVA_COPY=va_copy -DVPX_X86_ASM=1 -DXP_LINUX=1 -DXP_UNIX=1 -D_REENTRANT=1 -DAB_CD=en-US \
[task 2018-08-16T10:35:40.665Z] 10:35:40     INFO -  package> 	--format omni \
[task 2018-08-16T10:35:40.665Z] 10:35:40     INFO -  package> 	--removals /builds/worker/workspace/build/src/browser/installer/removed-files.in \
[task 2018-08-16T10:35:40.665Z] 10:35:40     INFO -  package> 	 \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> 	--minify \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> 	 \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> 	 \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> 	--optimizejars \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> 	--compress none \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> 	/builds/worker/workspace/build/src/browser/installer/package-manifest.in '../../dist' '../../dist'/firefox \
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package>
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> Executing /builds/worker/workspace/build/src/obj-firefox/dist/bin/shlibsign -v -o ../../dist/firefox/libsoftokn3.chk -i ../../dist/firefox/libsoftokn3.so
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> Library File: ../../dist/firefox/libsoftokn3.so 1473936 bytes
[task 2018-08-16T10:35:40.666Z] 10:35:40     INFO -  package> Check File: ../../dist/firefox/libsoftokn3.chk
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>   hash: 32 bytes
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     ee cf 8f c5 2f 78 e8 91 01 c8
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     20 1c 81 b0 d8 dd 4b d2 59 06
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     73 06 70 d9 cf ef af fc 56 2f
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     da c8
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>   signature: 64 bytes
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     74 b9 a7 a4 1b 20 16 24 64 e7
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     ca 11 b4 ff c5 aa 13 9e 26 07
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     ae 57 2e 63 5d a4 da 7f d2 30
[task 2018-08-16T10:35:40.667Z] 10:35:40     INFO -  package>     fa 92 22 26 37 74 2f aa 33 51
[task 2018-08-16T10:35:40.668Z] 10:35:40     INFO -  package>     68 ab 3a 52 cb 90 60 4b df 65
[task 2018-08-16T10:35:40.668Z] 10:35:40     INFO -  package>     63 dd 0c 19 01 b2 4e 93 db 08
[task 2018-08-16T10:35:40.668Z] 10:35:40     INFO -  package>     5d c6 75 0b
[task 2018-08-16T10:35:40.668Z] 10:35:40     INFO -  package> moduleSpec configdir='' certPrefix='' keyPrefix='' secmod='' flags=noCertDB, noModDB

Karl can this be related to Bug 1470914?
Summary: Intermittent make[1]: *** [automation/package-tests] Error 2 → Perma-failing tier2 make[1]: *** [automation/package-tests] Error 2
Kai, sorry for the above typo, can you take a look ?
Flags: needinfo?(kaie)
On first glance I don't see a connection to NSS.
The first indication of failure seems to be above the quoted lines.
From the raw log:

     INFO -  package-tests> make[2]: Entering directory '/builds/worker/workspace/build/src/obj-firefox'
     INFO -  package-tests> /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.test_archive  gtest '/builds/worker/artifacts/target.gtest.tests.zip'
     INFO -  package-tests> Traceback (most recent call last):
     INFO -  package-tests>   File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
     INFO -  package-tests>     "__main__", fname, loader, pkg_name)
     INFO -  package-tests>   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
     INFO -  package-tests>     exec code in run_globals
     INFO -  package-tests>   File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/test_archive.py", line 734, in <module>
     INFO -  package-tests>     sys.exit(main(sys.argv[1:]))
     INFO -  package-tests>   File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/test_archive.py", line 721, in main
     INFO -  package-tests>     skip_duplicates=True)
     INFO -  package-tests>   File "/builds/worker/workspace/build/src/python/mozbuild/mozpack/mozjar.py", line 626, in add
     INFO -  package-tests>     deflater.write(data)
     INFO -  package-tests>   File "/builds/worker/workspace/build/src/python/mozbuild/mozpack/mozjar.py", line 717, in write
     INFO -  package-tests>     self._deflated.write(self._deflater.compress(data))
     INFO -  package-tests> OverflowError: size does not fit in an int
     INFO -  package-tests> /builds/worker/workspace/build/src/testing/testsuite-targets.mk:164: recipe for target 'package-tests-gtest' failed
     INFO -  package-tests> make[2]: *** [package-tests-gtest] Error 1

Apparently the cause is:
  OverflowError: size does not fit in an int

Lines 717 and 626 were last touched by glandium in 2013.
Maybe a recent size increase causes an overflow?

I suggest to ask someone with experience on the mozbuild/mozpack python code to investigate what's going on.
Flags: needinfo?(kaie)
(In reply to Kai Engert (:kaie:) from comment #2)
> Apparently the cause is:
>   OverflowError: size does not fit in an int
> 
> Lines 717 and 626 were last touched by glandium in 2013.

FYI, bug 780561.
https://hg.mozilla.org/mozilla-central/rev/1bbce6c7e006
The size of libxul.so in target.gtest.tests.zip was 2146269280. That is exactly 1214367 less than 2^31-1, which is the maximum size the zip writer can store. So, somewhere between the previous merge and that merge made the size of libxul tip over the limit.

The file contains all debug info, and is 293M once stripped. Do we really need to have the full debug info on this file?
Flags: needinfo?(rforbes)
Flags: needinfo?(mcastelluccio)
The actual limit of the zip writer code is 2^32 - 1, but something else must be bailing out before that... maybe zlib. Part of the problem being that the code doesn't try to stream anything. For a quick fix, it's probably easier to figure out how to reduce the size of this file.
It's worth noting that it's green now. Something might have made libxul.so slightly smaller in the meanwhile. But it's bound to happen again some time.
Keywords: in-triage
I am going to leave this for :marco to answer. This is actually his build.
Flags: needinfo?(rforbes)
I think glandium's question in comment 4 stands--even if this is temporarily fixed, if we don't need the full unstripped gtest libxul.so we shouldn't be packaging it. We can almost certainly get away with partially stripping it without breaking anything, and that will not only help us avoid this in the future but also save time and storage cost shuffling around an enormous binary.
I guess we don't need the full debug info, how do we partially strip it?
Flags: needinfo?(mcastelluccio) → needinfo?(ted)
Blocks: 1487318
In the last 7 days there have been 35 failures, on Linux 64 ccov, mostly debug and some opt.
Whiteboard: [stockwell needswork]
I recommend backing out the patches from bug 1420996, they have permanently broken linux64 ccov builds since September 26th and we are not getting data for this platform.  Hopefully Botond has an idea of what is going on- based on this history of this bug, it is a tooling issue and we might just have crossed a line by adding more code.
Bug 1494671 is likely related.
Based on the previous comments in this bug, it seems like the issue is the size of libxul.so, which is close to some limit imposed by tooling, crossing that limit.

If bug 1420996 has increased the size of libxul.so, that may be addressed by bug 1494671.

However, I don't think either bug 1494671 or backing out bug 1420996 would be a proper solution to this issue, only a temporary one. The size of libxul.so is expected to increase over time as functionality is added to Gecko, so soon enough another bug will get us over the limit and we'll be back to the same problem.
Flags: needinfo?(botond)
Attached patch PatchSplinter Review
Assignee: nobody → mcastelluccio
Status: NEW → ASSIGNED
Flags: needinfo?(ted)
Attachment #9013569 - Flags: review?(mh+mozilla)
Attachment #9013569 - Flags: review?(mh+mozilla) → review+
Pushed by mcastelluccio@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5f6fd031d3a6
Emit less detailed debug info for Linux coverage builds. r=glandium
https://hg.mozilla.org/mozilla-central/rev/5f6fd031d3a6
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
FWIW, if you wanted to actually change how we strip the gtest libxul, what's happening now is that you have `--disable-install-strip` in your ccov mozconfigs:
https://hg.mozilla.org/integration/mozilla-inbound/file/5f6fd031d3a6/browser/config/mozconfigs/linux64/code-coverage#l7

and that sets `PKG_SKIP_STRIP=1` in the build system, which triggers the conditional here that just copies the gtest libxul:
https://dxr.mozilla.org/mozilla-central/rev/56b988a937689d5599400afa59b72c390b40abf2/testing/testsuite-targets.mk#201-220

An alternative would be to stop doing that, and instead set `STRIP_FLAGS` to something that preserves the info you need. For builds with `--enable-profiling` configure sets `STRIP_FLAGS=--strip-debug` by default:
https://dxr.mozilla.org/mozilla-central/rev/56b988a937689d5599400afa59b72c390b40abf2/old-configure.in#654
See Also: → 1504489
See Also: → 1507094
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: