Closed Bug 1044458 Opened 9 years ago Closed 9 years ago

Firefox mozilla-central linux, linux64, and android2.2 l10n nightly builds are busted

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

Product:
Infrastructure & Operations Graveyard
Component:
CIDuty
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jlund, Assigned: glandium)

Details

Attachments

(1 file)

Work around l10n jobs cherry-picking parts of the build system
5.76 KB, patch
mshal
: review+
Details | Diff | Splinter Review 
windows looks fine and I have not seen the issue with osx.

I thought this was massimo's recent changes for desktop mozharness repacks: http://hg.mozilla.org/build/buildbot-configs/rev/4dd562b32ad0

but it looks like we are hitting this issue with buildbot legacy (MozillaRepackFactory) and mobile_l10n.py builds too.

I'm having trouble finding them on tbpl even with showall=1 so sorry for the lack of public logs.

build examples that failed:
- https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-central/build/47598999
- https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-central/build/47599706
- https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-central/build/47600642

buildbot-master logs:

http://buildbot-master91.srv.releng.usw2.mozilla.com:8001/builders/Firefox%20mozilla-central%20linux%20l10n%20nightly/builds/1759

http://buildbot-master73.srv.releng.usw2.mozilla.com:8001/builders/Android%202.2%20mozilla-central%20l10n%20nightly-4/builds/10



log snippets:

Android 2.2 mozilla-central l10n nightly-4

04:24:18     INFO - Running command: ['mock_mozilla', '-r', 'mozilla-centos6-x86_64', '-q', '--cwd', '/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/config', '--unpriv', '--shell', '/usr/bin/env MOZ_AUTOMATION=1 "LESSOPEN=|/usr/bin/lesspipe.sh %s" TMOUT=86400 CVS_RSH=ssh LOGNAME=cltbld USER=cltbld MOZ_OBJDIR=obj-l10n PATH=/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/ SYMBOL_SERVER_USER=ffxbld DISPLAY=:2 CCACHE_UMASK=002 LANG=en_US.UTF-8 CCACHE_HASHDIR= TERM=linux SHELL=/bin/bash MOZ_SIGNING_SERVERS=signing4.srv.releng.scl3.mozilla.com:9100,signing5.srv.releng.scl3.mozilla.com:9100,signing6.srv.releng.scl3.mozilla.com:9100 SHLVL=1 G_BROKEN_FILENAMES=1 HISTSIZE=1000 SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_mob/ HG_SHARE_BASE_DIR=/builds/hg-shared MOZ_CRASHREPORTER_NO_REPORT=1 SYMBOL_SERVER_HOST=symbolpush.mozilla.org EN_US_BINARY_URL=http://stage.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android/en-US SHIP_LICENSED_FONTS=1 TINDERBOX_OUTPUT=1 "MOZ_SIGN_CMD=python /builds/slave/m-cen-and-l10n_4-0000000000000/build/tools/release/signing/signtool.py --cachedir /builds/slave/m-cen-and-l10n_4-0000000000000/build/signing_cache -t /builds/slave/m-cen-and-l10n_4-0000000000000/token -n /builds/slave/m-cen-and-l10n_4-0000000000000/nonce -c /builds/slave/m-cen-and-l10n_4-0000000000000/build/tools/release/signing/host.cert -f jar -H signing4.srv.releng.scl3.mozilla.com:9100 -H signing5.srv.releng.scl3.mozilla.com:9100 -H signing6.srv.releng.scl3.mozilla.com:9100" LC_ALL=C TOOLTOOL_HOME=/builds _=/tools/buildbot/bin/python LD_LIBRARY_PATH=/lib:/tools/gcc-4.7.2-0moz1/lib:/tools/gcc-4.7.2-0moz1/lib64 LOCALE_MERGEDIR=/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/merged/ MAIL=/var/spool/mail/cltbld MOZ_UPDATE_CHANNEL=nightly HOSTNAME=bld-linux64-spot-338.build.releng.usw2.mozilla.com SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/ffxbld_dsa HISTCONTROL=ignoredups POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py PWD=/builds/slave/m-cen-and-l10n_4-0000000000000 PROPERTIES_FILE=/builds/slave/m-cen-and-l10n_4-0000000000000/buildprops.json CCACHE_DIR=/builds/ccache CCACHE_COMPRESS=1 TOOLTOOL_CACHE=/builds/tooltool_cache make'] in /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/config
04:24:18     INFO - Copy/paste: mock_mozilla -r mozilla-centos6-x86_64 -q --cwd /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/config --unpriv --shell "/usr/bin/env MOZ_AUTOMATION=1 \"LESSOPEN=|/usr/bin/lesspipe.sh %s\" TMOUT=86400 CVS_RSH=ssh LOGNAME=cltbld USER=cltbld MOZ_OBJDIR=obj-l10n PATH=/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/ SYMBOL_SERVER_USER=ffxbld DISPLAY=:2 CCACHE_UMASK=002 LANG=en_US.UTF-8 CCACHE_HASHDIR= TERM=linux SHELL=/bin/bash MOZ_SIGNING_SERVERS=signing4.srv.releng.scl3.mozilla.com:9100,signing5.srv.releng.scl3.mozilla.com:9100,signing6.srv.releng.scl3.mozilla.com:9100 SHLVL=1 G_BROKEN_FILENAMES=1 HISTSIZE=1000 SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_mob/ HG_SHARE_BASE_DIR=/builds/hg-shared MOZ_CRASHREPORTER_NO_REPORT=1 SYMBOL_SERVER_HOST=symbolpush.mozilla.org EN_US_BINARY_URL=http://stage.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android/en-US SHIP_LICENSED_FONTS=1 TINDERBOX_OUTPUT=1 \"MOZ_SIGN_CMD=python /builds/slave/m-cen-and-l10n_4-0000000000000/build/tools/release/signing/signtool.py --cachedir /builds/slave/m-cen-and-l10n_4-0000000000000/build/signing_cache -t /builds/slave/m-cen-and-l10n_4-0000000000000/token -n /builds/slave/m-cen-and-l10n_4-0000000000000/nonce -c /builds/slave/m-cen-and-l10n_4-0000000000000/build/tools/release/signing/host.cert -f jar -H signing4.srv.releng.scl3.mozilla.com:9100 -H signing5.srv.releng.scl3.mozilla.com:9100 -H signing6.srv.releng.scl3.mozilla.com:9100\" LC_ALL=C TOOLTOOL_HOME=/builds _=/tools/buildbot/bin/python LD_LIBRARY_PATH=/lib:/tools/gcc-4.7.2-0moz1/lib:/tools/gcc-4.7.2-0moz1/lib64 LOCALE_MERGEDIR=/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/merged/ MAIL=/var/spool/mail/cltbld MOZ_UPDATE_CHANNEL=nightly HOSTNAME=bld-linux64-spot-338.build.releng.usw2.mozilla.com SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/ffxbld_dsa HISTCONTROL=ignoredups POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py PWD=/builds/slave/m-cen-and-l10n_4-0000000000000 PROPERTIES_FILE=/builds/slave/m-cen-and-l10n_4-0000000000000/buildprops.json CCACHE_DIR=/builds/ccache CCACHE_COMPRESS=1 TOOLTOOL_CACHE=/builds/tooltool_cache make"
04:24:18     INFO -  make export
04:24:18     INFO -  make[1]: Entering directory `/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/config'
04:24:18     INFO -  /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/_virtualenv/bin/python /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/config/nsinstall.py -t -m 644 '../mozilla-config.h' '../dist/include'
04:24:18     INFO -  /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/_virtualenv/bin/python /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/toolkit/xre/make-platformini.py --print-buildid > buildid
04:24:19     INFO -  mkdir -p 'system_wrappers/'
04:24:19     INFO -  /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/_virtualenv/bin/python -m mozbuild.action.preprocessor -DNO_NSPR_10_SUPPORT -DANDROID='1' -DCROSS_COMPILE='1' -DMOZ_THUMB2='1' -DHAVE_ARM_SIMD='1' -DHAVE_ARM_NEON='1' -DBUILD_ARM_NEON='1' -DMOZ_ENABLE_PROFILER_SPS='1' -DMOZILLA_VERSION='"34.0a1"' -DMOZILLA_VERSION_U='34.0a1' -DMOZILLA_UAVERSION='"34.0"' -DNO_PW_GECOS='1' -DXP_LINUX='1' -DMOZ_LINKER='1' -DD_INO='d_ino' -DMOZ_DEBUG_SYMBOLS='1' -DSTDC_HEADERS='1' -DHAVE_SSIZE_T='1' -DHAVE_ST_BLKSIZE='1' -DHAVE_SIGINFO_T='1' -DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE='1' -DHAVE_VISIBILITY_ATTRIBUTE='1' -DHAVE_DIRENT_H='1' -DHAVE_GETOPT_H='1' -DHAVE_MEMORY_H='1' -DHAVE_UNISTD_H='1' -DHAVE_MALLOC_H='1' -DHAVE_SYS_SYSMACROS_H='1' -DHAVE_LINUX_QUOTA_H='1' -DHAVE_LINUX_RTNETLINK_H='1' -DHAVE_SYS_TYPES_H='1' -DHAVE_NETINET_IN_H='1' -DHAVE_BYTESWAP_H='1' -DHAVE_SYS_CDEFS_H='1' -DHAVE_DLOPEN='1' -DHAVE_DLADDR='1' -DHAVE_MEMMEM='1' -DNO_X11='1' -DHAVE_LOCALTIME_R='1' -DHAVE_ARC4RANDOM='1' -DHAVE_ARC4RANDOM_BUF='1' -DHAVE_CLOCK_MONOTONIC='1' -DVA_COPY='va_copy' -DHAVE_VA_COPY='1' -DMALLOC_H='<malloc.h>' -DHAVE_STRNDUP='1' -DHAVE_MEMALIGN='1' -DHAVE_VALLOC='1' -DHAVE_I18N_LC_MESSAGES='1' -DTARGET_XPCOM_ABI='"arm-eabi-gcc3"' -DNS_ATTR_MALLOC='__attribute__((malloc))' -DNS_WARN_UNUSED_RESULT='__attribute__((warn_unused_result))' -DNIGHTLY_BUILD='1' -DMOZ_UPDATE_CHANNEL='nightly' -DEARLY_BETA_OR_EARLIER='1' -DMOZ_BUILD_APP='mobile/android' -DANDROID_TARGET_SDK='17' -DMOZ_WIDGET_ANDROID='1' -DMOZ_PDF_PRINTING='1' -DMOZ_INSTRUMENT_EVENT_LOOP='1' -DUSE_ARM_KUSER='1' -DNS_PRINTING='1' -DNS_PRINT_PREVIEW='1' -DMOZ_DISTRIBUTION_ID='"org.mozilla"' -DMOZ_ANDROID_HISTORY='1' -DMOZ_ANDROID_OMTC='1' -DMOZ_LOCALE_SWITCHER='1' -DMOZ_ANDROID_BEAM='1' -DACCESSIBILITY='1' -DMOZ_WEBRTC='1' -DMOZ_WEBRTC_ASSERT_ALWAYS='1' -DMOZ_WEBRTC_SIGNALING='1' -DMOZ_PEERCONNECTION='1' -DMOZ_SCTP='1' -DMOZ_SRTP='1' -DMOZ_SAMPLE_TYPE_S16='1' -DMOZ_WEBSPEECH='1' -DMOZ_RAW='1' -DATTRIBUTE_ALIGNED_MAX='64' -DMOZ_WEBM='1' -DMOZ_EME='1' -DMOZ_ANDROID_OMX='1' -DMOZ_MEDIA_NAVIGATOR='1' -DMOZ_OMX_PLUGIN='1' -DMOZ_VPX='1' -DMOZ_VPX_ERROR_CONCEALMENT='1' -DVPX_ARM_ASM='1' -DMOZ_WAVE='1' -DMOZ_TREMOR='1' -DMOZ_OPUS='1' -DENABLE_SYSTEM_EXTENSION_DIRS='1' -DMOZ_WEBGL='1' -DMOZ_WEBGL_CONFORMANT='1' -DMOZ_GAMEPAD='1' -DMOZ_CRASHREPORTER='1' -DMOZ_CRASHREPORTER_ENABLE_PERCENT='100' -DLIBJPEG_TURBO_ARM_ASM='1' -DMOZ_USE_NATIVE_POPUP_WINDOWS='1' -DMOZ_TREE_FREETYPE='1' -DHAVE_FT_BITMAP_SIZE_Y_PPEM='1' -DHAVE_FT_GLYPHSLOT_EMBOLDEN='1' -DHAVE_FT_LOAD_SFNT_TABLE='1' -DMOZ_SIGNING='1' -DMOZ_UPDATER='1' -DMOZ_DISABLE_CRYPTOLEGACY='1' -DMOZ_FEEDS='1' -DMOZ_SAFE_BROWSING='1' -DMOZ_URL_CLASSIFIER='1' -DGL_PROVIDER_='1' -DMOZ_GFX_OPTIMIZE_MOBILE='1' -DMOZ_LOGGING='1' -DMOZ_MEMORY='1' -DMOZ_MEMORY_LINUX='1' -DMOZ_MEMORY_ANDROID='1' -DJS_CRASH_DIAGNOSTICS='1' -DJSGC_INCREMENTAL='1' -DJSGC_USE_EXACT_ROOTING='1' -DMOZ_PAY='1' -DHAVE__UNWIND_BACKTRACE='1' -DJS_DEFAULT_JITREPORT_GRANULARITY='3' -DMOZ_OMNIJAR='1' -DMOZ_USER_DIR='".mozilla"' -DMOZ_TREE_PIXMAN='1' -DHAVE_STDINT_H='1' -DHAVE_INTTYPES_H='1' -DMOZ_TREE_CAIRO='1' -DMOZ_ENABLE_SKIA='1' -DUSE_SKIA='1' -DSK_BUILD_FOR_ANDROID_NDK='1' -DUSE_SKIA_GPU='1' -DMOZ_XUL='1' -DMOZ_PROFILELOCKING='1' -DBUILD_CTYPES='1' -DMOZ_SERVICES_HEALTHREPORT='1' -DMOZ_DEVICES='1' -DMOZ_APP_COMPONENT_INCLUDE='"nsBrowserComponents.h"' -DMOZ_MACBUNDLE_ID='org.mozilla.nightly' -DMOZ_B2G_VERSION='"1.0.0"' -DMOZ_B2G_OS_NAME='""' -DMOZ_APP_UA_NAME='"Firefox"' -DMOZ_APP_UA_VERSION='"34.0a1"' -DFIREFOX_VERSION='34.0a1' -DMOZ_TELEMETRY_DISPLAY_REV='2' -DMOZ_DATA_REPORTING='1' -DMOZ_DLL_SUFFIX='".so"' -DXP_UNIX='1' -DA11Y_LOG='1' -DMOZ_STATIC_JS='1' \
04:24:19     INFO -  	-DMOZ_TREE_CAIRO=1 \
04:24:19     INFO -  	-DMOZ_TREE_PIXMAN=1 \
04:24:19     INFO -  	-DMOZ_NATIVE_HUNSPELL= \
04:24:19     INFO -  	-DMOZ_NATIVE_BZ2= \
04:24:19     INFO -  	-DMOZ_NATIVE_ZLIB=1 \
04:24:19     INFO -  	-DMOZ_NATIVE_PNG= \
04:24:19     INFO -  	-DMOZ_NATIVE_JPEG= \
04:24:19     INFO -  	-DMOZ_NATIVE_LIBEVENT= \
04:24:19     INFO -  	-DMOZ_NATIVE_LIBVPX= \
04:24:19     INFO -  	-DMOZ_NATIVE_ICU= \
04:24:19     INFO -  	/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/config/system-headers | /usr/bin/perl /builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/nsprpub/config/make-system-wrappers.pl system_wrappers
04:24:19     INFO -  ../dist/bin/nsinstall -R system_wrappers ../dist
04:24:19     INFO -  make[1]: ../dist/bin/nsinstall: Command not found
04:24:19    ERROR -  make[1]: *** [export] Error 127
04:24:19     INFO -  make[1]: Leaving directory `/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/config'
04:24:19     INFO -  make: *** [default] Error 2
04:24:19    ERROR - Return code: 2
04:24:19    ERROR - 2 not in success codes: [0]
04:24:19    ERROR - failures found while parsing output
04:24:19    FATAL - Halting on failure while running ['mock_mozilla', '-r', 'mozilla-centos6-x86_64', '-q', '--cwd', '/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/config', '--unpriv', '--shell', '/usr/bin/env MOZ_AUTOMATION=1 "LESSOPEN=|/usr/bin/lesspipe.sh %s" TMOUT=86400 CVS_RSH=ssh LOGNAME=cltbld USER=cltbld MOZ_OBJDIR=obj-l10n PATH=/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/ SYMBOL_SERVER_USER=ffxbld DISPLAY=:2 CCACHE_UMASK=002 LANG=en_US.UTF-8 CCACHE_HASHDIR= TERM=linux SHELL=/bin/bash MOZ_SIGNING_SERVERS=signing4.srv.releng.scl3.mozilla.com:9100,signing5.srv.releng.scl3.mozilla.com:9100,signing6.srv.releng.scl3.mozilla.com:9100 SHLVL=1 G_BROKEN_FILENAMES=1 HISTSIZE=1000 SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_mob/ HG_SHARE_BASE_DIR=/builds/hg-shared MOZ_CRASHREPORTER_NO_REPORT=1 SYMBOL_SERVER_HOST=symbolpush.mozilla.org EN_US_BINARY_URL=http://stage.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android/en-US SHIP_LICENSED_FONTS=1 TINDERBOX_OUTPUT=1 "MOZ_SIGN_CMD=python /builds/slave/m-cen-and-l10n_4-0000000000000/build/tools/release/signing/signtool.py --cachedir /builds/slave/m-cen-and-l10n_4-0000000000000/build/signing_cache -t /builds/slave/m-cen-and-l10n_4-0000000000000/token -n /builds/slave/m-cen-and-l10n_4-0000000000000/nonce -c /builds/slave/m-cen-and-l10n_4-0000000000000/build/tools/release/signing/host.cert -f jar -H signing4.srv.releng.scl3.mozilla.com:9100 -H signing5.srv.releng.scl3.mozilla.com:9100 -H signing6.srv.releng.scl3.mozilla.com:9100" LC_ALL=C TOOLTOOL_HOME=/builds _=/tools/buildbot/bin/python LD_LIBRARY_PATH=/lib:/tools/gcc-4.7.2-0moz1/lib:/tools/gcc-4.7.2-0moz1/lib64 LOCALE_MERGEDIR=/builds/slave/m-cen-and-l10n_4-0000000000000/build/mozilla-central/obj-l10n/merged/ MAIL=/var/spool/mail/cltbld MOZ_UPDATE_CHANNEL=nightly HOSTNAME=bld-linux64-spot-338.build.releng.usw2.mozilla.com SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/ffxbld_dsa HISTCONTROL=ignoredups POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py PWD=/builds/slave/m-cen-and-l10n_4-0000000000000 PROPERTIES_FILE=/builds/slave/m-cen-and-l10n_4-0000000000000/buildprops.json CCACHE_DIR=/builds/ccache CCACHE_COMPRESS=1 TOOLTOOL_CACHE=/builds/tooltool_cache make']
04:24:19    FATAL - Running post_fatal callback...
04:24:19    FATAL - Exiting 2



Firefox mozilla-central linux l10n nightly

mock_mozilla -r mozilla-centos6-x86_64 --cwd /builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/config --unpriv --shell '/usr/bin/env DOWNLOAD_BASE_URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly" MOZ_AUTOMATION="1" TOOLTOOL_CACHE="/builds/tooltool_cache" MOZ_OBJDIR="obj-firefox" PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" LD_LIBRARY_PATH="/tools/gcc-4.3.3/installed/lib" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_ffx/" HG_SHARE_BASE_DIR="/builds/hg-shared" MOZ_MAKE_COMPLETE_MAR="1" SYMBOL_SERVER_HOST="symbolpush.mozilla.org" EN_US_BINARY_URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central" CCACHE_DIR="/builds/ccache" L10NBASEDIR="../../l10n" TINDERBOX_OUTPUT="1" MOZ_SIGN_CMD="python /builds/slave/m-cen-lx-l10n-ntly-00000000000/tools/release/signing/signtool.py --cachedir /builds/slave/m-cen-lx-l10n-ntly-00000000000/signing_cache -t /builds/slave/m-cen-lx-l10n-ntly-00000000000/token -n /builds/slave/m-cen-lx-l10n-ntly-00000000000/nonce -c /builds/slave/m-cen-lx-l10n-ntly-00000000000/tools/release/signing/host.cert -H signing4.srv.releng.scl3.mozilla.com:9100 -H signing5.srv.releng.scl3.mozilla.com:9100 -H signing6.srv.releng.scl3.mozilla.com:9100" LC_ALL="C" TOOLTOOL_HOME="/builds" MOZ_UPDATE_CHANNEL="nightly" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" MOZ_CRASHREPORTER_NO_REPORT="1" CCACHE_COMPRESS="1" make'
 in dir /builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/config (timeout 1200 secs)
 watching logfiles {}
 argv: ['mock_mozilla', '-r', 'mozilla-centos6-x86_64', '--cwd', '/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/config', '--unpriv', '--shell', '/usr/bin/env DOWNLOAD_BASE_URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly" MOZ_AUTOMATION="1" TOOLTOOL_CACHE="/builds/tooltool_cache" MOZ_OBJDIR="obj-firefox" PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" LD_LIBRARY_PATH="/tools/gcc-4.3.3/installed/lib" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_ffx/" HG_SHARE_BASE_DIR="/builds/hg-shared" MOZ_MAKE_COMPLETE_MAR="1" SYMBOL_SERVER_HOST="symbolpush.mozilla.org" EN_US_BINARY_URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central" CCACHE_DIR="/builds/ccache" L10NBASEDIR="../../l10n" TINDERBOX_OUTPUT="1" MOZ_SIGN_CMD="python /builds/slave/m-cen-lx-l10n-ntly-00000000000/tools/release/signing/signtool.py --cachedir /builds/slave/m-cen-lx-l10n-ntly-00000000000/signing_cache -t /builds/slave/m-cen-lx-l10n-ntly-00000000000/token -n /builds/slave/m-cen-lx-l10n-ntly-00000000000/nonce -c /builds/slave/m-cen-lx-l10n-ntly-00000000000/tools/release/signing/host.cert -H signing4.srv.releng.scl3.mozilla.com:9100 -H signing5.srv.releng.scl3.mozilla.com:9100 -H signing6.srv.releng.scl3.mozilla.com:9100" LC_ALL="C" TOOLTOOL_HOME="/builds" MOZ_UPDATE_CHANNEL="nightly" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" MOZ_CRASHREPORTER_NO_REPORT="1" CCACHE_COMPRESS="1" make']
 environment:
  CCACHE_COMPRESS=1
  CCACHE_DIR=/builds/ccache
  CCACHE_HASHDIR=
  CCACHE_UMASK=002
  CVS_RSH=ssh
  DISPLAY=:2
  DOWNLOAD_BASE_URL=http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly
  EN_US_BINARY_URL=http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central
  G_BROKEN_FILENAMES=1
  HG_SHARE_BASE_DIR=/builds/hg-shared
  HISTCONTROL=ignoredups
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=bld-linux64-spot-329.build.releng.usw2.mozilla.com
  L10NBASEDIR=../../l10n
  LANG=en_US.UTF-8
  LC_ALL=C
  LD_LIBRARY_PATH=/tools/gcc-4.3.3/installed/lib
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  MAIL=/var/spool/mail/cltbld
  MOZ_AUTOMATION=1
  MOZ_CRASHREPORTER_NO_REPORT=1
  MOZ_MAKE_COMPLETE_MAR=1
  MOZ_OBJDIR=obj-firefox
  MOZ_SIGN_CMD=python /builds/slave/m-cen-lx-l10n-ntly-00000000000/tools/release/signing/signtool.py --cachedir /builds/slave/m-cen-lx-l10n-ntly-00000000000/signing_cache -t /builds/slave/m-cen-lx-l10n-ntly-00000000000/token -n /builds/slave/m-cen-lx-l10n-ntly-00000000000/nonce -c /builds/slave/m-cen-lx-l10n-ntly-00000000000/tools/release/signing/host.cert -H signing4.srv.releng.scl3.mozilla.com:9100 -H signing5.srv.releng.scl3.mozilla.com:9100 -H signing6.srv.releng.scl3.mozilla.com:9100
  MOZ_UPDATE_CHANNEL=nightly
  PATH=/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin
  POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py
  PWD=/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/config
  SHELL=/bin/bash
  SHLVL=1
  SYMBOL_SERVER_HOST=symbolpush.mozilla.org
  SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_ffx/
  SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/ffxbld_dsa
  SYMBOL_SERVER_USER=ffxbld
  TERM=linux
  TINDERBOX_OUTPUT=1
  TMOUT=86400
  TOOLTOOL_CACHE=/builds/tooltool_cache
  TOOLTOOL_HOME=/builds
  USER=cltbld
  _=/tools/buildbot/bin/python
 using PTY: False
INFO: mock_mozilla.py version 1.0.3 starting...
State Changed: init plugins
INFO: selinux disabled
State Changed: start
State Changed: lock buildroot
State Changed: shell
make export
make[1]: Entering directory `/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/config'
/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/_virtualenv/bin/python /builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/config/nsinstall.py -t -m 644 '../mozilla-config.h' '../dist/include'
/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/_virtualenv/bin/python /builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/toolkit/xre/make-platformini.py --print-buildid > buildid
/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/_virtualenv/bin/python -m mozbuild.action.preprocessor -DMOZ_GLUE_IN_PROGRAM -DNO_NSPR_10_SUPPORT -DMOZ_ENABLE_PROFILER_SPS='1' -DMOZILLA_VERSION='"34.0a1"' -DMOZILLA_VERSION_U='34.0a1' -DMOZILLA_UAVERSION='"34.0"' -DXP_LINUX='1' -DD_INO='d_ino' -DMOZ_DEBUG_SYMBOLS='1' -DSTDC_HEADERS='1' -DHAVE_SSIZE_T='1' -DHAVE_ST_BLKSIZE='1' -DHAVE_SIGINFO_T='1' -DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE='1' -DHAVE_VISIBILITY_ATTRIBUTE='1' -DHAVE_DIRENT_H='1' -DHAVE_GETOPT_H='1' -DHAVE_SYS_BITYPES_H='1' -DHAVE_MEMORY_H='1' -DHAVE_UNISTD_H='1' -DHAVE_GNU_LIBC_VERSION_H='1' -DHAVE_NL_TYPES_H='1' -DHAVE_MALLOC_H='1' -DHAVE_X11_XKBLIB_H='1' -DHAVE_CPUID_H='1' -DHAVE_SYS_QUOTA_H='1' -DHAVE_SYS_SYSMACROS_H='1' -DHAVE_LINUX_QUOTA_H='1' -DHAVE_LINUX_IF_ADDR_H='1' -DHAVE_LINUX_RTNETLINK_H='1' -DHAVE_SYS_TYPES_H='1' -DHAVE_NETINET_IN_H='1' -DHAVE_BYTESWAP_H='1' -DHAVE_SYS_CDEFS_H='1' -DHAVE_DLOPEN='1' -DHAVE_DLADDR='1' -DHAVE_MEMMEM='1' -DFUNCPROTO='15' -D_REENTRANT='1' -DHAVE_STAT64='1' -DHAVE_LSTAT64='1' -DHAVE_TRUNCATE64='1' -DHAVE_LOCALTIME_R='1' -DHAVE_CLOCK_MONOTONIC='1' -DHAVE_RES_NINIT='1' -DHAVE_LANGINFO_CODESET='1' -DVA_COPY='va_copy' -DHAVE_VA_COPY='1' -DHAVE_THREAD_TLS_KEYWORD='1' -DMALLOC_H='<malloc.h>' -DHAVE_STRNDUP='1' -DHAVE_POSIX_MEMALIGN='1' -DHAVE_MEMALIGN='1' -DHAVE_VALLOC='1' -DHAVE_MALLOC_USABLE_SIZE='1' -DHAVE_I18N_LC_MESSAGES='1' -DHAVE_LOCALECONV='1' -DTARGET_XPCOM_ABI='"x86-gcc3"' -DNS_ATTR_MALLOC='__attribute__((malloc))' -DNS_WARN_UNUSED_RESULT='__attribute__((warn_unused_result))' -DNIGHTLY_BUILD='1' -DMOZ_UPDATE_CHANNEL='nightly' -DEARLY_BETA_OR_EARLIER='1' -DMOZ_PHOENIX='1' -DMOZ_BUILD_APP='browser' -DMOZ_X11='1' -DMOZ_WIDGET_GTK2='1' -DMOZ_WIDGET_GTK='2' -DMOZ_PDF_PRINTING='1' -DMOZ_ENABLE_XREMOTE='1' -DMOZ_INSTRUMENT_EVENT_LOOP='1' -DNS_PRINTING='1' -DNS_PRINT_PREVIEW='1' -DMOZ_DISTRIBUTION_ID='"org.mozilla"' -DMOZ_ENABLE_GIO='1' -DMOZ_ENABLE_GCONF='1' -DMOZ_ENABLE_DBUS='1' -DACCESSIBILITY='1' -DMOZ_WEBRTC='1' -DMOZ_WEBRTC_ASSERT_ALWAYS='1' -DMOZ_WEBRTC_SIGNALING='1' -DMOZ_PEERCONNECTION='1' -DMOZ_SCTP='1' -DMOZ_SRTP='1' -DMOZ_SAMPLE_TYPE_FLOAT32='1' -DMOZ_WEBSPEECH='1' -DMOZ_RAW='1' -DATTRIBUTE_ALIGNED_MAX='64' -DMOZ_WEBM='1' -DMOZ_FFMPEG='1' -DMOZ_FMP4='1' -DMOZ_EME='1' -DMOZ_MEDIA_NAVIGATOR='1' -DMOZ_VPX='1' -DMOZ_VPX_ERROR_CONCEALMENT='1' -DVPX_X86_ASM='1' -DMOZ_WAVE='1' -DMOZ_VORBIS='1' -DMOZ_OPUS='1' -DMOZ_WEBM_ENCODER='1' -DMOZ_GSTREAMER='1' -DGST_API_VERSION='"0.10"' -DENABLE_SYSTEM_EXTENSION_DIRS='1' -DMOZ_WEBGL='1' -DMOZ_WEBGL_CONFORMANT='1' -DMOZ_GAMEPAD='1' -DMOZ_CRASHREPORTER='1' -DMOZ_CRASHREPORTER_ENABLE_PERCENT='100' -DLIBJPEG_TURBO_X86_ASM='1' -DMOZ_WEBAPP_RUNTIME='1' -DMOZ_SIGNING='1' -DMOZ_UPDATER='1' -DENABLE_TESTS='1' -DGTEST_HAS_RTTI='0' -DMOZ_FEEDS='1' -DMOZ_SAFE_BROWSING='1' -DMOZ_URL_CLASSIFIER='1' -DGL_PROVIDER_GLX='1' -DMOZ_LOGGING='1' -DMOZ_MEMORY='1' -DMOZ_MEMORY_LINUX='1' -DJSGC_INCREMENTAL='1' -DJSGC_USE_EXACT_ROOTING='1' -DJSGC_GENERATIONAL='1' -DMOZ_PAY='1' -DMOZ_ACTIVITIES='1' -DHAVE___CXA_DEMANGLE='1' -DHAVE__UNWIND_BACKTRACE='1' -DJS_DEFAULT_JITREPORT_GRANULARITY='3' -DMOZ_OMNIJAR='1' -DMOZ_USER_DIR='".mozilla"' -DHAVE_FT_BITMAP_SIZE_Y_PPEM='1' -DHAVE_FT_GLYPHSLOT_EMBOLDEN='1' -DHAVE_FT_LOAD_SFNT_TABLE='1' -DHAVE_FONTCONFIG_FCFREETYPE_H='1' -DMOZ_TREE_PIXMAN='1' -DHAVE_STDINT_H='1' -DHAVE_INTTYPES_H='1' -DMOZ_TREE_CAIRO='1' -DMOZ_ENABLE_SKIA='1' -DUSE_SKIA='1' -DUSE_SKIA_GPU='1' -DMOZ_XUL='1' -DMOZ_PROFILELOCKING='1' -DENABLE_MARIONETTE='1' -DBUILD_CTYPES='1' -DMOZ_PLACES='1' -DMOZ_SOCIAL='1' -DMOZ_SERVICES_COMMON='1' -DMOZ_SERVICES_CRYPTO='1' -DMOZ_SERVICES_HEALTHREPORT='1' -DMOZ_SERVICES_METRICS='1' -DMOZ_SERVICES_SYNC='1' -DMOZ_JSDOWNLOADS='1' -DMOZ_MACBUNDLE_ID='org.mozilla.nightly' -DMOZ_B2G_VERSION='"1.0.0"' -DMOZ_B2G_OS_NAME='""' -DMOZ_APP_UA_NAME='""' -DMOZ_APP_UA_VERSION='"34.0a1"' -DFIREFOX_VERSION='34.0a1' -DMOZ_TELEMETRY_DISPLAY_REV='2' -DMOZ_DATA_REPORTING='1' -DMOZ_DLL_SUFFIX='".so"' -DHAVE_POSIX_FADVISE='1' -DHAVE_POSIX_FALLOCATE='1' -DXP_UNIX='1' -DMOZ_ACCESSIBILITY_ATK='1' -DATK_MAJOR_VERSION='1' -DATK_MINOR_VERSION='28' -DATK_REV_VERSION='0' -DA11Y_LOG='1' -DEXPOSE_INTL_API='1' -DENABLE_INTL_API='1' -DU_STATIC_IMPLEMENTATION='1' -DU_USING_ICU_NAMESPACE='0' -DMOZ_STATIC_JS='1' \
	-DMOZ_TREE_CAIRO=1 \
	-DMOZ_TREE_PIXMAN=1 \
	-DMOZ_NATIVE_HUNSPELL= \
	-DMOZ_NATIVE_BZ2= \
	-DMOZ_NATIVE_ZLIB= \
	-DMOZ_NATIVE_PNG= \
	-DMOZ_NATIVE_JPEG= \
	-DMOZ_NATIVE_LIBEVENT= \
	-DMOZ_NATIVE_LIBVPX= \
	-DMOZ_NATIVE_ICU= \
	/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/config/system-headers | /usr/bin/perl /builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/nsprpub/config/make-system-wrappers.pl system_wrappers
../dist/bin/nsinstall -R system_wrappers ../dist
make[1]: ../dist/bin/nsinstall: Command not found
make[1]: *** [export] Error 127
make[1]: Leaving directory `/builds/slave/m-cen-lx-l10n-ntly-00000000000/build/mozilla-central/obj-firefox/config'
make: *** [default] Error 2
State Changed: unlock buildroot
program finished with exit code 2
elapsedTime=0.445014
Assignee: nobody → mh+mozilla
Status: NEW → ASSIGNED
Note we should stop the cherry-picking.
Comment on attachment 8463177 [details] [diff] [review]
Work around l10n jobs cherry-picking parts of the build system

Would we be removing the 'ac_add_options --disable-stdcxx-compat' lines again if l10n didn't cherry-pick make targets? If so, please comment.
Attachment #8463177 - Flags: review?(mshal) → review+
We could make it so.
https://hg.mozilla.org/mozilla-central/rev/dd8b06f73253
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.