1235098 - Windows builds on mozilla-inbound time out

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Starting with https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=6a9d0f869621 Windows build jobs time out.

Log: https://treeherder.mozilla.org/logviewer.html#?job_id=19002246&repo=mozilla-inbound

Successful log from previous build: https://treeherder.mozilla.org/logviewer.html#?job_id=19001797&repo=mozilla-inbound

From log of failing job:
11:04:13     INFO -  c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/toolkit/mozapps/installer/informulate.py \
11:04:13     INFO -  	../../dist//firefox-46.0a1.en-US.win32.json \
11:04:13     INFO -  	BUILDID=20151224101847 \
11:04:13     INFO -  	MOZ_SOURCE_REPO=MOZ_SOURCE_REPO=https://hg.mozilla.org/integration/mozilla-inbound \
11:04:13     INFO -  	MOZ_SOURCE_STAMP=6a9d0f8696218fe9d5b7805a1d800e28166b67a0 \
11:04:13     INFO -  	MOZ_PKG_PLATFORM=win32
11:04:13     INFO -  mozmake.EXE[4]: Leaving directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
11:04:13     INFO -  mozmake.EXE[4]: Entering directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
11:04:13     INFO -  cp ../../mozinfo.json ../../dist//firefox-46.0a1.en-US.win32.mozinfo.json
11:04:13     INFO -  mozmake.EXE[4]: Leaving directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
12:24:13     INFO - Automation Error: mozprocess timed out after 4800 seconds running ['c:\\mozilla-build\\python27\\python.exe', 'mach', '--log-no-times', 'build', '-v']
12:24:13    ERROR - timed out after 4800 seconds of no output
12:24:13    ERROR - Return code: 572
12:24:13  WARNING - setting return code to 2

Successful job:
10:52:27     INFO -  mozmake.EXE[4]: Entering directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
10:52:27     INFO -  c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/toolkit/mozapps/installer/informulate.py \
10:52:27     INFO -  	../../dist//firefox-46.0a1.en-US.win32.json \
10:52:27     INFO -  	BUILDID=20151224095647 \
10:52:27     INFO -  	MOZ_SOURCE_REPO=MOZ_SOURCE_REPO=https://hg.mozilla.org/integration/mozilla-inbound \
10:52:27     INFO -  	MOZ_SOURCE_STAMP=6c49bb716593014c07d3736ea2124f74bfe3bb06 \
10:52:27     INFO -  	MOZ_PKG_PLATFORM=win32
10:52:27     INFO -  mozmake.EXE[4]: Leaving directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
10:52:27     INFO -  mozmake.EXE[4]: Entering directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
10:52:27     INFO -  cp ../../mozinfo.json ../../dist//firefox-46.0a1.en-US.win32.mozinfo.json
10:52:27     INFO -  mozmake.EXE[4]: Leaving directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
10:52:42     INFO -  mozmake.EXE[4]: Entering directory 'c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/browser/installer'
10:52:42     INFO -  OMNIJAR_NAME=omni.ja \
10:52:42     INFO -  NO_PKG_FILES="core bsdecho js js-config jscpucfg nsinstall viewer TestGtkEmbed elf-dynstr-gc mangle* maptsv* mfc* msdump* msmap* nm2tsv* nsinstall* res/samples res/throbber shlibsign* certutil* pk12util* BadCertServer* OCSPStaplingServer* GenerateOCSPResponse* chrome/chrome.rdf chrome/app-chrome.manifest chrome/overlayinfo components/compreg.dat components/xpti.dat content_unit_tests necko_unit_tests *.dSYM " \
10:52:42     INFO -  c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-in-w32-000000000000000000000/build/src/toolkit/mozapps/installer/packager.py -DMOZ_APP_NAME=firefox -DPREF_DIR=defaults/preferences -DJAREXT= -DMOZ_ANGLE_RENDERER=1 -DMOZ_D3DCOMPILER_VISTA_DLL=d3dcompiler_47.dll -DMOZ_D3DCOMPILER_XP_DLL=D3DCompiler_43.dll -DMOZ_CHILD_PROCESS_NAME=plugin-container.exe -DMOZ_PACKAGE_MSVC_DLLS=1 -DMSVC_C_RUNTIME_DLL=msvcr120.dll -DMSVC_CXX_RUNTIME_DLL=msvcp120.dll -DMOZ_SHARED_MOZGLUE=1 -DNECKO_WIFI -DHAVE_MAKENSISU=1 -DDLL_PREFIX= -DDLL_SUFFIX=.dll -DBIN_SUFFIX=.exe -DDIR_MACOS= -DDIR_RESOURCES= -DMOZ_FOLD_LIBS=1 -DBINPATH=bin -DRESPATH=bin -DLPROJ_ROOT=en -DMOZ_ICU_VERSION=56 -DMOZ_SHARED_ICU -DMOZ_ICU_DBG_SUFFIX= -DWOW_HELPER -D_CRT_SECURE_NO_WARNINGS=1 -D_CRT_NONSTDC_NO_WARNINGS=1 -D_USE_MATH_DEFINES=1 -DHAVE_SEH_EXCEPTIONS=1 -DMOZ_MSVC_STL_WRAP_RAISE=1 -DWINVER=0x502 -D_WIN32_WINNT=0x502 -D_WIN32_IE=0x0603 -DMOZ_WINSDK_TARGETVER=0x06030000 -DMOZ_WINSDK_MAXVER=0x06030000 -DHAVE_IO_H=1 -DHAVE_SETBUF=1 -DHAVE_ISATTY=1 -DX_DISPLAY_MISSING=1 -DMOZ_ENABLE_PROFILER_SPS=1 -DMOZ_PROFILING=1 -DMOZILLA_VERSION='"46.0a1"' -DMOZILLA_VERSION_U=46.0a1 -DMOZILLA_UAVERSION='"46.0"' -DHAVE_SNPRINTF=1 -D_WINDOWS=1 -DWIN32=1 -DXP_WIN=1 -DXP_WIN32=1 -DHW_THREADS=1 -DSTDC_HEADERS=1 -DWIN32_LEAN_AND_MEAN=1 -DNOMINMAX=1 -DNO_X11=1 -D_X86_=1 -DD_INO=d_ino -DMOZ_DEBUG_SYMBOLS=1 -DMALLOC_H='<malloc.h>' -DHAVE_MALLOC_H=1 -DMALLOC_USABLE_SIZE_CONST_PTR=const -DTARGET_XPCOM_ABI='"x86-msvc"' -DMALLOC_H='<malloc.h>' -DHAVE_FORCEINLINE=1 -DHAVE_LOCALECONV=1 -DNIGHTLY_BUILD=1 -DE10S_TESTING_ONLY=1 -DNS_ENABLE_TSF=1 -DMOZ_UPDATE_CHANNEL=default -DEARLY_BETA_OR_EARLIER=1 -DMOZ_PHOENIX=1 -DMOZ_BUILD_APP=browser -DMOZ_PDF_PRINTING=1 -DMOZ_INSTRUMENT_EVENT_LOOP=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DMOZ_DISTRIBUTION_ID='"org.mozilla"' -DACCESSIBILITY=1 -DMOZ_WEBRTC=1 -DMOZ_WEBRTC_ASSERT_ALWAYS=1 -DMOZ_WEBRTC_SIGNALING=1 -DMOZ_PEERCONNECTION=1 -DMOZ_SCTP=1 -DMOZ_SRTP=1 -DMOZ_SAMPLE_TYPE_FLOAT32=1 -DMOZ_WEBSPEECH=1 -DMOZ_WEBSPEECH_TEST_BACKEND=1 -DMOZ_RAW=1 -DMOZ_DIRECTSHOW=1 -DMOZ_WMF=1 -DMOZ_FMP4=1 -DMOZ_EME=1 -DMOZ_MEDIA_NAVIGATOR=1 -DMOZ_VPX_ERROR_CONCEALMENT=1 -DVPX_X86_ASM=1 -DMOZ_VPX_NO_MEM_REPORTING=1 -DMOZ_VORBIS=1 -DMOZ_WEBM_ENCODER=1 -DMOZ_PERMISSIONS=1 -DENABLE_SYSTEM_EXTENSION_DIRS=1 -DMOZ_WEBGL_CONFORMANT=1 -DMOZ_GAMEPAD=1 -DMOZ_CRASHREPORTER=1 -DMOZ_CRASHREPORTER_INJECTOR=1 -DMOZ_CRASHREPORTER_ENABLE_PERCENT=100 -DMOZ_LIBAV_FFT=1 -DMOZ_WEBAPP_RUNTIME=1 -DMOZ_MAINTENANCE_SERVICE=1 -DMOZ_VERIFY_MAR_SIGNATURE=1 -DMOZ_ENABLE_SIGNMAR=1 -DMOZ_UPDATER=1 -DENABLE_TESTS=1 -DGTEST_HAS_RTTI=0 -D_VARIADIC_MAX=10 -DMOZ_CONTENT_SANDBOX=1 -DMOZ_GMP_SANDBOX=1 -DMOZ_SANDBOX=1 -DMOZ_FEEDS=1 -DMOZ_SAFE_BROWSING=1 -DMOZ_URL_CLASSIFIER=1 -DGL_PROVIDER_=1 -DMOZ_STACKWALKING=1 -DMOZ_LOGGING=1 -DFORCE_PR_LOG=1 -DMOZ_REPLACE_MALLOC=1 -DMOZ_MEMORY=1 -DMOZ_MEMORY_WINDOWS=1 -DMOZ_PAY=1 -DMOZ_ACTIVITIES=1 -DMOZ_SECUREELEMENT=1 -DJS_DEFAULT_JITREPORT_GRANULARITY=3 -DMOZ_OMNIJAR=1 -DMOZ_USER_DIR='"Mozilla"' -DMOZ_TREE_PIXMAN=1 -DHAVE_STDINT_H=1 -DHAVE_INTTYPES_H=1 -DMOZ_TREE_CAIRO=1 -DHAVE_UINT64_T=1 -DMOZ_ENABLE_SKIA=1 -DUSE_SKIA=1 -DUSE_SKIA_GPU=1 -DMOZ_XUL=1 -DMOZ_PROFILELOCKING=1 -DENABLE_MARIONETTE=1 -DBUILD_CTYPES=1 -DMOZ_PLACES=1 -DMOZ_SOCIAL=1 -DMOZ_SERVICES_COMMON=1 -DMOZ_SERVICES_CRYPTO=1 -DMOZ_SERVICES_HEALTHREPORT=1 -DMOZ_SERVICES_METRICS=1 -DMOZ_SERVICES_SYNC=1 -DMOZ_SERVICES_CLOUDSYNC=1 -DMOZ_JSDOWNLOADS=1 -DMOZ_MACBUNDLE_ID=org.mozilla.nightly -DMOZ_B2G_VERSION='"1.0.0"' -DMOZ_B2G_OS_NAME='""' -DMOZ_APP_UA_NAME='""' -DMOZ_APP_UA_VERSION='"46.0a1"' -DFIREFOX_VERSION=46.0a1 -DMOZILLA_OFFICIAL=1 -DMOZ_TELEMETRY_DISPLAY_REV=2 -DMOZ_TELEMETRY_REPORTING=1 -DMOZ_TELEMETRY_ON_BY_DEFAULT=1 -DMOZ_DATA_REPORTING=1 -DMOZ_DLL_SUFFIX='".dll"' -DA11Y_LOG=1 -DEXPOSE_INTL_API=1 -DENABLE_INTL_API=1 -DU_USING_ICU_NAMESPACE=0 -DMOZ_STATIC_JS=1 -DNO_NSPR_10_SUPPORT=1 -DAB_CD=en-US \

So are there issue building omni.ja?

Comment 1

[Nathan Froyd [:froydnj]]

This is the same thing as bug 1147271?

Comment 2

[Daniel Holbert [:dholbert]]

(In reply to Sebastian H. [:aryx][:archaeopteryx] from comment #0)
> Starting with
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=6a9d0f869621 Windows build jobs time out.

Looks like this issue went away when that changeset was backed out, here:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=691f2e687e46

So I think this is fixed (by backout), and the tree can reopen...  (froydnj, does that make sense?)

Comment 3

[Daniel Holbert [:dholbert]]

Looking at bug 1147271 more closely (froydnj's comments / try runs there in particular), I think that does make sense -- so I'm going to call this perma-fail FIXED-by-backout.

And since this is the only thing holding the tree closed, I'll reopen the tree as well (just in time for Christmas, hooray!)