Closed Bug 927088 Opened 6 years ago Closed 6 years ago

Intermittent Windows build failure "rm: cannot remove `../../dist/xpi-stage/specialpowers/chrome.manifest': Permission denied" or "IOError: [Errno 2] No such file or directory: '../../dist/xpi-stage/specialpowers/chrome\\..\\chrome.manifest'"

Categories

(Firefox Build System :: General, defect)

x86
Windows 7
defect
Not set

Tracking

(firefox25 wontfix, firefox26 wontfix, firefox27 fixed, firefox-esr24 wontfix)

RESOLVED FIXED
mozilla27
Tracking Status
firefox25 --- wontfix
firefox26 --- wontfix
firefox27 --- fixed
firefox-esr24 --- wontfix

People

(Reporter: RyanVM, Assigned: glandium)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

We saw a couple of these yesterday but assumed they were some funky needs-clobber issue. Guess not.

https://tbpl.mozilla.org/php/getParsedLog.php?id=29135413&tree=Mozilla-Central

WINNT 5.2 mozilla-central pgo-build on 2013-10-15 06:00:04 PDT for push febfe3c7732b
slave: w64-ix-slave07

evaluation from e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\config\rules.mk:1653:4:2:0$ rm -f "../../dist/xpi-stage/specialpowers/chrome.manifest"
e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\config\rules.mk:1454:0$ mozbuild.action.jar_maker main   -j ../../dist/xpi-stage/specialpowers/chrome -t e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build -f flat  -e --relativesrcdir=testing/specialpowers -c e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/testing/specialpowers/en-US -DNDEBUG -DTRIMMED -DXPI_NAME=specialpowers -DNO_NSPR_10_SUPPORT -DMOZ_METRO=1 -D_CRT_SECURE_NO_WARNINGS=1 -D_CRT_NONSTDC_NO_WARNINGS=1 -DHAVE_SEH_EXCEPTIONS=1 -DMOZ_MSVC_STL_WRAP__RAISE=1 -DHAVE_WINSDKVER_H=1 -DWINVER=0x502 -D_WIN32_WINNT=0x502 -D_WIN32_IE=0x0603 -DMOZ_WINSDK_TARGETVER=0x06020000 -DMOZ_NTDDI_WIN7=0x06010000 -DMOZ_WINSDK_MAXVER=0x06020000 -DHAVE_IO_H=1 -DHAVE_SETBUF=1 -DHAVE_ISATTY=1 -DX_DISPLAY_MISSING=1 -DJSGC_INCREMENTAL=1 -DMOZ_ENABLE_PROFILER_SPS=1 -DMOZ_PROFILING=1 -DMOZILLA_VERSION=\"27.0a1\" -DMOZILLA_VERSION_U=27.0a1 -DMOZILLA_UAVERSION=\"27.0\" -DHAVE_SNPRINTF=1 -D_WINDOWS=1 -DWIN32=1 -DXP_WIN=1 -DXP_WIN32=1 -DHW_THREADS=1 -DSTDC_HEADERS=1 -DWIN32_LEAN_AND_MEAN=1 -DNO_X11=1 -D_X86_=1 -DD_INO=d_ino -DMOZ_DEBUG_SYMBOLS=1 -DNS_ATTR_MALLOC= -DNS_WARN_UNUSED_RESULT= -DMALLOC_H=\<malloc.h\> -DHAVE_FORCEINLINE=1 -DHAVE_LOCALECONV=1 -DNIGHTLY_BUILD=1 -DNS_ENABLE_TSF=1 -DMOZ_UPDATE_CHANNEL=default -DEARLY_BETA_OR_EARLIER=1 -DMOZ_PHOENIX=1 -DMOZ_BUILD_APP=browser -DMOZ_PDF_PRINTING=1 -DMOZ_INSTRUMENT_EVENT_LOOP=1 -DMOZ_DISTRIBUTION_ID=\"org.mozilla\" -DIBMBIDI=1 -DACCESSIBILITY=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DMOZ_WEBRTC=1 -DMOZ_WEBRTC_ASSERT_ALWAYS=1 -DMOZ_WEBRTC_SIGNALING=1 -DMOZ_PEERCONNECTION=1 -DMOZ_SCTP=1 -DMOZ_SRTP=1 -DMOZ_SAMPLE_TYPE_FLOAT32=1 -DMOZ_WEBSPEECH=1 -DMOZ_RAW=1 -DMOZ_OGG=1 -DMOZ_WEBM=1 -DMOZ_DIRECTSHOW=1 -DMOZ_WMF=1 -DMOZ_MEDIA_NAVIGATOR=1 -DMOZ_VP8=1 -DMOZ_VP8_ERROR_CONCEALMENT=1 -DMOZ_VP8_ENCODER=1 -DVPX_X86_ASM=1 -DMOZ_WAVE=1 -DMOZ_SPEEX_RESAMPLER=1 -DMOZ_SOUNDTOUCH=1 -DMOZ_CUBEB=1 -DMOZ_VORBIS=1 -DMOZ_OPUS=1 -DENABLE_SYSTEM_EXTENSION_DIRS=1 -DMOZ_WEBGL=1 -DMOZ_WEBGL_CONFORMANT=1 -DMOZ_GAMEPAD=1 -DMOZ_CRASHREPORTER=1 -DMOZ_CRASHREPORTER_INJECTOR=1 -DMOZ_CRASHREPORTER_ENABLE_PERCENT=100 -DLIBJPEG_TURBO_X86_ASM=1 -DMOZ_WEBAPP_RUNTIME=1 -DMOZ_SIGNING=1 -DMOZ_MAINTENANCE_SERVICE=1 -DMOZ_VERIFY_MAR_SIGNATURE=1 -DMOZ_ENABLE_SIGNMAR=1 -DMOZ_UPDATER=1 -DMOZ_ENABLE_GTEST=1 -DGTEST_HAS_RTTI=0 -D_VARIADIC_MAX=10 -DMOZ_FEEDS=1 -DMOZ_SAFE_BROWSING=1 -DMOZ_URL_CLASSIFIER=1 -DGL_PROVIDER_=1 -DMOZ_LOGGING=1 -DMOZ_MEMORY=1 -DMOZ_MEMORY_WINDOWS=1 -DJSGC_INCREMENTAL=1 -DJS_CRASH_DIAGNOSTICS=1 -DMOZ_PAY=1 -DJS_DEFAULT_JITREPORT_GRANULARITY=3 -DMOZ_OMNIJAR=1 -DMOZ_USER_DIR=\"Mozilla\" -DMOZ_TREE_PIXMAN=1 -DHAVE_STDINT_H=1 -DMOZ_TREE_CAIRO=1 -DHAVE_UINT64_T=1 -DMOZ_ENABLE_SKIA=1 -DUSE_SKIA=1 -DSKIA_DLL=1 -DGR_DLL=1 -DUSE_SKIA_GPU=1 -DMOZ_XUL=1 -DMOZ_PROFILELOCKING=1 -DBUILD_CTYPES=1 -DMOZ_PLACES=1 -DMOZ_SOCIAL=1 -DMOZ_SERVICES_COMMON=1 -DMOZ_SERVICES_CRYPTO=1 -DMOZ_SERVICES_HEALTHREPORT=1 -DMOZ_SERVICES_METRICS=1 -DMOZ_SERVICES_SYNC=1 -DMOZ_MACBUNDLE_ID=org.mozilla.nightly -DMOZ_B2G_VERSION=\"1.0.0\" -DMOZ_B2G_OS_NAME=\"\" -DMOZ_APP_UA_NAME=\"\" -DMOZ_APP_UA_VERSION=\"27.0a1\" -DFIREFOX_VERSION=27.0a1 -DMOZ_TELEMETRY_DISPLAY_REV=2 -DMOZ_TELEMETRY_REPORTING=1 -DMOZ_TELEMETRY_ON_BY_DEFAULT=1 -DMOZ_DATA_REPORTING=1 -DMOZ_DLL_SUFFIX=\".dll\" e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/testing/specialpowers/jar.mn
evaluation from e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\config\rules.mk:1653:4:2:0: command 'rm -f "../../dist/xpi-stage/specialpowers/chrome.manifest"' failed, return code 1
<libs>: Found error
e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\config\recurse.mk:159:0: command 'C:/mozilla-build/python27/python.exe e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/build/pymake/pymake/../make.py -C testing/specialpowers libs' failed, return code 2processing e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/testing/specialpowers/jar.mn
rm: cannot remove `../../dist/xpi-stage/specialpowers/chrome.manifest': Permission denied

e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\config\rules.mk:695:0: command 'C:/mozilla-build/python27/python.exe e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/build/pymake/pymake/../make.py libs' failed, return code 2
e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\client.mk:401:0: command 'C:/mozilla-build/python27/python.exe e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/build/pymake/pymake/../make.py -j4 -C obj-firefox' failed, return code 2
e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\client.mk:250:0: command 'C:/mozilla-build/python27/python.exe e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/build/pymake/pymake/../make.py -f e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/client.mk realbuild MOZ_PROFILE_USE=1' failed, return code 2
e:\builds\moz2_slave\m-cen-w32-pgo-0000000000000000\build\client.mk:186:0: command 'C:/mozilla-build/python27/python.exe e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/build/pymake/pymake/../make.py -f e:/builds/moz2_slave/m-cen-w32-pgo-0000000000000000/build/client.mk profiledbuild' failed, return code 2
From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 23 files and 5 directories.
From ./dist/public: Kept 0 existing; Added/updated 0; Removed 0 files and 1 directories.
From ./dist/private: Kept 0 existing; Added/updated 0; Removed 157 files and 3 directories.
From _tests: Kept 11441 existing; Added/updated 2; Removed 4180 files and 404 directories.
From ../../dist/idl: Kept 1163 existing; Added/updated 0; Removed 0 files and 0 directories.
From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 23 files and 5 directories.
From ../../dist/idl: Kept 1163 existing; Added/updated 0; Removed 0 files and 0 directories.
From ./dist/include: Kept 3573 existing; Added/updated 10; Removed 320 files and 6 directories.
From ./dist/idl: Kept 1163 existing; Added/updated 0; Removed 0 files and 0 directories.
program finished with exit code 2
Whines when it can't remove it, whines when it isn't there.

https://tbpl.mozilla.org/php/getParsedLog.php?id=29172851&tree=Mozilla-Inbound
Summary: Intermittent Windows PGO build failure "rm: cannot remove `../../dist/xpi-stage/specialpowers/chrome.manifest': Permission denied" → Intermittent Windows PGO build failure "rm: cannot remove `../../dist/xpi-stage/specialpowers/chrome.manifest': Permission denied" or "IOError: [Errno 2] No such file or directory: '../../dist/xpi-stage/specialpowers/chrome\\..\\chrome.manifest'"
There's a race condition in testing/specialpowers/chrome.manifest.
(In reply to Mike Hommey [:glandium] from comment #2)
> There's a race condition in testing/specialpowers/chrome.manifest.

Err, testing/specialpowers/Makefile.in.
hah, i created that race condition
Blocks: 770426
It turns out that instead of relying on luck putting things in order between the chrome.manifest generated by jarmaker and the one copied over from source, we can just make jarmaker create a proper chrome.manifest.

AFAICT from a local build, this creates the same file, albeit in a different order.
Attachment #817654 - Flags: review?(gps)
Assignee: nobody → mh+mozilla
Status: NEW → ASSIGNED
Race conditions be racy, not just on PGO...
Summary: Intermittent Windows PGO build failure "rm: cannot remove `../../dist/xpi-stage/specialpowers/chrome.manifest': Permission denied" or "IOError: [Errno 2] No such file or directory: '../../dist/xpi-stage/specialpowers/chrome\\..\\chrome.manifest'" → Intermittent Windows build failure "rm: cannot remove `../../dist/xpi-stage/specialpowers/chrome.manifest': Permission denied" or "IOError: [Errno 2] No such file or directory: '../../dist/xpi-stage/specialpowers/chrome\\..\\chrome.manifest'"
Attachment #817654 - Flags: review?(gps) → review+
https://hg.mozilla.org/mozilla-central/rev/8d21216df458
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla27
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.