Race condition causing IOError: [Errno 2] No such file or directory: '.deps/nsINavHistoryService.h.pp'

RESOLVED FIXED in mozilla12

Status

()

Core
Build Config
--
critical
RESOLVED FIXED
6 years ago
6 years ago

People

(Reporter: emorley, Unassigned)

Tracking

Trunk
mozilla12
x86
Linux
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

6 years ago
Created attachment 575667 [details]
Log.txt.gz

Callek spotted that the first nightly linux aurora build here failed:
https://tbpl.mozilla.org/?tree=Mozilla-Aurora&rev=9ece8c9cfebe

(For some reason the log isn't accessible via TBPL, so I've attached it)

It failed with:
IOError: [Errno 2] No such file or directory: '.deps/nsINavHistoryService.h.pp'

Which then caused pythonpath.py (running header.py) to throw at:
https://hg.mozilla.org/releases/mozilla-aurora/file/dfc153c765b8/xpcom/idl-parser/header.py#l524

{
nsINavHistoryService.idl
/tools/python/bin/python2.5 /builds/slave/m-aurora-lnx-ntly/build/config/pythonpath.py \
	  -I/builds/slave/m-aurora-lnx-ntly/build/other-licenses/ply \
	  -I/builds/slave/m-aurora-lnx-ntly/build/xpcom/idl-parser \
	  /builds/slave/m-aurora-lnx-ntly/build/xpcom/idl-parser/header.py --cachedir=../../../xpcom/idl-parser -I/builds/slave/m-aurora-lnx-ntly/build/toolkit/components/places -I../../../dist/idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/places/nsINavHistoryService.idl -d .deps/nsINavHistoryService.h.pp -o _xpidlgen/nsINavHistoryService.h
rm -f *.gcda
/builds/slave/m-aurora-lnx-ntly/build/obj-firefox/config/nsinstall -R -m 644 /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginInfo.idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginManager.idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginManagerCrypto.idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginManagerIEMigrationHelper.idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginManagerPrompter.idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginManagerStorage.idl /builds/slave/m-aurora-lnx-ntly/build/toolkit/components/passwordmgr/nsILoginMetaInfo.idl ../../../dist/idl
Traceback (most recent call last):
  File "/builds/slave/m-aurora-lnx-ntly/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/m-aurora-lnx-ntly/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/m-aurora-lnx-ntly/build/xpcom/idl-parser/header.py", line 524, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsINavHistoryService.h.pp'
}
(Reporter)

Comment 1

6 years ago
Philor, any idea why the log was missing via TBPL?
(Reporter)

Comment 2

6 years ago
On a try run yesterday (based on inbound tip), I got the following Linux debug build failure, which I'm presuming is due to the same root cause:
https://tbpl.mozilla.org/php/getParsedLog.php?id=7493960&tree=Try
{
make -C battery export
make[6]: Entering directory `/builds/slave/try-lnx-dbg/build/obj-firefox/dom/battery'
/usr/bin/perl /builds/slave/try-lnx-dbg/build/build/autoconf/make-makefile -t /builds/slave/try-lnx-dbg/build -d ../.. test/Makefile
mkdir -p /builds/slave/try-lnx-dbg/build/obj-firefox/dom/battery/.deps
mkdir -p _xpidlgen
nsIDOMBatteryManager.idl
/tools/python/bin/python2.5 /builds/slave/try-lnx-dbg/build/config/pythonpath.py \
	  -I/builds/slave/try-lnx-dbg/build/other-licenses/ply \
	  -I/builds/slave/try-lnx-dbg/build/xpcom/idl-parser \
	  /builds/slave/try-lnx-dbg/build/xpcom/idl-parser/header.py --cachedir=../../xpcom/idl-parser -I/builds/slave/try-lnx-dbg/build/dom/battery -I../../dist/idl /builds/slave/try-lnx-dbg/build/dom/battery/nsIDOMBatteryManager.idl -d .deps/nsIDOMBatteryManager.h.pp -o _xpidlgen/nsIDOMBatteryManager.h
nsIDOMNavigatorBattery.idl
/tools/python/bin/python2.5 /builds/slave/try-lnx-dbg/build/config/pythonpath.py \
	  -I/builds/slave/try-lnx-dbg/build/other-licenses/ply \
	  -I/builds/slave/try-lnx-dbg/build/xpcom/idl-parser \
	  /builds/slave/try-lnx-dbg/build/xpcom/idl-parser/header.py --cachedir=../../xpcom/idl-parser -I/builds/slave/try-lnx-dbg/build/dom/battery -I../../dist/idl /builds/slave/try-lnx-dbg/build/dom/battery/nsIDOMNavigatorBattery.idl -d .deps/nsIDOMNavigatorBattery.h.pp -o _xpidlgen/nsIDOMNavigatorBattery.h
Traceback (most recent call last):
  File "/builds/slave/try-lnx-dbg/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/try-lnx-dbg/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/try-lnx-dbg/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIDOMBatteryManager.h.pp'
Traceback (most recent call last):
  File "/builds/slave/try-lnx-dbg/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/try-lnx-dbg/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/try-lnx-dbg/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIDOMNavigatorBattery.h.pp'
creating dom/battery/test/Makefile
make[6]: *** [_xpidlgen/nsIDOMBatteryManager.h] Error 1
make[6]: *** Deleting file `_xpidlgen/nsIDOMBatteryManager.h'
make[6]: *** Waiting for unfinished jobs....
make[6]: *** [_xpidlgen/nsIDOMNavigatorBattery.h] Error 1
make[6]: *** Deleting file `_xpidlgen/nsIDOMNavigatorBattery.h'
}
(Reporter)

Comment 3

6 years ago
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=3b3c9e09abe3
https://tbpl.mozilla.org/php/getParsedLog.php?id=7514783&tree=Mozilla-Inbound
{
make[7]: Entering directory `/builds/slave/m-in-lnx64/build/obj-firefox/toolkit/components/exthelper'
mkdir -p /builds/slave/m-in-lnx64/build/obj-firefox/toolkit/components/exthelper/.deps
amIWebInstaller.idl
/tools/python/bin/python2.5 /builds/slave/m-in-lnx64/build/config/pythonpath.py \
	  -I/builds/slave/m-in-lnx64/build/other-licenses/ply \
	  -I/builds/slave/m-in-lnx64/build/xpcom/idl-parser \
	  /builds/slave/m-in-lnx64/build/xpcom/idl-parser/header.py --cachedir=../../../xpcom/idl-parser -I/builds/slave/m-in-lnx64/build/toolkit/mozapps/extensions -I../../../dist/idl /builds/slave/m-in-lnx64/build/toolkit/mozapps/extensions/amIWebInstaller.idl -d .deps/amIWebInstaller.h.pp -o _xpidlgen/amIWebInstaller.h
/builds/slave/m-in-lnx64/build/obj-firefox/config/nsinstall -R -m 644 /builds/slave/m-in-lnx64/build/toolkit/mozapps/extensions/amIInstallTrigger.idl /builds/slave/m-in-lnx64/build/toolkit/mozapps/extensions/amIWebInstallListener.idl /builds/slave/m-in-lnx64/build/toolkit/mozapps/extensions/amIWebInstaller.idl ../../../dist/idl
/builds/slave/m-in-lnx64/build/obj-firefox/config/nsinstall -R -m 644 _xpidlgen/amIInstallTrigger.h _xpidlgen/amIWebInstallListener.h _xpidlgen/amIWebInstaller.h ../../../dist/include
make[6]: Leaving directory `/builds/slave/m-in-lnx64/build/obj-firefox/toolkit/mozapps/extensions'
nsIDownload.idl
/tools/python/bin/python2.5 /builds/slave/m-in-lnx64/build/config/pythonpath.py \
	  -I/builds/slave/m-in-lnx64/build/other-licenses/ply \
	  -I/builds/slave/m-in-lnx64/build/xpcom/idl-parser \
	  /builds/slave/m-in-lnx64/build/xpcom/idl-parser/header.py --cachedir=../../../xpcom/idl-parser -I/builds/slave/m-in-lnx64/build/toolkit/components/downloads -I../../../dist/idl /builds/slave/m-in-lnx64/build/toolkit/components/downloads/nsIDownload.idl -d .deps/nsIDownload.h.pp -o _xpidlgen/nsIDownload.h
Traceback (most recent call last):
  File "/builds/slave/m-in-lnx64/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/m-in-lnx64/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/m-in-lnx64/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIDownload.h.pp'
make[7]: *** [_xpidlgen/nsIDownload.h] Error 1
make[7]: *** Deleting file `_xpidlgen/nsIDownload.h'
make[7]: *** Waiting for unfinished jobs....
make[7]: Entering directory `/builds/slave/m-in-lnx64/build/obj-firefox/toolkit/components/filepicker'
mkdir -p /builds/slave/m-in-lnx64/build/obj-firefox/toolkit/components/filepicker/.deps
make[7]: Leaving directory `/builds/slave/m-in-lnx64/build/obj-firefox/toolkit/components/downloads'
}
(Reporter)

Comment 4

6 years ago
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=38e445672780
https://tbpl.mozilla.org/php/getParsedLog.php?id=7515114&tree=Mozilla-Inbound
{
make -C indexedDB export
make[6]: Entering directory `/builds/slave/m-in-lnx/build/obj-firefox/dom/indexedDB'
/usr/bin/perl /builds/slave/m-in-lnx/build/build/autoconf/make-makefile -t /builds/slave/m-in-lnx/build -d ../.. test/Makefile
mkdir -p /builds/slave/m-in-lnx/build/obj-firefox/dom/indexedDB/.deps
mkdir -p _xpidlgen
nsIIDBCursor.idl
/tools/python/bin/python2.5 /builds/slave/m-in-lnx/build/config/pythonpath.py \
	  -I/builds/slave/m-in-lnx/build/other-licenses/ply \
	  -I/builds/slave/m-in-lnx/build/xpcom/idl-parser \
	  /builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py --cachedir=../../xpcom/idl-parser -I/builds/slave/m-in-lnx/build/dom/indexedDB -I../../dist/idl /builds/slave/m-in-lnx/build/dom/indexedDB/nsIIDBCursor.idl -d .deps/nsIIDBCursor.h.pp -o _xpidlgen/nsIIDBCursor.h
nsIIDBCursorWithValue.idl
/tools/python/bin/python2.5 /builds/slave/m-in-lnx/build/config/pythonpath.py \
	  -I/builds/slave/m-in-lnx/build/other-licenses/ply \
	  -I/builds/slave/m-in-lnx/build/xpcom/idl-parser \
	  /builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py --cachedir=../../xpcom/idl-parser -I/builds/slave/m-in-lnx/build/dom/indexedDB -I../../dist/idl /builds/slave/m-in-lnx/build/dom/indexedDB/nsIIDBCursorWithValue.idl -d .deps/nsIIDBCursorWithValue.h.pp -o _xpidlgen/nsIIDBCursorWithValue.h
Traceback (most recent call last):
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIIDBCursor.h.pp'
Traceback (most recent call last):
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIIDBCursorWithValue.h.pp'
creating dom/indexedDB/test/Makefile
make[6]: *** [_xpidlgen/nsIIDBCursor.h] Error 1
make[6]: *** Deleting file `_xpidlgen/nsIIDBCursor.h'
make[6]: *** Waiting for unfinished jobs....
make[6]: *** [_xpidlgen/nsIIDBCursorWithValue.h] Error 1
}
Severity: major → critical
(Reporter)

Comment 5

6 years ago
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=9838cc0b62ee
https://tbpl.mozilla.org/php/getParsedLog.php?id=7521873&tree=Mozilla-Inbound
{
make -C battery export
make[6]: Entering directory `/builds/slave/m-in-lnx/build/obj-firefox/dom/battery'
/usr/bin/perl /builds/slave/m-in-lnx/build/build/autoconf/make-makefile -t /builds/slave/m-in-lnx/build -d ../.. test/Makefile
mkdir -p /builds/slave/m-in-lnx/build/obj-firefox/dom/battery/.deps
mkdir -p _xpidlgen
/tools/python/bin/python2.5 /builds/slave/m-in-lnx/build/config/pythonpath.py \
	  -I/builds/slave/m-in-lnx/build/other-licenses/ply \
	  -I/builds/slave/m-in-lnx/build/xpcom/idl-parser \
	  /builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py --cachedir=../../xpcom/idl-parser -I/builds/slave/m-in-lnx/build/dom/battery -I../../dist/idl /builds/slave/m-in-lnx/build/dom/battery/nsIDOMBatteryManager.idl -d .deps/nsIDOMBatteryManager.h.pp -o _xpidlgen/nsIDOMBatteryManager.h
/tools/python/bin/python2.5 /builds/slave/m-in-lnx/build/config/pythonpath.py \
	  -I/builds/slave/m-in-lnx/build/other-licenses/ply \
	  -I/builds/slave/m-in-lnx/build/xpcom/idl-parser \
	  /builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py --cachedir=../../xpcom/idl-parser -I/builds/slave/m-in-lnx/build/dom/battery -I../../dist/idl /builds/slave/m-in-lnx/build/dom/battery/nsIDOMNavigatorBattery.idl -d .deps/nsIDOMNavigatorBattery.h.pp -o _xpidlgen/nsIDOMNavigatorBattery.h
Traceback (most recent call last):
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIDOMBatteryManager.h.pp'
Traceback (most recent call last):
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 52, in <module>
    main(sys.argv[1:])
  File "/builds/slave/m-in-lnx/build/config/pythonpath.py", line 44, in main
    execfile(script, frozenglobals)
  File "/builds/slave/m-in-lnx/build/xpcom/idl-parser/header.py", line 530, in <module>
    depfd = open(options.depfile, 'w')
IOError: [Errno 2] No such file or directory: '.deps/nsIDOMNavigatorBattery.h.pp'
make[6]: *** [_xpidlgen/nsIDOMBatteryManager.h] Error 1
make[6]: *** Deleting file `_xpidlgen/nsIDOMBatteryManager.h'
make[6]: *** Waiting for unfinished jobs....
make[6]: *** [_xpidlgen/nsIDOMNavigatorBattery.h] Error 1
}
This looks like a race condition to create the .deps directory, maybe? I'm not sure what made this start showing up now.
https://tbpl.mozilla.org/php/getParsedLog.php?id=7558792&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=7618156&tree=Try
https://tbpl.mozilla.org/php/getParsedLog.php?id=7777752&tree=Mozilla-Beta
Created attachment 583812 [details] [diff] [review]
Patch

Joerg Sonnenberger submitted this patch, which I believe will fix this bug.
Attachment #583812 - Flags: review+
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #10)
> Created attachment 583812 [details] [diff] [review]
> Patch
> 
> Joerg Sonnenberger submitted this patch, which I believe will fix this bug.

I did significant testing on this and this patch certainly seems to fix the issue for me.
(In reply to Bill Gianopoulos from comment #11)
 
> I did significant testing on this and this patch certainly seems to fix the
        ^^^^^^^^^^^

Word I was looking for was extensive.
https://hg.mozilla.org/integration/mozilla-inbound/rev/2a1f2758ad0d
Target Milestone: --- → mozilla12
(Reporter)

Comment 14

6 years ago
https://hg.mozilla.org/mozilla-central/rev/2a1f2758ad0d
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Duplicate of this bug: 721735
You need to log in before you can comment on or make changes to this bug.