Closed Bug 936451 Opened 9 years ago Closed 6 years ago

xpcshell crash kills 'mach package' step

Categories

(Core :: DOM: Core & HTML, defect)

defect
Not set
normal

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: jwalker, Unassigned)

Details

Recent patches to bug 933727, create usable Firefox builds, however they fail the 'mach package' step:

$ mach package

(blah blah)

 0:08.34 Executing /Users/joe/projects/mozilla/devtools/obj-plain/dist/bin/xpcshell -g /Users/joe/projects/mozilla/devtools/obj-plain/dist/Nightly.app/Contents/MacOS -a /Users/joe/projects/mozilla/devtools/obj-plain/dist/Nightly.app/Contents/MacOS/browser -f /Users/joe/projects/mozilla/devtools/toolkit/mozapps/installer/precompile_cache.js -e precompile_startupcache("resource://app/");

(blah blah)

 0:18.05 Traceback (most recent call last):
 0:18.05 File "/Users/joe/projects/mozilla/devtools/toolkit/mozapps/installer/packager.py", line 375, in <module>
 0:18.05 main()
 0:18.05 File "/Users/joe/projects/mozilla/devtools/toolkit/mozapps/installer/packager.py", line 367, in main
 0:18.05 args.source, gre_path, base)
 0:18.05 File "/Users/joe/projects/mozilla/devtools/toolkit/mozapps/installer/packager.py", line 148, in precompile_cache
 0:18.05 errors.fatal('Error while running startup cache precompilation')
 0:18.05 File "/Users/joe/projects/mozilla/devtools/python/mozbuild/mozpack/errors.py", line 101, in fatal
 0:18.05 self._handle(self.FATAL, msg)
 0:18.05 File "/Users/joe/projects/mozilla/devtools/python/mozbuild/mozpack/errors.py", line 96, in _handle
 0:18.05 raise ErrorMessage(msg)
 0:18.05 mozpack.errors.ErrorMessage: Error: Error while running startup cache precompilation
 0:18.07 make[3]: *** [stage-package] Error 1
 0:18.07 make[3]: Leaving directory `/Users/joe/projects/mozilla/devtools/obj-plain/browser/installer'
 0:18.07 make[2]: *** [make-package] Error 2
 0:18.07 make[2]: Leaving directory `/Users/joe/projects/mozilla/devtools/obj-plain/browser/installer'
 0:18.07 make[1]: *** [default] Error 2
 0:18.07 make[1]: Leaving directory `/Users/joe/projects/mozilla/devtools/obj-plain/browser/installer'
 0:18.07 make: *** [package] Error 2
 0:18.07 make: Leaving directory `/Users/joe/projects/mozilla/devtools/obj-plain'



If I run the xpcshell command directly (note - I added the dump() at the end):

$ /Users/joe/projects/mozilla/devtools/obj-plain/dist/bin/xpcshell -g /Users/joe/projects/mozilla/devtools/obj-plain/dist/Nightly.app/Contents/MacOS -a /Users/joe/projects/mozilla/devtools/obj-plain/dist/Nightly.app/Contents/MacOS/browser -f /Users/joe/projects/mozilla/devtools/toolkit/mozapps/installer/precompile_cache.js -e 'precompile_startupcache("resource://app/");dump("done\n")'

(blah blah)

resource://app/modules/UITour.jsm
resource://app/modules/webappsUI.jsm
resource://app/modules/webrtcUI.jsm
done
[1]    42264 segmentation fault  /Users/joe/projects/mozilla/devtools/obj-plain/dist/bin/xpcshell -g  -a  -f

The cache file pointed to by MOZ_STARTUP_CACHE does not appear to be created.

This is currently preventing bug 933727 from landing.
(In reply to Joe Walker [:jwalker] from comment #0)
> [1]    42264 segmentation fault 
> /Users/joe/projects/mozilla/devtools/obj-plain/dist/bin/xpcshell -g  -a  -f

This means your patch triggers a crash in some parts of gecko. This is either a problem with your patch or a problem with gecko. Either way, it's not a build config problem.
Component: Build Config → JavaScript Engine
Product: Firefox → Core
(gdb) run

Starting program: /Users/joe/projects/mozilla/devtools/obj-plain/dist/bin/xpcshell -g /Users/joe/projects/mozilla/devtools/obj-plain/dist/Nightly.app/Contents/MacOS -a /Users/joe/projects/mozilla/devtools/obj-plain/dist/Nightly.app/Contents/MacOS/browser -f /Users/joe/projects/mozilla/devtools/toolkit/mozapps/installer/precompile_cache.js -e pscra\(\)
Reading symbols for shared libraries ++++++++++........................................................................................................................................................................................... done
Reading symbols for shared libraries . done

(blah blah)

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
0x0000000100552adb in nsContentUtils::IsSystemPrincipal (aPrincipal=0x107c70220) at /Users/joe/projects/mozilla/devtools/content/base/src/nsContentUtils.cpp:4438
4438	  nsresult rv = sSecurityManager->IsSystemPrincipal(aPrincipal, &isSystem);

(gdb) backtrace

#0  0x0000000100552adb in nsContentUtils::IsSystemPrincipal (aPrincipal=0x107c70220) at /Users/joe/projects/mozilla/devtools/content/base/src/nsContentUtils.cpp:4438
#1  0x00000001005f2cf6 in nsXMLHttpRequest::IsSystemXHR () at /Users/joe/projects/mozilla/devtools/content/base/src/nsXMLHttpRequest.h:1496
Previous frame inner to this frame (gdb could not unwind past this frame)
And again with --enable-debug

(gdb) backtrace
#0  0x0000000100cb1677 in nsContentUtils::IsSystemPrincipal (aPrincipal=0x10f16d320) at /Users/joe/projects/mozilla/devtools/content/base/src/nsContentUtils.cpp:4438
#1  0x0000000100e22750 in nsXMLHttpRequest::IsSystemXHR (this=0x110bf9400) at /Users/joe/projects/mozilla/devtools/content/base/src/nsXMLHttpRequest.cpp:1496
#2  0x0000000100e2a4fc in nsXMLHttpRequest::OnStartRequest (this=0x110bf9400, request=0x1101d7f80, ctxt=0x0) at /Users/joe/projects/mozilla/devtools/content/base/src/nsXMLHttpRequest.cpp:1903
#3  0x0000000100e2b807 in non-virtual thunk to nsXMLHttpRequest::OnStartRequest(nsIRequest*, nsISupports*) (this=0x110bf9468, request=0x1101d7f80, ctxt=0x0) at /Users/joe/projects/mozilla/devtools/content/base/src/nsXMLHttpRequest.cpp:2061
#4  0x00000001002a6c06 in nsStreamListenerWrapper::OnStartRequest (this=0x110bc08e0, aRequest=0x1101d7f80, aContext=0x0) at nsStreamListenerWrapper.h:27
#5  0x0000000100225d2d in nsBaseChannel::OnStartRequest (this=0x1101d7f30, request=0x110b8ba00, ctxt=0x0) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsBaseChannel.cpp:718
#6  0x0000000100226007 in non-virtual thunk to nsBaseChannel::OnStartRequest(nsIRequest*, nsISupports*) (this=0x1101d7fb0, request=0x110b8ba00, ctxt=0x0) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsBaseChannel.cpp:720
#7  0x00000001002479a3 in nsInputStreamPump::OnStateStart (this=0x110b8ba00) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsInputStreamPump.cpp:516
#8  0x0000000100247535 in nsInputStreamPump::OnInputStreamReady (this=0x110b8ba00, stream=0x1105c8798) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsInputStreamPump.cpp:430
#9  0x00000001002485df in non-virtual thunk to nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) (this=0x110b8ba08, stream=0x1105c8798) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsInputStreamPump.cpp:489
#10 0x0000000102ef7250 in nsInputStreamReadyEvent::Run (this=0x110bd9b00) at /Users/joe/projects/mozilla/devtools/xpcom/io/nsStreamUtils.cpp:85
#11 0x0000000102f22be3 in nsThread::ProcessNextEvent (this=0x10dc890e0, mayWait=true, result=0x7fff5fbfea9e) at /Users/joe/projects/mozilla/devtools/xpcom/threads/nsThread.cpp:610
#12 0x0000000102e824f9 in NS_ProcessNextEvent (thread=0x10dc890e0, mayWait=true) at nsThreadUtils.cpp:251
#13 0x0000000102f22350 in nsThread::Shutdown (this=0x10dc89420) at /Users/joe/projects/mozilla/devtools/xpcom/threads/nsThread.cpp:465
#14 0x0000000100290b71 in nsSocketTransportService::Shutdown (this=0x10ee7a310) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsSocketTransportService2.cpp:500
#15 0x000000010024bdbb in nsIOService::SetOffline (this=0x10dcfba80, offline=true) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsIOService.cpp:748
#16 0x000000010024e545 in nsIOService::Observe (this=0x10dcfba80, subject=0x10dc53338, topic=0x104c23aae "xpcom-shutdown", data=0x0) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsIOService.cpp:944
#17 0x000000010024e64f in _ZThn8_N11nsIOService7ObserveEP11nsISupportsPKcPKDs (this=0x10dcfba88, subject=0x10dc53338, topic=0x104c23aae "xpcom-shutdown", data=0x0) at /Users/joe/projects/mozilla/devtools/netwerk/base/src/nsIOService.cpp:955
#18 0x0000000102eadd49 in nsObserverList::NotifyObservers (this=0x10dc06150, aSubject=0x10dc53338, aTopic=0x104c23aae "xpcom-shutdown", someData=0x0) at /Users/joe/projects/mozilla/devtools/xpcom/ds/nsObserverList.cpp:96
#19 0x0000000102eb1204 in nsObserverService::NotifyObservers (this=0x10dc10740, aSubject=0x10dc53338, aTopic=0x104c23aae "xpcom-shutdown", someData=0x0) at /Users/joe/projects/mozilla/devtools/xpcom/ds/nsObserverService.cpp:326
#20 0x0000000102e8b5b6 in mozilla::ShutdownXPCOM (servMgr=0x0) at /Users/joe/projects/mozilla/devtools/xpcom/build/nsXPComInit.cpp:657
#21 0x0000000102e8b385 in NS_ShutdownXPCOM (servMgr=0x0) at /Users/joe/projects/mozilla/devtools/xpcom/build/nsXPComInit.cpp:618
#22 0x0000000101be2f22 in XRE_XPCShellMain (argc=4, argv=0x7fff5fbff758, envp=0x7fff5fbff780) at /Users/joe/projects/mozilla/devtools/js/xpconnect/src/XPCShellImpl.cpp:1587
#23 0x0000000100000e28 in main (argc=9, argv=0x7fff5fbff730, envp=0x7fff5fbff780) at /Users/joe/projects/mozilla/devtools/js/xpconnect/shell/xpcshell.cpp:45
So here's my best guess at what's going on.

xpcshell is computing a startup cache by loading each of the JS files in turn and running a single pass through the event loop on each to see what things they load. The patch in question splits out a large file (gcli.jsm) into a number of smaller files, some of which use other resources fetched using an XHR request to a resource://gre/modules/devtools/... URL.

It looks like there is some sort of security principle which is normally setup for a working Firefox, which isn't there in the xpcshell environment, so when XHR tries to find out if it's allowed to read from the resource://gre/modules/devtools/... URL, it dumps on an null pointer.

So there's an easy workaround, I simply setTimeout before doing the XHR so the call doesn't happen on the first pass through the event loop. This somewhat subverts the whole point of this process, however since these files should not be parsed on startup, I'm not sure what the performance impact will be.
No longer blocks: 933727
I'm not sure why this got filed in JS, since there's no JS in that stack. I guess this was probably a transient bug in DOM somewhere and no longer needs attention.
Status: NEW → RESOLVED
Closed: 6 years ago
Component: JavaScript Engine → DOM
Resolution: --- → INCOMPLETE
The analysis in comment 4 explains why this was filed against the JS engine. I doubt this has anything to do with the DOM. Did you see that?
(In reply to Joe Walker [:jwalker] (needinfo me or ping on irc) from comment #6)
> The analysis in comment 4 explains why this was filed against the JS engine.
> I doubt this has anything to do with the DOM. Did you see that?

Yes. However, xpcshell, XHR, setTimeout, the event loop, etc are all DOM. And the crash listed has no JS whatsoever in its stack.

Does the crash it still reproduce for you?
Status: RESOLVED → REOPENED
Flags: needinfo?(jwalker)
Resolution: INCOMPLETE → ---
Can't reproduce, closing. Thanks.
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Flags: needinfo?(jwalker)
Resolution: --- → INCOMPLETE
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.