Closed Bug 701942 Opened 9 years ago Closed 9 years ago

Frequent mochitest-plain-3 Shutdown | application timed out after 330 seconds with no output

Categories

(Release Engineering :: General, defect)

x86_64
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 614146

People

(Reporter: mbrubeck, Assigned: jhford)

Details

This has been happening for a while, but they've just been dropping into the catch-all bug 614146.

For example:

https://tbpl.mozilla.org/php/getParsedLog.php?id=7357931&tree=Mozilla-Inbound
Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitests-3/5 on 2011-11-11 14:11:17 PST for push 259bae11858b

The screenshot shows no visible browser windows; the "Utilities" folder is open in the Finder.

--DOMWINDOW == 12 (0x146fd14b8) [serial = 2553] [outer = 0x0] [url = about:blank]
--DOMWINDOW == 11 (0x145f11688) [serial = 2548] [outer = 0x0] [url = about:blank]
--DOMWINDOW == 10 (0x1469b35b8) [serial = 2543] [outer = 0x0] [url = about:blank]
--DOMWINDOW == 9 (0x14793d568) [serial = 2593] [outer = 0x0] [url = about:blank]
--DOMWINDOW == 8 (0x146da6858) [serial = 2568] [outer = 0x0] [url = about:blank]
--DOMWINDOW == 7 (0x11b68ca98) [serial = 12] [outer = 0x0] [url = http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=3&chunkByDir=4&hideResultsTable=1]
--DOMWINDOW == 6 (0x118f663e8) [serial = 2] [outer = 0x0] [url = about:blank]
--DOMWINDOW == 5 (0x152d25f48) [serial = 5274] [outer = 0x0] [url = about:blank]
WARNING: NS_ENSURE_TRUE(asyncCloseWasCalled) failed: file /builds/slave/m-in-osx64-dbg/build/storage/src/mozStorageConnection.cpp, line 835
WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/m-in-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/m-in-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
TEST-UNEXPECTED-FAIL | Shutdown | application timed out after 330 seconds with no output

Can't trigger Breakpad, just killing process
INFO | automation.py | Application ran for: 0:12:25.677817
INFO | automation.py | Reading PID log: /var/folders/Hs/HsDn6a9SG8idoIya6p9mtE+++TI/-Tmp-/tmpjaUZu_pidlog
TEST-UNEXPECTED-FAIL | automationutils.processLeakLog() | missing output line for total leaks!

INFO | runtests.py | Running tests: end.
program finished with exit code 247
elapsedTime=751.297119
TinderboxPrint: mochitest-plain-3<br/>14253/0/2845&nbsp;<em class="testfail">L-FAIL</em>
Unknown Error: command finished with exit code: 247
Approximately 100% of recent cases of bug 614146 fall under this bug.  Some examples from the past couple days:

https://tbpl.mozilla.org/php/getParsedLog.php?id=7358681&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=7338548&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=7338133&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=7334635&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=7330654&tree=Firefox
https://tbpl.mozilla.org/php/getParsedLog.php?id=7318174&tree=Mozilla-Inbound

Moving this to Server Ops because it seems to be specific to the new rev4 slaves.  Feel free to reassign it elsewhere if appropriate.
Assignee: nobody → server-ops-releng
Component: General → Server Operations: RelEng
Product: Core → mozilla.org
QA Contact: general → zandr
Version: 11 Branch → other
Blocks: 614146
This isn't actionable for us - releng should investigate, and if it's a hardware issue, we'll take care of it.
Assignee: server-ops-releng → nobody
Component: Server Operations: RelEng → Release Engineering
QA Contact: zandr → release
Joel,

have you seen this or done any triage?
Assignee: nobody → jhford
I don't see anything in the screenshots in these logs that I have checked (comment 14,13) that points to machine configuration.  Interesting is that the NightlyDebug application is showing in the menubar but there are no windows of Firefox being drawn on screen.  The state of the finder window in the screenshots suggest that NightlyDebug also has focus.

Comparing the last common bits of the failing log:

WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/srv-cen-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
WARNING: NS_ENSURE_TRUE(asyncCloseWasCalled) failed: file /builds/slave/srv-cen-osx64-dbg/build/storage/src/mozStorageConnection.cpp, line 835
WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/srv-cen-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
TEST-UNEXPECTED-FAIL | Shutdown | application timed out after 330 seconds with no output

With a passing log [1]:

WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/srv-cen-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/srv-cen-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
WARNING: not an nsIRDFRemoteDataSource: 'remote != nsnull', file /builds/slave/srv-cen-osx64-dbg/build/rdf/datasource/src/nsLocalStore.cpp, line 312
WARNING: NS_ENSURE_TRUE(mTextInputHandler) failed: file /builds/slave/srv-cen-osx64-dbg/build/widget/src/cocoa/nsChildView.mm, line 4068
WARNING: OOPDeinit() without successful OOPInit(): file /builds/slave/srv-cen-osx64-dbg/build/toolkit/crashreporter/nsExceptionHandler.cpp, line 1938
nsStringStats
 => mAllocCount:        2459159
 => mReallocCount:       205499
 => mFreeCount:         2459159
 => mShareCount:        3770692
 => mAdoptCount:         230551
 => mAdoptFreeCount:     230551
INFO | automation.py | Application ran for: 0:06:50.225722
INFO | automation.py | Reading PID log: /var/folders/Hs/HsDn6a9SG8idoIya6p9mtE+++TI/-Tmp-/tmpTUsQWwpidlog

It looks like the difference here is that asyncCloseWasCalled fails when the browser hangs.  I wonder if this is an async timing issue that is exacerbated, not caused, by the hardware change. 




[1] https://tbpl.mozilla.org/php/getParsedLog.php?id=7397619&tree=Services-Central&full=1
This issue is happening on Fedora 64 (comment 23, 20) and Windows (comment 22,21).  Updating bug to reflect that this is not a rev4 osx specific issue.

I am also going to mark this as a dup of bug 614146 since there is no reason to track OS X failures separately from Windows and Fedora ones, especially as they are now being reported in either bug.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → DUPLICATE
Summary: Frequent OS X64 rev4 mochitest-plain-3 Shutdown | application timed out after 330 seconds with no output → Frequent mochitest-plain-3 Shutdown | application timed out after 330 seconds with no output
Duplicate of bug: 614146
No longer blocks: 438871, 614146
Whiteboard: [orange]
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.