Closed
Bug 1351354
Opened 7 years ago
Closed 6 years ago
Intermittent Assertion failure: IsIdle(oldState), at /home/worker/workspace/build/src/xpcom/ds/PLDHashTable.h:132
Categories
(Toolkit :: Crash Reporting, defect, P1)
Toolkit
Crash Reporting
Tracking
()
RESOLVED
FIXED
mozilla62
People
(Reporter: aryx, Assigned: haik)
References
Details
(Keywords: assertion, intermittent-failure, regression)
Attachments
(1 file, 1 obsolete file)
https://treeherder.mozilla.org/logviewer.html#?job_id=87007910&repo=mozilla-inbound This in a tab crash submission test, so don't get confused. [task 2017-03-28T07:28:17.399046Z] 07:28:17 INFO - GECKO(1728) | XPCOM_MEM_BLOAT_LOG: /tmp/tmppqEaCG.mozrunner/runtests_leaks.log [task 2017-03-28T07:28:17.400792Z] 07:28:17 INFO - GECKO(1728) | Writing to log: /tmp/tmppqEaCG.mozrunner/runtests_leaks_tab_pid1879.log [task 2017-03-28T07:28:17.601478Z] 07:28:17 INFO - GECKO(1728) | --DOCSHELL 0xe94d9000 == 0 [pid = 1857] [id = {f91683fa-31f5-4f48-b96c-1b12f825b26f}] [task 2017-03-28T07:28:18.271318Z] 07:28:18 INFO - GECKO(1728) | Assertion failure: IsIdle(oldState), at /home/worker/workspace/build/src/xpcom/ds/PLDHashTable.h:132 [task 2017-03-28T07:28:18.273078Z] 07:28:18 INFO - GECKO(1728) | #01: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x4b59d4] [task 2017-03-28T07:28:18.274488Z] 07:28:18 INFO - GECKO(1728) | #02: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x4b5a0b] [task 2017-03-28T07:28:18.275875Z] 07:28:18 INFO - GECKO(1728) | #03: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x8ac5d0] [task 2017-03-28T07:28:18.279579Z] 07:28:18 INFO - GECKO(1728) | #04: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e30177] [task 2017-03-28T07:28:18.280990Z] 07:28:18 INFO - GECKO(1728) | #05: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x509d22] [task 2017-03-28T07:28:18.282390Z] 07:28:18 INFO - GECKO(1728) | #06: ???[/home/worker/workspace/build/application/firefox/libxul.so +0xc0c31a] [task 2017-03-28T07:28:18.284898Z] 07:28:18 INFO - GECKO(1728) | #07: ???[/home/worker/workspace/build/application/firefox/libxul.so +0xc0c4c7] [task 2017-03-28T07:28:18.289992Z] 07:28:18 INFO - GECKO(1728) | #08: ???[/home/worker/workspace/build/application/firefox/libxul.so +0xc12f94] [task 2017-03-28T07:28:18.292871Z] 07:28:18 INFO - GECKO(1728) | #09: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f714fa] [task 2017-03-28T07:28:18.295946Z] 07:28:18 INFO - GECKO(1728) | #10: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92a86] [task 2017-03-28T07:28:18.299202Z] 07:28:18 INFO - GECKO(1728) | #11: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.309171Z] 07:28:18 INFO - GECKO(1728) | #12: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f852bd] [task 2017-03-28T07:28:18.311292Z] 07:28:18 INFO - GECKO(1728) | #13: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92680] [task 2017-03-28T07:28:18.314501Z] 07:28:18 INFO - GECKO(1728) | Assertion failure: IsRead(oldState), at /home/worker/workspace/build/src/xpcom/ds/PLDHashTable.h:125 [task 2017-03-28T07:28:18.316706Z] 07:28:18 INFO - GECKO(1728) | #01: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x4b1280] [task 2017-03-28T07:28:18.328256Z] 07:28:18 INFO - GECKO(1728) | #02: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e2c6b8] [task 2017-03-28T07:28:18.331443Z] 07:28:18 INFO - GECKO(1728) | #03: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e2e4ec] [task 2017-03-28T07:28:18.335736Z] 07:28:18 INFO - GECKO(1728) | #04: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e2f5be] [task 2017-03-28T07:28:18.337831Z] 07:28:18 INFO - GECKO(1728) | #14: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92c2b] [task 2017-03-28T07:28:18.340059Z] 07:28:18 INFO - GECKO(1728) | #05: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e37a7b] [task 2017-03-28T07:28:18.342856Z] 07:28:18 INFO - GECKO(1728) | #06: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e37b30] [task 2017-03-28T07:28:18.353488Z] 07:28:18 INFO - GECKO(1728) | #07: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e37b66] [task 2017-03-28T07:28:18.355402Z] 07:28:18 INFO - GECKO(1728) | #08: ???[/lib/i386-linux-gnu/libpthread.so.0 +0x6d4c] [task 2017-03-28T07:28:18.359190Z] 07:28:18 INFO - GECKO(1728) | #15: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.364151Z] 07:28:18 INFO - GECKO(1728) | #16: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92fa7] [task 2017-03-28T07:28:18.367500Z] 07:28:18 INFO - GECKO(1728) | #17: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x34a85ce] [task 2017-03-28T07:28:18.369527Z] 07:28:18 INFO - GECKO(1728) | #18: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x34981f4] [task 2017-03-28T07:28:18.371539Z] 07:28:18 INFO - GECKO(1728) | #19: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x349e5b5] [task 2017-03-28T07:28:18.373787Z] 07:28:18 INFO - GECKO(1728) | #20: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x349e674] [task 2017-03-28T07:28:18.378023Z] 07:28:18 INFO - GECKO(1728) | #21: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f714fa] [task 2017-03-28T07:28:18.379954Z] 07:28:18 INFO - GECKO(1728) | #22: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92cc9] [task 2017-03-28T07:28:18.383590Z] 07:28:18 INFO - GECKO(1728) | #23: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.385515Z] 07:28:18 INFO - GECKO(1728) | #24: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f852bd] [task 2017-03-28T07:28:18.388451Z] 07:28:18 INFO - GECKO(1728) | #25: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92680] [task 2017-03-28T07:28:18.390455Z] 07:28:18 INFO - GECKO(1728) | #26: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92c2b] [task 2017-03-28T07:28:18.400129Z] 07:28:18 INFO - GECKO(1728) | #27: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.402034Z] 07:28:18 INFO - GECKO(1728) | #28: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f852bd] [task 2017-03-28T07:28:18.403906Z] 07:28:18 INFO - GECKO(1728) | #29: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92680] [task 2017-03-28T07:28:18.405770Z] 07:28:18 INFO - GECKO(1728) | #30: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92c2b] [task 2017-03-28T07:28:18.408883Z] 07:28:18 INFO - GECKO(1728) | #31: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.410784Z] 07:28:18 INFO - GECKO(1728) | #32: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92fa7] [task 2017-03-28T07:28:18.412690Z] 07:28:18 INFO - GECKO(1728) | #33: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x3419fea] [task 2017-03-28T07:28:18.414768Z] 07:28:18 INFO - GECKO(1728) | #34: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f714fa] [task 2017-03-28T07:28:18.418241Z] 07:28:18 INFO - GECKO(1728) | #35: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92a86] [task 2017-03-28T07:28:18.420146Z] 07:28:18 INFO - GECKO(1728) | #36: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.422088Z] 07:28:18 INFO - GECKO(1728) | #37: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92fa7] [task 2017-03-28T07:28:18.423997Z] 07:28:18 INFO - GECKO(1728) | #38: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x34a85ce] [task 2017-03-28T07:28:18.426000Z] 07:28:18 INFO - GECKO(1728) | #39: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x34981f4] [task 2017-03-28T07:28:18.429374Z] 07:28:18 INFO - GECKO(1728) | #40: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x349e5b5] [task 2017-03-28T07:28:18.431333Z] 07:28:18 INFO - GECKO(1728) | #41: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x349e674] [task 2017-03-28T07:28:18.435510Z] 07:28:18 INFO - GECKO(1728) | #42: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f714fa] [task 2017-03-28T07:28:18.437711Z] 07:28:18 INFO - GECKO(1728) | #43: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92cc9] [task 2017-03-28T07:28:18.439696Z] 07:28:18 INFO - GECKO(1728) | #44: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.441608Z] 07:28:18 INFO - GECKO(1728) | #45: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f852bd] [task 2017-03-28T07:28:18.443500Z] 07:28:18 INFO - GECKO(1728) | #46: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92680] [task 2017-03-28T07:28:18.445875Z] 07:28:18 INFO - GECKO(1728) | #47: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92c2b] [task 2017-03-28T07:28:18.448262Z] 07:28:18 INFO - GECKO(1728) | #48: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92edd] [task 2017-03-28T07:28:18.452084Z] 07:28:18 INFO - GECKO(1728) | #49: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2f92fa7] [task 2017-03-28T07:28:18.453998Z] 07:28:18 INFO - GECKO(1728) | #50: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x33c6b29] [task 2017-03-28T07:28:18.455871Z] 07:28:18 INFO - GECKO(1728) | #51: ???[/home/worker/workspace/build/application/firefox/libxul.so +0xc116b7] [task 2017-03-28T07:28:18.457780Z] 07:28:18 INFO - GECKO(1728) | #52: ???[/home/worker/workspace/build/application/firefox/libxul.so +0xc121fa] [task 2017-03-28T07:28:18.463595Z] 07:28:18 INFO - GECKO(1728) | #53: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x50a899] [task 2017-03-28T07:28:18.466140Z] 07:28:18 INFO - GECKO(1728) | #54: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x5234b5] [task 2017-03-28T07:28:18.468557Z] 07:28:18 INFO - GECKO(1728) | #55: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x51d65a] [task 2017-03-28T07:28:18.470898Z] 07:28:18 INFO - GECKO(1728) | #56: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x51d92f] [task 2017-03-28T07:28:18.474279Z] 07:28:18 INFO - GECKO(1728) | #57: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x51d98e] [task 2017-03-28T07:28:18.476215Z] 07:28:18 INFO - GECKO(1728) | #58: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x52038b] [task 2017-03-28T07:28:18.478544Z] 07:28:18 INFO - GECKO(1728) | #59: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x70a2d7] [task 2017-03-28T07:28:18.480568Z] 07:28:18 INFO - GECKO(1728) | #60: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x70187b] [task 2017-03-28T07:28:18.482643Z] 07:28:18 INFO - GECKO(1728) | #61: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x501a80] [task 2017-03-28T07:28:18.485029Z] 07:28:18 INFO - GECKO(1728) | #62: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x503d44] [task 2017-03-28T07:28:18.487135Z] 07:28:18 INFO - GECKO(1728) | #63: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x8c1928] [task 2017-03-28T07:28:18.490241Z] 07:28:18 INFO - GECKO(1728) | #64: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88eed6] [task 2017-03-28T07:28:18.492203Z] 07:28:18 INFO - GECKO(1728) | #65: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88eefb] [task 2017-03-28T07:28:18.494856Z] 07:28:18 INFO - GECKO(1728) | #66: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x1fd30bf] [task 2017-03-28T07:28:18.498774Z] 07:28:18 INFO - GECKO(1728) | #67: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2d9e054] [task 2017-03-28T07:28:18.500731Z] 07:28:18 INFO - GECKO(1728) | #68: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e1d72b] [task 2017-03-28T07:28:18.502630Z] 07:28:18 INFO - GECKO(1728) | #69: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e1de85] [task 2017-03-28T07:28:18.504502Z] 07:28:18 INFO - GECKO(1728) | #70: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x2e1e131] [task 2017-03-28T07:28:18.506452Z] 07:28:18 INFO - GECKO(1728) | #71: ???[/home/worker/workspace/build/application/firefox/firefox +0x551d] [task 2017-03-28T07:28:18.508543Z] 07:28:18 INFO - GECKO(1728) | #72: ???[/home/worker/workspace/build/application/firefox/firefox +0x4d3d] [task 2017-03-28T07:28:18.561829Z] 07:28:18 INFO - GECKO(1728) | ExceptionHandler::GenerateDump cloned child 1901 [task 2017-03-28T07:28:18.566934Z] 07:28:18 INFO - GECKO(1728) | ExceptionHandler::SendContinueSignalToChild sent continue signal to child [task 2017-03-28T07:28:18.571547Z] 07:28:18 INFO - GECKO(1728) | ExceptionHandler::WaitForContinueSignal waiting for continue signal... [task 2017-03-28T07:28:19.673233Z] 07:28:19 INFO - GECKO(1728) | [GFX1-]: Receive IPC close with reason=AbnormalShutdown [task 2017-03-28T07:28:19.681169Z] 07:28:19 INFO - GECKO(1728) | [GFX1-]: Receive IPC close with reason=AbnormalShutdown [task 2017-03-28T07:28:19.684580Z] 07:28:19 INFO - GECKO(1728) | [Child 1857] WARNING: pipe error (3): Connection reset by peer: file /home/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 346 [task 2017-03-28T07:28:19.694202Z] 07:28:19 INFO - GECKO(1728) | Hit MOZ_CRASH(Aborting on channel error.) at /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:2277 [task 2017-03-28T07:28:19.694600Z] 07:28:19 INFO - GECKO(1728) | #01: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x8bf513] [task 2017-03-28T07:28:19.701854Z] 07:28:19 INFO - GECKO(1728) | #02: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x8bf671] [task 2017-03-28T07:28:19.707602Z] 07:28:19 INFO - GECKO(1728) | #03: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x89c533] [task 2017-03-28T07:28:19.713404Z] 07:28:19 INFO - GECKO(1728) | #04: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88b10c] [task 2017-03-28T07:28:19.714804Z] 07:28:19 INFO - GECKO(1728) | #05: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x8a1776] [task 2017-03-28T07:28:19.717233Z] 07:28:19 INFO - GECKO(1728) | #06: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88da79] [task 2017-03-28T07:28:19.718648Z] 07:28:19 INFO - GECKO(1728) | #07: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88eed6] [task 2017-03-28T07:28:19.720576Z] 07:28:19 INFO - GECKO(1728) | #08: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88eefb] [task 2017-03-28T07:28:19.784575Z] 07:28:19 INFO - GECKO(1728) | #09: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x8997f7] [task 2017-03-28T07:28:19.799125Z] 07:28:19 INFO - GECKO(1728) | #10: ???[/home/worker/workspace/build/application/firefox/libxul.so +0x88ae3d] [task 2017-03-28T07:28:19.802593Z] 07:28:19 INFO - GECKO(1728) | #11: ???[/lib/i386-linux-gnu/libpthread.so.0 +0x6d4c] [task 2017-03-28T07:28:19.823803Z] 07:28:19 INFO - GECKO(1728) | [GFX1-]: Receive IPC close with reason=AbnormalShutdown
Comment 1•7 years ago
|
||
This is reproducible in bughunter on Windows.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 4•7 years ago
|
||
It's too bad the original stacks don't have any useful information in them...
Priority: -- → P3
Comment 5•7 years ago
|
||
Bughunter reproduces this assertion on Windows and Linux Beta/57, Nightly/58 on attachment 8909139 [details]. If you need, I can attach the crash reports and logs if reproducing on that attachment isn't sufficient.
Comment 6•7 years ago
|
||
Though, I think this was fixed in bug 1400701 where the test case is attached. This matches the last time orange factor flagged this bug and when bughunter last reproduced this as well. I think you might mark this fixed.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 15•6 years ago
|
||
It looks like these assertions are hitting inside CrashReporter::AnnotateCrashReport(). Maybe another thread is also trying to write to the hashtable? But there's a lock so I'm not sure how that would happen. 00:02:09 INFO - GECKO(5596) | Assertion failure: IsIdle(oldState), at z:\build\build\src\xpcom\ds\PLDHashTable.h:132 00:02:09 INFO - GECKO(5596) | Assertion failure: IsRead(oldState), at z:\build\build\src\xpcom\ds\PLDHashTable.h:125 00:02:09 INFO - GECKO(5596) | #01: Checker::StartWriteOp() [xpcom/ds/PLDHashTable.h:130] 00:02:09 INFO - GECKO(5596) | #02: nsBaseHashtable<nsCStringHashKey,nsTString<char>,nsTString<char> >::Put(nsTSubstring<char> const &,nsTString<char> const &,std::nothrow_t const &) [xpcom/ds/nsBaseHashtable.h:146] 00:02:09 INFO - GECKO(5596) | #03: nsBaseHashtable<nsCStringHashKey,nsTString<char>,nsTString<char> >::Put(nsTSubstring<char> const &,nsTString<char> const &) [xpcom/ds/nsBaseHashtable.h:137] 00:02:09 INFO - GECKO(5596) | #04: CrashReporter::AnnotateCrashReport(nsTSubstring<char> const &,nsTSubstring<char> const &) [toolkit/crashreporter/nsExceptionHandler.cpp:2148] 00:02:09 INFO - GECKO(5596) | #05: nsXULAppInfo::AnnotateCrashReport(nsTSubstring<char> const &,nsTSubstring<char> const &) [toolkit/xre/nsAppRunner.cpp:1340]
Component: XPCOM → Crash Reporting
Product: Core → Toolkit
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 18•6 years ago
|
||
For Marionette tests those assertions happen for the test test_crash.py, and especially when we trigger a crash of the content process. Haik, given that you fixed some issues lately maybe you are interested. Please see comment 15.
Flags: needinfo?(haftandilian)
Assignee | ||
Comment 19•6 years ago
|
||
Taking what mccr8 said in comment 15 a little further, the two asserts hit here https://treeherder.mozilla.org/logviewer.html#?job_id=169571172&repo=autoland&lineNumber=24779 00:02:09 INFO - GECKO(5596) | Assertion failure: IsIdle(oldState), at z:\build\build\src\xpcom\ds\PLDHashTable.h:132 00:02:09 INFO - GECKO(5596) | Assertion failure: IsRead(oldState), at z:\build\build\src\xpcom\ds\PLDHashTable.h:125 00:02:09 INFO - GECKO(5596) | #01: Checker::StartWriteOp() [xpcom/ds/PLDHashTable.h:130] 00:02:09 INFO - GECKO(5596) | #02: nsBaseHashtable<nsCStringHashKey,nsTString<char>,nsTString<char> >::Put(nsTSubstring<char> const &,nsTString<char> const &,std::nothrow_t const &) [xpcom/ds/nsBaseHashtable.h:146] are from 122 void EndReadOp() 123 { 124 uint32_t oldState = mState--; // this is an atomic decrement 125 MOZ_ASSERT(IsRead(oldState)); 126 } 127 128 void StartWriteOp() 129 { 130 MOZ_ASSERT(IsWritable()); 131 uint32_t oldState = mState.exchange(kWrite); 132 MOZ_ASSERT(IsIdle(oldState)); 133 } which implies (assuming this isn't memory corruption) we have a read/write race where another thread is doing a read of the hash table |crashReporterAPIData_Hash| without acquiring the lock |crashReporterAPILock| first. In AnnotateCrashReport(), where we hit the crash, we are holding crashReporterAPILock while we do a Put()/write, but there are other uses of crashReporterAPIData_Hash that don't acquire the lock. In GetAnnotation() we call crashReporterAPIData_Hash->Get() without the lock which looks like a bug. And in nsExceptionHandler.cpp - WriteExtraData() we iterate over the table (passed in via the |data| argument) without acquring the lock, but that needs more debugging to determine if the lock is needed. That looks like its called from breakpad in exception context where threads can be suspended and hence locks are problematic (see bug 1457501 for an example.) I'll take some time to look at the uses of crashReporterAPIData_Hash to try to determine if we're missing locking. Leaving ni for myself.
Assignee: nobody → haftandilian
Priority: P3 → P1
Comment hidden (Intermittent Failures Robot) |
Assignee | ||
Comment 21•6 years ago
|
||
I haven't reproduced the crash yet, but I think this patch is likely to address the problem. It addresses two cases where we access crashReporterAPIData_Hash without acquiring the required lock. Given that this is a crash test, it's probably the use of the hash table from WriteExtraForMinidump() that is causing the problem. I'll work on reproducing this crash and then trying to validate the fix.
Flags: needinfo?(haftandilian)
Assignee | ||
Comment 22•6 years ago
|
||
I've run toolkit/content/tests/browser/browser_crash_previous_frameloader.js for ~500 iterations on OS X and haven't been able to reproduce the crash so far. I'm going to move forward with posted fix which I've run for ~100 iterations of browser_crash_previous_frameloader.js and got a clean run on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7af65fdaa584666364ffe9b1747af3e1e05ac6ed
Comment hidden (mozreview-request) |
Assignee | ||
Updated•6 years ago
|
Attachment #8985387 -
Attachment is obsolete: true
Comment 24•6 years ago
|
||
mozreview-review |
Comment on attachment 8985684 [details] Bug 1351354 - Add missing lock protection to crashReporterAPIData_Hash https://reviewboard.mozilla.org/r/251216/#review257570 Nice find. Though every time we run into these kind of issues I can't help but think we should mercilessly refactor this code.
Attachment #8985684 -
Flags: review?(gsvelto) → review+
Comment 25•6 years ago
|
||
Pushed by haftandilian@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/578ef56130c9 Add missing lock protection to crashReporterAPIData_Hash r=gsvelto
Comment 26•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/578ef56130c9
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox62:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
Updated•6 years ago
|
status-firefox60:
--- → wontfix
status-firefox61:
--- → wontfix
status-firefox-esr52:
--- → wontfix
status-firefox-esr60:
--- → wontfix
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Updated•5 years ago
|
Keywords: regression
You need to log in
before you can comment on or make changes to this bug.
Description
•