Intermittent Linux crashtest TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code -11 or test_webvtt_disabled.html | application terminated with exit code -11 after Assertion failure: IsEmpty(), at xpcom/threads/nsEventQueue.cpp:35

RESOLVED DUPLICATE of bug 1240178

Status

()

P3
normal
RESOLVED DUPLICATE of bug 1240178
3 years ago
4 months ago

People

(Reporter: nigelb, Unassigned)

Tracking

({intermittent-failure})

Firefox Tracking Flags

(Not tracked)

Details

(URL)

(Reporter)

Description

3 years ago
https://treeherder.mozilla.org/logviewer.html#?job_id=20018826&repo=mozilla-inbound


 07:28:16     INFO -  --DOMWINDOW == 10 (0x7f05d0e72c00) [pid = 1106] [serial = 1636] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/483444.xml]

 07:28:16     INFO -  --DOMWINDOW == 9 (0x7f05d0f62000) [pid = 1106] [serial = 1638] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/485217.xml]

 07:28:16     INFO -  --DOMWINDOW == 8 (0x7f05d2962400) [pid = 1106] [serial = 1641] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/485286.xml]

 07:28:16     INFO -  --DOMWINDOW == 7 (0x7f05d0a77400) [pid = 1106] [serial = 1644] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/527558_1.xml]

 07:28:16     INFO -  --DOMWINDOW == 6 (0x7f05d0b27400) [pid = 1106] [serial = 1647] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/528300.xml]

 07:28:16     INFO -  --DOMWINDOW == 5 (0x7f05d0f42800) [pid = 1106] [serial = 1650] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/528488.xml]

 07:28:16     INFO -  --DOMWINDOW == 4 (0x7f05d2964800) [pid = 1106] [serial = 1653] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/528963.xml]

 07:28:16     INFO -  --DOMWINDOW == 3 (0x7f05d0a86000) [pid = 1106] [serial = 1665] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/667315.xml]

 07:28:16     INFO -  --DOMWINDOW == 2 (0x7f05d0b1c000) [pid = 1106] [serial = 1669] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/1205163.xml]

 07:28:16     INFO -  --DOMWINDOW == 1 (0x7f05d0b3e800) [pid = 1106] [serial = 1775] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/editor/composer/crashtests/428844-1-inner.xhtml]

 07:28:16     INFO -  --DOMWINDOW == 0 (0x7f05d0a76800) [pid = 1106] [serial = 1629] [outer = (nil)] [url = file:///home/worker/workspace/build/tests/reftest/tests/dom/xslt/crashtests/182460-select.xml]

 07:28:16     INFO -  Assertion failure: IsEmpty(), at /home/worker/workspace/build/src/xpcom/threads/nsEventQueue.cpp:35

 07:28:16     INFO -  #01: ??? (/home/worker/workspace/build/application/firefox/libxul.so)

 07:28:16     INFO -  #02: ??? (/home/worker/workspace/build/application/firefox/libxul.so)

 07:28:16     INFO -  #03: ??? (/home/worker/workspace/build/application/firefox/libxul.so)

 07:28:16     INFO -  #04: ??? (/home/worker/workspace/build/application/firefox/libxul.so)

 07:28:16     INFO -  #05: ??? (/home/worker/workspace/build/application/firefox/libnspr4.so)

 07:28:16     INFO -  #06: ??? (/home/worker/workspace/build/application/firefox/libnspr4.so)

 07:28:16     INFO -  #07: ??? (/home/worker/workspace/build/application/firefox/libnspr4.so)

 07:28:16     INFO -  #08: __nptl_deallocate_tsd (/lib/x86_64-linux-gnu/libpthread.so.0)

 07:28:16     INFO -  #09: start_thread (/lib/x86_64-linux-gnu/libpthread.so.0)

 07:28:16     INFO -  #10: clone (/lib/x86_64-linux-gnu/libc.so.6)

 07:28:16  WARNING -  TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code -139

 07:28:16     INFO -  TEST-INFO | leakcheck | default process: leak threshold set at 0 bytes

 07:28:16     INFO -  TEST-INFO | leakcheck | plugin process: leak threshold set at 0 bytes

 07:28:17     INFO -  TEST-INFO | leakcheck | tab process: leak threshold set at 5000 bytes

 07:28:17     INFO -  TEST-INFO | leakcheck | geckomediaplugin process: leak threshold set at 0 bytes

 07:28:17  WARNING -  TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!

 07:28:17     INFO -  TEST-INFO | leakcheck | missing output line from log file /tmp/tmpDOIz3M.mozrunner/runreftest_leaks.log

 07:28:17     INFO -  == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, plugin process 1439

 07:28:17     INFO -       |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|

 07:28:17     INFO -       |                                      | Per-Inst   Leaked|   Total      Rem|

 07:28:17     INFO -     0 |TOTAL                                 |       52        0|     563        0|

 07:28:17     INFO -  nsTraceRefcnt::DumpStatistics: 28 entries

 07:28:17     INFO -  TEST-PASS | leakcheck | plugin process: no leaks detected!

 07:28:17     INFO -  ==> process 1350 will purposefully crash

 07:28:17     INFO -  TEST-INFO | leakcheck | plugin process: deliberate crash and thus no leak log

 07:28:17     INFO -  == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, plugin process 1356

 07:28:17     INFO -       |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|

 07:28:17     INFO -       |                                      | Per-Inst   Leaked|   Total      Rem|

 07:28:17     INFO -     0 |TOTAL                                 |       47        0|     911        0|

 07:28:17     INFO -  nsTraceRefcnt::DumpStatistics: 30 entries

 07:28:17     INFO -  TEST-PASS | leakcheck | plugin process: no leaks detected!

 07:28:17     INFO -  REFTEST INFO | runreftest.py | Running tests: end.

 07:28:17    ERROR - Return code: 117

 07:28:17     INFO - TinderboxPrint: reftest-crashtest<br/>2966/0/52

 07:28:17  WARNING - # TBPL FAILURE #

 07:28:17  WARNING - setting return code to 2

 07:28:17  WARNING - The reftest suite: crashtest ran with return status: FAILURE

 07:28:17     INFO - Running post-action listener: _package_coverage_data

 07:28:17     INFO - Running post-action listener: _resource_record_post_action

 07:28:17     INFO - Running post-run listener: _resource_record_post_run

 07:28:18     INFO - Total resource usage - Wall time: 1541s; CPU: 96.0%; Read bytes: 171171840; Write bytes: 5605457920; Read time: 59544; Write time: 2430652

 07:28:18     INFO - install - Wall time: 27s; CPU: 100.0%; Read bytes: 0; Write bytes: 70139904; Read time: 0; Write time: 48032

 07:28:18     INFO - run-tests - Wall time: 1514s; CPU: 96.0%; Read bytes: 170598400; Write bytes: 5530320896; Read time: 59328; Write time: 2381032

 07:28:18     INFO - Running post-run listener: _upload_blobber_files

 07:28:18  WARNING - Blob upload gear skipped. Missing cmdline options.

 07:28:18     INFO - Running post-run listener: copy_logs_to_upload_dir

 07:28:18     INFO - Copying logs to upload dir...

 07:28:18     INFO - mkdir: /home/worker/workspace/build/upload/logs

 07:28:18     INFO - Copying logs to upload dir...

 07:28:18  WARNING - returning nonzero exit status 2
15 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 10
* fx-team: 5

Platform breakdown:
* linux64: 14
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-01-19&endday=2016-01-19&tree=all

Comment 2

3 years ago
Some event queue is being destroyed without being properly emptied. From the log it's very difficult to tell which process or thread this is. There are no symbols in the stack trace. Fortunately since this is Linux, an rr recording would help a lot. Does this happen enough that we could isolate it in rr?
Flags: needinfo?(nigelbabu)
Yes, you can - the relatively few number of instances starred as this bug just reflect the fact that it's tough to get a summary that will match what treeherder's looking for, since we stick a branch-specific path in assertion failure message, so nobody but Nigel knew he had filed it. If crashtest is too uncooperative about failing, mochitest-4 might work, or the fact that it's also failing in a media/ mochitest might be a clue about which crashtests to run-until-failure.
Flags: needinfo?(nigelbabu)
Summary: Intermittent Assertion failure: IsEmpty(), at xpcom/threads/nsEventQueue.cpp:35 → Intermittent Linux crashtest TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code -11 or test_webvtt_disabled.html | application terminated with exit code -11 after Assertion failure: IsEmpty(), at xpcom/threads/nsEventQueue.cpp:35
36 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 24
* fx-team: 6
* b2g-inbound: 4
* try: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 28
* linux32: 5
* b2g-emu-ics: 2
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-01-21&endday=2016-01-21&tree=all
32 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 22
* fx-team: 6
* mozilla-central: 3
* b2g-inbound: 1

Platform breakdown:
* linux64: 23
* osx-10-6: 3
* linux32: 3
* b2g-emu-ics: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-01-22&endday=2016-01-22&tree=all
Note: better assertion stack:
 10:14:29     INFO -  --DOMWINDOW == 0 (0x7f3c37c0f000) [pid = 1935] [serial = 1626] [outer = (nil)] [url = file:///builds/slave/test/build/tests/reftest/tests/dom/xslt/crashtests/182460-select.xml]
 10:14:29     INFO -  Assertion failure: IsEmpty(), at /builds/slave/m-in-l64-d-0000000000000000000/build/src/xpcom/threads/nsEventQueue.cpp:35
 10:14:29     INFO -  #01: nsThread::nsChainedEventQueue::~nsChainedEventQueue() [mfbt/RefPtr.h:55]
 10:14:29     INFO -  #02: nsThread::~nsThread() [xpcom/glue/nsTArray.h:2095]
 10:14:29     INFO -  #03: nsThread::~nsThread() [xpcom/threads/nsThread.cpp:508]
 10:14:29     INFO -  #04: nsThread::Release() [xpcom/threads/nsThread.cpp:179]
 10:14:29     INFO -  #05: _PR_DestroyThreadPrivate [nsprpub/pr/src/threads/prtpd.c:237]
 10:14:29     INFO -  #06: _pt_thread_death_internal [nsprpub/pr/src/pthreads/ptthread.c:888]
 10:14:29     INFO -  #07: _pt_thread_death [nsprpub/pr/src/pthreads/ptthread.c:865]
 10:14:29     INFO -  #08: libpthread.so.0 + 0x7c83
 10:14:29     INFO -  #09: libpthread.so.0 + 0x7ea8
 10:14:29     INFO -  #10: libc.so.6 + 0xf42ed
Flags: needinfo?(nfroyd)

Updated

3 years ago
Blocks: 1204784
21 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 11
* fx-team: 4
* try: 3
* mozilla-central: 3

Platform breakdown:
* linux64: 10
* linux32: 9
* b2g-emu-ics: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-01-23&endday=2016-01-23&tree=all
149 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 93
* fx-team: 28
* try: 11
* b2g-inbound: 10
* mozilla-central: 7

Platform breakdown:
* linux64: 105
* linux32: 30
* b2g-emu-ics: 8
* osx-10-6: 5
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-01-18&endday=2016-01-24&tree=all
khuey flipped this back to an NS_ASSERTION.  I'm not sure if that answers the ni?...
Flags: needinfo?(nfroyd)
6 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* fx-team: 3
* try: 1
* mozilla-aurora: 1
* b2g-inbound: 1

Platform breakdown:
* linux64: 4
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-01-25&endday=2016-01-31&tree=all
9 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 9

Platform breakdown:
* windows7-32: 9

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-06-06&endday=2016-06-12&tree=all
8 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 8

Platform breakdown:
* windows7-32: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-06-20&endday=2016-06-26&tree=all
10 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 10

Platform breakdown:
* windows7-32: 8
* windows8-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-06-27&endday=2016-07-03&tree=all
6 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 6

Platform breakdown:
* windows7-32: 5
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-07-04&endday=2016-07-10&tree=all
5 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 5

Platform breakdown:
* windows7-32: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1240672&startday=2016-07-11&endday=2016-07-17&tree=all

Comment 16

2 years ago
Bulk assigning P3 to all open intermittent bugs without a priority set in Firefox components per bug 1298978.
Priority: -- → P3
Status: NEW → RESOLVED
Last Resolved: 4 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1240178
You need to log in before you can comment on or make changes to this bug.