Closed Bug 1479273 Opened 6 years ago Closed 3 years ago

Intermittent Reftest/XPCshell/Mochitest application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] for WebRender debug builds

Categories

(Core :: Graphics: WebRender, defect, P3)

Product:
Core
Component:
Graphics: WebRender
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: crash, intermittent-failure, Whiteboard: [comment 134/135])

Crash Data

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=190775157&repo=mozilla-central

https://queue.taskcluster.net/v1/task/RN_V5psVRU6s8RiqdlOeuQ/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/RN_V5psVRU6s8RiqdlOeuQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1&only_show_unexpected=1

11:15:50    ERROR -  REFTEST PROCESS-CRASH | file:///C:/Users/task_1532857122/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent.html | application crashed [@ nsThreadManager::ReleaseThread(void *)]
11:15:50     INFO -  Crash dump filename: c:\users\genericworker\appdata\local\temp\tmp6mo9oo.mozrunner\minidumps\6ebe326c-029d-4ba6-8215-662b5df4609e.dmp
11:15:50     INFO -  Operating system: Windows NT
11:15:50     INFO -                    10.0.15063
11:15:50     INFO -  CPU: amd64
11:15:50     INFO -       family 6 model 94 stepping 3
11:15:50     INFO -       8 CPUs
11:15:50     INFO -  GPU: UNKNOWN
11:15:50     INFO -  Crash reason:  EXCEPTION_ACCESS_VIOLATION_READ
11:15:50     INFO -  Crash address: 0xffffffffffffffff
11:15:50     INFO -  Assertion: Unknown assertion type 0x00000000
11:15:50     INFO -  Process uptime: 5 seconds
11:15:50     INFO -  Thread 7 (crashed)
11:15:50     INFO -   0  xul.dll!nsThreadManager::ReleaseThread(void *) [nsThreadManager.cpp:0be4463d29159905dded07f1dbddc5bb7dfaa336 : 116 + 0xb]
11:15:50     INFO -      rax = 0xe5e5e5e5e5e5e5e5   rdx = 0x0000000000000000
11:15:50     INFO -      rcx = 0x00000250dc10a160   rbx = 0x0000000000000004
11:15:50     INFO -      rsi = 0x00000250dc1a7800   rdi = 0x0000000000000080
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fae8
11:15:50     INFO -       r8 = 0x00000250dc11d400    r9 = 0x00007ff8d617cfd8
11:15:50     INFO -      r10 = 0x00000250dc11c020   r11 = 0x00000250dc1f8c80
11:15:50     INFO -      r12 = 0x00007ff8b816c3c0   r13 = 0x000000c4dadf4000
11:15:50     INFO -      r14 = 0x0000000000000004   r15 = 0x0000000000000000
11:15:50     INFO -      rip = 0x00007ff89e2a5113
11:15:50     INFO -      Found by: given as instruction pointer in context
11:15:50     INFO -   1  0x7ffe0385
11:15:50     INFO -      rbx = 0x0000000000000004   rbp = 0x00000250dc11d400
11:15:50     INFO -      rsp = 0x000000c4dc83fb18   r12 = 0x00007ff8b816c3c0
11:15:50     INFO -      r13 = 0x000000c4dadf4000   r14 = 0x0000000000000004
11:15:50     INFO -      r15 = 0x0000000000000000   rip = 0x000000007ffe0385
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   2  nss3.dll!PR_CleanupThread [prcthr.c:0be4463d29159905dded07f1dbddc5bb7dfaa336 : 30 + 0x5]
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fb40
11:15:50     INFO -      rip = 0x00007ff8b80899dd
11:15:50     INFO -      Found by: stack scanning
11:15:50     INFO -   3  nss3.dll!PRI_DetachThread [pruthr.c:0be4463d29159905dded07f1dbddc5bb7dfaa336 : 1482 + 0x8]
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fb70
11:15:50     INFO -      rip = 0x00007ff8b80888cc
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   4  nss3.dll!DllMain [w95dllmain.c:0be4463d29159905dded07f1dbddc5bb7dfaa336 : 32 + 0x5]
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fba0
11:15:50     INFO -      rip = 0x00007ff8b8073252
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   5  nss3.dll!static int dllmain_dispatch(struct HINSTANCE__ *, const unsigned long, void *) [dll_dllmain.cpp : 199 + 0xd]
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fbd0
11:15:50     INFO -      rip = 0x00007ff8b816c312
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   6  ntdll.dll!LdrpCallInitRoutine + 0x6b
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fc40
11:15:50     INFO -      rip = 0x00007ff8dc55486f
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   7  ntdll.dll!LdrShutdownThread + 0x151
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fcb0
11:15:50     INFO -      rip = 0x00007ff8dc551f01
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   8  ntdll.dll!RtlExitUserThread + 0x3e
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fdb0
11:15:50     INFO -      rip = 0x00007ff8dc56da2e
11:15:50     INFO -      Found by: call frame info
11:15:50     INFO -   9  kernel32.dll!LdrpLoadResourceFromAlternativeModule + 0x284
11:15:50     INFO -      rbp = 0x00000250dc11d400   rsp = 0x000000c4dc83fdf0
11:15:50     INFO -      rip = 0x00007ff8d9d4277c
11:15:50     INFO -      Found by: call frame info
Presumably the same as bug 1479253, but Kris can confirm.
Blocks: 1476405
Flags: needinfo?(kmaglione+bmo)
See Also: → 1479253
No longer blocks: 1476405
Depends on: 1479035
Flags: needinfo?(kmaglione+bmo)
Update: there are 72 total failures in the last 7 days:
- windows10-64-qr: 72
(the failures on other platforms from the above 2 orangefactor comments are missclassiications)


Emilio, Kris any updates here?
Flags: needinfo?(kmaglione+bmo)
Flags: needinfo?(emilio)
Not really from me, no... Kris is the person to ask, maybe Nathan as well?
Flags: needinfo?(emilio) → needinfo?(nfroyd)
(In reply to Emilio Cobos Álvarez (:emilio) from comment #13)
> Not really from me, no... Kris is the person to ask, maybe Nathan as well?

I think this is Kris's department, possibly fixed by the patch in bug 1479035 which I need to find mental space to review or pawn off on erahm or somebody.
Flags: needinfo?(nfroyd)
Crash Signature: [@ nsThreadManager::ReleaseThread(void *)] → [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()]
In the last 7 days, there are 155 failures on this bug.
They occur on windows10-64-qr debug.
Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=194233844&repo=autoland&lineNumber=14842

froydnj: Hello, Could you please have a look at this?
Component: Layout: Form Controls → XPCOM
Flags: needinfo?(nfroyd)
This should be fixed by bug 1479035. It's still waiting for review.
Flags: needinfo?(nfroyd)
Flags: needinfo?(kmaglione+bmo)
Backout by ebalazs@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/de10fbe0ee57
Backed out changeset bb2a3ca0fa4d for landing the patch with the wrong bug number. CLOSED TREE
Summary: Intermittent Reftests with application crashed [@ nsThreadManager::ReleaseThread(void *)] → Intermittent Reftests with application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()]
The failures here are caused by Bug 1476405 based on  https://tinyurl.com/y9mwjpn9

There is a total of 193 failures in the last 7 days, the majority on windows10-64-qr. 

Kris, please take a look at this as the failure rate is pretty high.
Flags: needinfo?(kmaglione+bmo)
There's nothing I can do. My patch is still awaiting review.
Flags: needinfo?(kmaglione+bmo)
Crash Signature: [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()] → [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()] [@ mozilla::ThreadEventTarget::Release()]
Crash Signature: [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()] [@ mozilla::ThreadEventTarget::Release()] → [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()]
Summary: Intermittent Reftests with application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] → Intermittent Reftest/XPCshell application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()]
(In reply to Intermittent Failures Robot from comment #71)
> 138 failures in 2902 pushes (0.048 failures/push) were associated with this
> bug in the last 7 days.
> 
> This is the #7 most frequent failure this week.

Kris, those failures seem to all depend on your patch on bug 1479035. Could you please have a look to finish it up, so that we can get rid of those failures? Thanks.
Flags: needinfo?(kmaglione+bmo)
(In reply to Henrik Skupin (:whimboo) from comment #72)
> (In reply to Intermittent Failures Robot from comment #71)
> > 138 failures in 2902 pushes (0.048 failures/push) were associated with this
> > bug in the last 7 days.
> > 
> > This is the #7 most frequent failure this week.
> 
> Kris, those failures seem to all depend on your patch on bug 1479035. Could
> you please have a look to finish it up, so that we can get rid of those
> failures? Thanks.

Henrik, i think that is just missing the checkin-needed tag, asked about it now and if so, i will land it tonight.
Kris, your patch on bug 1479035 didn't fix this particular problem. When I had a look at the OF failures those are all only happening for WebRender debug builds on Windows10 64.

Can you please have a look again? Thanks.
Summary: Intermittent Reftest/XPCshell application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] → Intermittent Reftest/XPCshell application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] for WebRender debug builds
See Also: → 1496276
Any updates on this?
Hi Kris, can you take a look at comment 78?
Flags: needinfo?(kmaglione+bmo)
Flags: needinfo?(kmaglione+bmo)
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Crash Signature: [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] → [@ nsThreadManager::ReleaseThread(void *)] [@ nsThreadManager::Shutdown()] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] [@ mozilla::LinkedListElement<nsThread>::removeFrom(mozilla::LinkedList<nsThread> con…
Summary: Intermittent Reftest/XPCshell application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] for WebRender debug builds → Intermittent Reftest/XPCshell/Mochitest application crashed [@ nsThreadManager::ReleaseThread(void *)] [@ mozilla::ThreadEventTarget::Release()] [@ mozilla::LinkedList<nsThread>::~LinkedList()] for WebRender debug builds

Moving these bugs (intermittent test failures with crashes) out of P5.

Priority: P5 → --
Crash Signature: const &)] → const &)] [@ nsThreadManager::ReleaseThread(void*)]
Crash Signature: const &)] [@ nsThreadManager::ReleaseThread(void*)] → const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const]

Hi Kris, this has reached the disable-recommended queue, is there any plan on a fix?

There are 152 total failures in the last 30 days and 68 total failures in the last 7 days on linux64-qr and windows10-64-qr, both debug builds.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=252973573&repo=autoland&lineNumber=18116

[task 2019-06-22T20:44:50.018Z] 20:44:50 INFO - [Child 7201, Main Thread] WARNING: NS_ENSURE_SUCCESS(mStatus, *this) failed with result 0x80004005: file /builds/worker/workspace/build/src/obj-firefox/dist/include/nsIURIMutator.h, line 489
[task 2019-06-22T20:44:50.027Z] 20:44:50 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent-ref.html
[task 2019-06-22T20:44:50.031Z] 20:44:50 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent.html | 28 / 29 (96%)
[task 2019-06-22T20:44:50.072Z] 20:44:50 INFO - ++DOMWINDOW == 82 (0x7efc6ec61800) [pid = 7201] [serial = 84] [outer = 0x7efc71075660]
[task 2019-06-22T20:44:50.422Z] 20:44:50 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent-ref.html | 28 / 29 (96%)
[task 2019-06-22T20:44:50.450Z] 20:44:50 INFO - ++DOMWINDOW == 83 (0x7efc6e3e7c00) [pid = 7201] [serial = 85] [outer = 0x7efc71075660]
[task 2019-06-22T20:44:50.539Z] 20:44:50 INFO - REFTEST TEST-PASS | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent-ref.html | image comparison, max difference: 0, number of differing pixels: 0
[task 2019-06-22T20:44:50.540Z] 20:44:50 INFO - REFTEST TEST-END | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-text-indent-ref.html
[task 2019-06-22T20:44:50.556Z] 20:44:50 INFO - ++DOMWINDOW == 84 (0x7efc6e3eac00) [pid = 7201] [serial = 86] [outer = 0x7efc71075660]
[task 2019-06-22T20:44:50.563Z] 20:44:50 INFO - [Child 7201, Main Thread] WARNING: NS_ENSURE_SUCCESS(mStatus, *this) failed with result 0x80004005: file /builds/worker/workspace/build/src/obj-firefox/dist/include/nsIURIMutator.h, line 489
[task 2019-06-22T20:44:50.584Z] 20:44:50 INFO - REFTEST INFO | Slowest test took 1705ms (file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html)
[task 2019-06-22T20:44:50.585Z] 20:44:50 INFO - REFTEST INFO | Total canvas count = 2

[task 2019-06-22T20:45:01.434Z] 20:45:01 INFO - REFTEST PROCESS-CRASH | Last test finished | application crashed [@ mozilla::ThreadEventTarget::Release()]
[task 2019-06-22T20:45:01.435Z] 20:45:01 INFO - Crash dump filename: /tmp/tmpKxmTRU.mozrunner/minidumps/0d5dd180-4aa5-0897-16e8-8749529f7007.dmp
[task 2019-06-22T20:45:01.436Z] 20:45:01 INFO - Operating system: Linux
[task 2019-06-22T20:45:01.438Z] 20:45:01 INFO - 0.0.0 Linux 4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018 x86_64
[task 2019-06-22T20:45:01.439Z] 20:45:01 INFO - CPU: amd64
[task 2019-06-22T20:45:01.441Z] 20:45:01 INFO - family 6 model 62 stepping 4
[task 2019-06-22T20:45:01.442Z] 20:45:01 INFO - 2 CPUs
[task 2019-06-22T20:45:01.444Z] 20:45:01 INFO -
[task 2019-06-22T20:45:01.445Z] 20:45:01 INFO - GPU: UNKNOWN
[task 2019-06-22T20:45:01.446Z] 20:45:01 INFO -
[task 2019-06-22T20:45:01.448Z] 20:45:01 INFO - Crash reason: SIGSEGV /0x00000080
[task 2019-06-22T20:45:01.449Z] 20:45:01 INFO - Crash address: 0x0
[task 2019-06-22T20:45:01.451Z] 20:45:01 INFO - Process uptime: not available
[task 2019-06-22T20:45:01.452Z] 20:45:01 INFO -
[task 2019-06-22T20:45:01.454Z] 20:45:01 INFO - Thread 1 (crashed)
[task 2019-06-22T20:45:01.455Z] 20:45:01 INFO - 0 libxul.so!mozilla::ThreadEventTarget::Release() [atomic_base.h:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 396 + 0x0]
[task 2019-06-22T20:45:01.457Z] 20:45:01 INFO - rax = 0x4512d186feb46600 rdx = 0x0000000000000000
[task 2019-06-22T20:45:01.458Z] 20:45:01 INFO - rcx = 0x4512d186feb46601 rbx = 0x00007f5267904070
[task 2019-06-22T20:45:01.460Z] 20:45:01 INFO - rsi = 0x00007f527328b87a rdi = 0xe5e5e5e5e5e5e5e5
[task 2019-06-22T20:45:01.461Z] 20:45:01 INFO - rbp = 0x00007f5267da3e00 rsp = 0x00007f5267da3df0
[task 2019-06-22T20:45:01.462Z] 20:45:01 INFO - r8 = 0x0000000000000000 r9 = 0x0000000000000200
[task 2019-06-22T20:45:01.464Z] 20:45:01 INFO - r10 = 0x00007f527ffdbc10 r11 = 0x00007f526dc547a8
[task 2019-06-22T20:45:01.465Z] 20:45:01 INFO - r12 = 0x0000000000000000 r13 = 0x00007f52814822c8
[task 2019-06-22T20:45:01.467Z] 20:45:01 INFO - r14 = 0x00007f5267904020 r15 = 0x00007f5267902040
[task 2019-06-22T20:45:01.468Z] 20:45:01 INFO - rip = 0x00007f526dcd5821
[task 2019-06-22T20:45:01.468Z] 20:45:01 INFO - Found by: given as instruction pointer in context
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - 1 libxul.so!nsThread::~nsThread() [nsThread.cpp:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 656 + 0x3f]
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - rbx = 0x00007f5267904070 rbp = 0x00007f5267da3e20
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - rsp = 0x00007f5267da3e10 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x00007f5267904020
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - r15 = 0x00007f5267902040 rip = 0x00007f526dcdd2a7
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - 2 libxul.so!nsThread::~nsThread() [nsThread.cpp:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 639 + 0xe]
[task 2019-06-22T20:45:01.469Z] 20:45:01 INFO - rbx = 0x00007f5267904020 rbp = 0x00007f5267da3e40
[task 2019-06-22T20:45:01.470Z] 20:45:01 INFO - rsp = 0x00007f5267da3e30 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.470Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x00007f5267904020
[task 2019-06-22T20:45:01.470Z] 20:45:01 INFO - r15 = 0x00007f5267902040 rip = 0x00007f526dcdd300
[task 2019-06-22T20:45:01.470Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.470Z] 20:45:01 INFO - 3 libxul.so!nsThread::Release() [nsThread.cpp:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 191 + 0x46]
[task 2019-06-22T20:45:01.470Z] 20:45:01 INFO - rbx = 0x0000000000000000 rbp = 0x00007f5267da3e60
[task 2019-06-22T20:45:01.471Z] 20:45:01 INFO - rsp = 0x00007f5267da3e50 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.471Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x00007f5267904020
[task 2019-06-22T20:45:01.471Z] 20:45:01 INFO - r15 = 0x00007f5267902040 rip = 0x00007f526dcdbd81
[task 2019-06-22T20:45:01.471Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.471Z] 20:45:01 INFO - 4 libnspr4.so!_PR_DestroyThreadPrivate [prtpd.c:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 237 + 0x3]
[task 2019-06-22T20:45:01.471Z] 20:45:01 INFO - rbx = 0x0000000000000003 rbp = 0x00007f5267da3e90
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - rsp = 0x00007f5267da3e70 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x0000000000000004
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - r15 = 0x00007f5267902040 rip = 0x00007f5281670372
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - 5 libnspr4.so!_pt_thread_death_internal [ptthread.c:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 855 + 0x8]
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - rbx = 0x00007f5267902040 rbp = 0x00007f5267da3eb0
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - rsp = 0x00007f5267da3ea0 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.472Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x0000000000000001
[task 2019-06-22T20:45:01.473Z] 20:45:01 INFO - r15 = 0x00007f5267da4c18 rip = 0x00007f528166bb68
[task 2019-06-22T20:45:01.473Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.473Z] 20:45:01 INFO - 6 libnspr4.so!_pt_thread_death [ptthread.c:11dc1c09bb580dfec90d93bfdf2fae386f670d0e : 828 + 0xd]
[task 2019-06-22T20:45:01.473Z] 20:45:01 INFO - rbx = 0x00007f5267902040 rbp = 0x00007f5267da3ed0
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - rsp = 0x00007f5267da3ec0 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x0000000000000004
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - r15 = 0x00007f5267da4c18 rip = 0x00007f528166bc70
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - 7 libpthread-2.23.so + 0x6439
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - rbx = 0x00007f5267da4a48 rbp = 0x00007f52814822f8
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - rsp = 0x00007f5267da3ee0 r12 = 0x0000000000000000
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - r13 = 0x00007f52814822c8 r14 = 0x0000000000000004
[task 2019-06-22T20:45:01.474Z] 20:45:01 INFO - r15 = 0x00007f5267da4c18 rip = 0x00007f5281270439
[task 2019-06-22T20:45:01.475Z] 20:45:01 INFO - Found by: call frame info
[task 2019-06-22T20:45:01.475Z] 20:45:01 INFO - 8 libpthread-2.23.so + 0x7870
[task 2019-06-22T20:45:01.475Z] 20:45:01 INFO - rbp = 0x00007f52814822f8 rsp = 0x00007f5267da3f20
[task 2019-06-22T20:45:01.475Z] 20:45:01 INFO - rip = 0x00007f5281271870
[task 2019-06-22T20:45:01.476Z] 20:45:01 INFO - Found by: stack scanning
[task 2019-06-22T20:45:01.476Z] 20:45:01 INFO - 9 libc-2.23.so + 0x10741d
[task 2019-06-22T20:45:01.476Z] 20:45:01 INFO - rbp = 0x00007f52814822f8 rsp = 0x00007f5267da3fc0
[task 2019-06-22T20:45:01.476Z] 20:45:01 INFO - rip = 0x00007f52802fa41d
[task 2019-06-22T20:45:01.476Z] 20:45:01 INFO - Found by: stack scanning

Joel any suggestion here?

Crash Signature: const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] → const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const]
Flags: needinfo?(kmaglione+bmo) → needinfo?(jmaher)
Flags: needinfo?(kmaglione+bmo)

I wonder if we had this in rr it would be solvable? The crash seems to move between different tests and directories of tests, not a simple root cause

:kats, this crash seems to be on -qr configs only, any concerns you have here?

Flags: needinfo?(jmaher) → needinfo?(kats)

Looks like a shutdown problem. The fact that it's only happening on -qr configs is concerning to me, yes. If we can get this in rr/pernosco it should be much easier to track down. Do you know what the process is these days for requesting a pernosco capture of this?

Flags: needinfo?(kats)

:bholley, could you help :kats get setup with instructions for how to get a pernosco capture?

Flags: needinfo?(bobbyholley)

There's already like 8 bugs on file for this issue. Nical seems to have already figured out what is going on in bug 1557208 comment 4.

Flags: needinfo?(kmaglione+bmo)
Flags: needinfo?(bobbyholley)
Component: XPCOM → Graphics: WebRender
See Also: → 1479743

(In reply to Andrew McCreight [:mccr8] from comment #129)

There's already like 8 bugs on file for this issue. Nical seems to have already figured out what is going on in bug 1557208 comment 4.

There are no updates there in the last 11 days, the bug does not have an assignee. Also, could you close the other 8 as duplicates to this one? or vice-versa?

This has 177 total failures in the last 30 days and 75 total failures in the last 7, reaching the disable-recommended queue. Since this occurs in various tests we cannot disable them.

Thanks.

Flags: needinfo?(continuation)

(In reply to Andreea Pavel [:apavel] from comment #131)

There are no updates there in the last 11 days, the bug does not have an assignee. Also, could you close the other 8 as duplicates to this one? or vice-versa?

I haven't been duping the intermittent failures unless the symptom is the same, to try to reduce the number of dupes that get filed.

This has 177 total failures in the last 30 days and 75 total failures in the last 7, reaching the disable-recommended queue. Since this occurs in various tests we cannot disable them.

Well, the failures are all in QR builds, so technically you could disable those. I guess we're shipping it, so that would be require closing the trees, which is not good.

Flags: needinfo?(continuation)

Thank you Andrew,

Joel how should we proceed here?

Flags: needinfo?(jmaher)

I needinfo'd nical in bug 1557208. If that doesn't go anywhere, you should find somebody else on the graphics team to escalate this, though it looks like jbonisteel is on PTO right now.

it sounds like we need to be patient here, the failures are across a wider spectrum than just "qr xpcshell", so disabling isn't ideal.

This is something we need to prioritize in engineering to get a fix for, there seem to be a small number of bugs in any month or two that seem to be a generic firefox issue and not a specific issue isolated to a test or specific feature.

:apavel, I assume we can let this sit until end of the week 27th or 28th, and then start needinfo'ing more people.

Flags: needinfo?(jmaher)

Okay, thank you both.

Whiteboard: [stockwell disable-recommended] → [comment 134/135]

Still work going on in https://bugzilla.mozilla.org/show_bug.cgi?id=1557208#c12.

Whiteboard: [comment 134/135][stockwell disable-recommended] → [comment 134/135]
Crash Signature: const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] → const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)]
Crash Signature: const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)] → const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)]
Whiteboard: [comment 134/135][stockwell disable-recommended] → [comment 134/135]

There are patches waiting to land in Bug 1557208. Will see if that fixes also this bug.

Whiteboard: [comment 134/135][stockwell disable-recommended] → [comment 134/135]
Crash Signature: const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)] → const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)] [@ nsTArray_Impl<RefPtr<nsThread>, nsTArrayInfallibleAllocator>::ClearAndRetainStor…

The priority flag is not set for this bug.
:jbonisteel, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(jbonisteel)
Crash Signature: , nsTArrayInfallibleAllocator>::ClearAndRetainStorage()] → , nsTArrayInfallibleAllocator>::ClearAndRetainStorage()] [@ nsThread::~nsThread()]
Crash Signature: const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)] [@ nsTArray_Impl<RefPtr<nsThread>, nsTArrayInfallibleAllocator>::ClearAndRetainStor… → const &)] [@ nsThreadManager::ReleaseThread(void*)] [@ mozilla::LinkedList<nsThread>::assertContains(nsThread*) const] [@ arena_dalloc(void*, unsigned long, arena_t*)] [@ nsTArray_Impl<RefPtr<nsThread>, nsTArrayInfallibleAllocator>::ClearAndRetainSto…
Flags: needinfo?(jbonisteel)
Priority: -- → P3
Crash Signature: , nsTArrayInfallibleAllocator>::ClearAndRetainStorage()] [@ nsThread::~nsThread()] → , nsTArrayInfallibleAllocator>::ClearAndRetainStorage()] [@ nsThread::~nsThread()] [@ gfxFontInfoLoader::FinalizeLoader(FontInfoData*)]

Bug 1557208 is fixed, let's see if the failure rate goes down after that.

Whiteboard: [comment 134/135][stockwell disable-recommended] → [comment 134/135]

Seems to be fixed. No occurrences in the last days.

Whiteboard: [comment 134/135][stockwell disable-recommended] → [comment 134/135]
Whiteboard: [comment 134/135][stockwell disable-recommended] → [comment 134/135]

Hello! I am closing this issue with RESOLVED->WORKSFORME since there where no changes made in the past 6 months.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.