Open
Bug 1719400
Opened 3 years ago
Updated 3 years ago
Intermittent Fission OOM - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root] MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
Categories
(Core :: Graphics: Layers, defect, P3)
Tracking
()
NEW
| Fission Milestone | Future |
| Tracking | Status | |
|---|---|---|
| firefox-esr78 | --- | unaffected |
| firefox-esr91 | --- | unaffected |
| firefox90 | --- | unaffected |
| firefox91 | --- | unaffected |
| firefox92 | --- | affected |
| firefox93 | --- | affected |
| firefox94 | --- | affected |
| firefox95 | --- | affected |
People
(Reporter: intermittent-bug-filer, Unassigned, NeedInfo)
References
Details
(Keywords: crash, intermittent-failure, Whiteboard: [stockwell unknown])
Crash Data
Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=344528817&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YkdxiZaiRjSJGeN2gC4W9A/runs/0/artifacts/public/logs/live_backing.log
[task 2021-07-07T01:28:10.780Z] 01:28:10 ERROR - TEST-UNEXPECTED-FAIL | Last test finished | application terminated with exit code 1
[task 2021-07-07T01:28:10.781Z] 01:28:10 INFO - runtests.py | Application ran for: 0:18:43.947924
[task 2021-07-07T01:28:10.781Z] 01:28:10 INFO - zombiecheck | Reading PID log: /var/folders/9y/g98kq9jn4gg8dkkd4tldh4dh000014/T/tmpuheev59ipidlog
[task 2021-07-07T01:28:10.781Z] 01:28:10 INFO - ==> process 3985 launched child process 3986
[task 2021-07-07T01:28:10.782Z] 01:28:10 INFO - ==> process 3985 launched child process 3987
[task 2021-07-07T01:28:10.782Z] 01:28:10 INFO - ==> process 3985 launched child process 3988
[task 2021-07-07T01:28:10.782Z] 01:28:10 INFO - ==> process 3985 launched child process 3989
[task 2021-07-07T01:28:10.783Z] 01:28:10 INFO - ==> process 3985 launched child process 3990
[task 2021-07-07T01:28:10.783Z] 01:28:10 INFO - ==> process 3985 launched child process 3995
[task 2021-07-07T01:28:10.784Z] 01:28:10 INFO - ==> process 3985 launched child process 3996
[task 2021-07-07T01:28:10.784Z] 01:28:10 INFO - ==> process 3985 launched child process 3997
[task 2021-07-07T01:28:10.784Z] 01:28:10 INFO - ==> process 3985 launched child process 3998
[task 2021-07-07T01:28:10.784Z] 01:28:10 INFO - ==> process 3985 launched child process 3999
[task 2021-07-07T01:28:10.785Z] 01:28:10 INFO - ==> process 3985 launched child process 4000
[task 2021-07-07T01:28:10.785Z] 01:28:10 INFO - ==> process 3985 launched child process 4001
[task 2021-07-07T01:28:10.786Z] 01:28:10 INFO - ==> process 3985 launched child process 4002
[task 2021-07-07T01:28:10.786Z] 01:28:10 INFO - ==> process 3985 launched child process 4003
[task 2021-07-07T01:28:10.786Z] 01:28:10 INFO - ==> process 3985 launched child process 4004
[task 2021-07-07T01:28:10.787Z] 01:28:10 INFO - ==> process 3985 launched child process 4005
[task 2021-07-07T01:28:10.787Z] 01:28:10 INFO - ==> process 3985 launched child process 4006
[task 2021-07-07T01:28:10.787Z] 01:28:10 INFO - ==> process 3985 launched child process 4007
[task 2021-07-07T01:28:10.788Z] 01:28:10 INFO - ==> process 3985 launched child process 4008
[task 2021-07-07T01:28:10.788Z] 01:28:10 INFO - ==> process 3985 launched child process 4009
[task 2021-07-07T01:28:10.788Z] 01:28:10 INFO - ==> process 3985 launched child process 4010
[task 2021-07-07T01:28:10.788Z] 01:28:10 INFO - ==> process 3985 launched child process 4011
[task 2021-07-07T01:28:10.789Z] 01:28:10 INFO - ==> process 3985 launched child process 4012
[task 2021-07-07T01:28:10.789Z] 01:28:10 INFO - ==> process 3985 launched child process 4013
[task 2021-07-07T01:28:10.790Z] 01:28:10 INFO - ==> process 3985 launched child process 4018
[task 2021-07-07T01:28:10.790Z] 01:28:10 INFO - ==> process 3985 launched child process 4019
[task 2021-07-07T01:28:10.790Z] 01:28:10 INFO - ==> process 3985 launched child process 4020
[task 2021-07-07T01:28:10.790Z] 01:28:10 INFO - ==> process 3985 launched child process 4021
[task 2021-07-07T01:28:10.791Z] 01:28:10 INFO - ==> process 3985 launched child process 4022
[task 2021-07-07T01:28:10.791Z] 01:28:10 INFO - ==> process 3985 launched child process 4023
[task 2021-07-07T01:28:10.791Z] 01:28:10 INFO - ==> process 3985 launched child process 4051
[task 2021-07-07T01:28:10.792Z] 01:28:10 INFO - ==> process 3985 launched child process 4059
[task 2021-07-07T01:28:10.792Z] 01:28:10 INFO - ==> process 3985 launched child process 4068
[task 2021-07-07T01:28:10.792Z] 01:28:10 INFO - ==> process 3985 launched child process 4077
[task 2021-07-07T01:28:10.793Z] 01:28:10 INFO - ==> process 3985 launched child process 4096
[task 2021-07-07T01:28:10.793Z] 01:28:10 INFO - ==> process 3985 launched child process 4101
[task 2021-07-07T01:28:10.793Z] 01:28:10 INFO - ==> process 3985 launched child process 4106
[task 2021-07-07T01:28:10.793Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4096
[task 2021-07-07T01:28:10.794Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4101
[task 2021-07-07T01:28:10.794Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4106
[task 2021-07-07T01:28:10.795Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3986
[task 2021-07-07T01:28:10.795Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3987
[task 2021-07-07T01:28:10.795Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3988
[task 2021-07-07T01:28:10.796Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3989
[task 2021-07-07T01:28:10.796Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3990
[task 2021-07-07T01:28:10.796Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3995
[task 2021-07-07T01:28:10.796Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3996
[task 2021-07-07T01:28:10.797Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3997
[task 2021-07-07T01:28:10.797Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3998
[task 2021-07-07T01:28:10.797Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 3999
[task 2021-07-07T01:28:10.797Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4000
[task 2021-07-07T01:28:10.798Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4001
[task 2021-07-07T01:28:10.798Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4002
[task 2021-07-07T01:28:10.798Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4003
[task 2021-07-07T01:28:10.798Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4004
[task 2021-07-07T01:28:10.799Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4005
[task 2021-07-07T01:28:10.799Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4006
[task 2021-07-07T01:28:10.799Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4007
[task 2021-07-07T01:28:10.800Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4008
[task 2021-07-07T01:28:10.800Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4009
[task 2021-07-07T01:28:10.800Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4010
[task 2021-07-07T01:28:10.800Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4011
[task 2021-07-07T01:28:10.801Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4012
[task 2021-07-07T01:28:10.801Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4013
[task 2021-07-07T01:28:10.801Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4018
[task 2021-07-07T01:28:10.801Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4019
[task 2021-07-07T01:28:10.802Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4020
[task 2021-07-07T01:28:10.802Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4021
[task 2021-07-07T01:28:10.802Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4022
[task 2021-07-07T01:28:10.803Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4023
[task 2021-07-07T01:28:10.803Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4051
[task 2021-07-07T01:28:10.803Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4059
[task 2021-07-07T01:28:10.803Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4068
[task 2021-07-07T01:28:10.804Z] 01:28:10 INFO - zombiecheck | Checking for orphan process with PID: 4077
[task 2021-07-07T01:28:10.804Z] 01:28:10 INFO - mozcrash Copy/paste: /opt/worker/tasks/task_162561859219897/fetches/minidump_stackwalk/minidump_stackwalk /var/folders/9y/g98kq9jn4gg8dkkd4tldh4dh000014/T/tmpdepyhnwx.mozrunner/minidumps/44574F58-5128-4E1D-AF37-4FE517E1E674.dmp /opt/worker/tasks/task_162561859219897/build/symbols
[task 2021-07-07T01:28:17.249Z] 01:28:17 INFO - mozcrash Saved minidump as /opt/worker/tasks/task_162561859219897/build/blobber_upload_dir/44574F58-5128-4E1D-AF37-4FE517E1E674.dmp
[task 2021-07-07T01:28:17.250Z] 01:28:17 INFO - mozcrash Saved app info as /opt/worker/tasks/task_162561859219897/build/blobber_upload_dir/44574F58-5128-4E1D-AF37-4FE517E1E674.extra
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root]
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Mozilla crash reason: MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Crash dump filename: /var/folders/9y/g98kq9jn4gg8dkkd4tldh4dh000014/T/tmpdepyhnwx.mozrunner/minidumps/44574F58-5128-4E1D-AF37-4FE517E1E674.dmp
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Operating system: Mac OS X
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - 10.15.7 19H524
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - CPU: amd64
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - family 6 model 158 stepping 10
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - 12 CPUs
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO -
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - GPU: UNKNOWN
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO -
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Crash reason: EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Crash address: 0x0
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Process uptime: 288 seconds
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO -
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - Thread 48 (crashed) - Shutdown Hang Terminator 0 XUL!mozilla::(anonymous namespace)::RunWatchdog(void*) [nsTerminator.cpp:e806428569824e8b75664663ad5289ad5c6a3c31 : 246 + 0x22]
[task 2021-07-07T01:28:17.450Z] 01:28:17 INFO - rax = 0x000000010dea3934 rdx = 0x0000000000000000
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - rcx = 0x0000000100ac63d8 rbx = 0x0000000000000276
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - rsi = 0x00000000000120a8 rdi = 0x00007fff96522ca8
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - rbp = 0x000070000cdc3f70 rsp = 0x000070000cdc3f50
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - r8 = 0x00000000000130a8 r9 = 0x0000000000000000
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - r10 = 0x00007fff96522cc8 r11 = 0x00007fff96522cc0
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - r12 = 0x00000001194481b0 r13 = 0x0000000000000000
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - r14 = 0x0000000100d0f378 r15 = 0x0000000000000007
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - rip = 0x00000001098cf96c
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - Found by: given as instruction pointer in context
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - 1 libnss3.dylib!_pt_root [ptthread.c:e806428569824e8b75664663ad5289ad5c6a3c31 : 201 + 0xa]
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - rbx = 0x0000000000000276 rbp = 0x000070000cdc3fb0
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - rsp = 0x000070000cdc3f80 r12 = 0x00000001194481b0
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - r15 = 0x0000000000000007 rip = 0x0000000100f883c9
[task 2021-07-07T01:28:17.451Z] 01:28:17 INFO - Found by: call frame info
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - 2 libsystem_pthread.dylib!_pthread_start + 0x94
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - rbx = 0x6e776f6474756853 rbp = 0x000070000cdc3fd0
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - rsp = 0x000070000cdc3fc0 r12 = 0x00000001194481b0
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - r15 = 0x0000000a0000001f rip = 0x00007fff6fe7b109
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - Found by: call frame info
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - 3 libsystem_pthread.dylib!thread_start + 0xf
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - rbx = 0x6e776f6474756853 rbp = 0x000070000cdc3ff0
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - rsp = 0x000070000cdc3fe0 r12 = 0x0000000000000000
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - r15 = 0x0000000a0000001f rip = 0x00007fff6fe76b8b
[task 2021-07-07T01:28:17.452Z] 01:28:17 INFO - Found by: call frame info
|
||
Comment 1•3 years ago
|
||
Reported after toolkit/components/url-classifier/tests/mochitest/test_socialtracking_annotate.html
|
||
Updated•3 years ago
|
Component: Safe Browsing → IPC
Product: Toolkit → Core
Summary: Intermittent PROCESS-CRASH | Last test finished | application crashed [@ _pt_root] → Intermittent Fission OOM - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root]
|
||
Comment 2•3 years ago
|
||
Component: IPC → Graphics: Layers
|
||
Comment 3•3 years ago
|
||
There are 39 total failures in the last 7 days on
- macosx1015-64-qr debug
- linux1804-64-qr debug
- linux1804-64 debug
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=344881072&repo=mozilla-central&lineNumber=54158
[task 2021-07-11T10:41:31.612Z] 10:41:31 INFO - TEST-START | toolkit/components/url-classifier/tests/mochitest/test_socialtracking_annotate.html
[task 2021-07-11T10:41:31.636Z] 10:41:31 INFO - GECKO(12482) | [Child 12566, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3357
[task 2021-07-11T10:41:31.647Z] 10:41:31 INFO - GECKO(12482) | [Child 12566, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4438
[task 2021-07-11T10:41:35.031Z] 10:41:35 INFO - GECKO(12482) | [Parent 12482, Main Thread] WARNING: WebProgress Ignored: no longer current window global: file /builds/worker/checkouts/gecko/dom/ipc/BrowserParent.cpp:2928
[task 2021-07-11T10:41:36.200Z] 10:41:36 INFO - GECKO(12482) | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to /var/folders/dv/hnm8h5w55bv3mzsfk0_zk888000014/T/tmpybstp5m1.mozrunner/runtests_leaks_tab_pid12605.log
[task 2021-07-11T10:41:36.201Z] 10:41:36 INFO - GECKO(12482) | [12605, Main Thread] WARNING: XPCOM_MEM_BLOAT_LOG is set, disabling native allocations.: file /builds/worker/checkouts/gecko/tools/profiler/core/platform.cpp:248
[task 2021-07-11T10:41:38.887Z] 10:41:38 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-07-11T10:41:38.887Z] 10:41:38 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: '!ipcActor->SendAccumulateChildHistograms(histogramsToSend)', file /builds/worker/checkouts/gecko/toolkit/components/telemetry/core/ipc/TelemetryIPCAccumulator.cpp:279
[task 2021-07-11T10:41:38.888Z] 10:41:38 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-07-11T10:41:38.888Z] 10:41:38 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: '!ipcActor->SendRecordDiscardedData(discardedData)', file /builds/worker/checkouts/gecko/toolkit/components/telemetry/core/ipc/TelemetryIPCAccumulator.cpp:298
[task 2021-07-11T10:41:48.874Z] 10:41:48 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-07-11T10:41:48.874Z] 10:41:48 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: '!ipcActor->SendAccumulateChildHistograms(histogramsToSend)', file /builds/worker/checkouts/gecko/toolkit/components/telemetry/core/ipc/TelemetryIPCAccumulator.cpp:279
[task 2021-07-11T10:41:48.875Z] 10:41:48 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-07-11T10:41:48.875Z] 10:41:48 INFO - GECKO(12482) | [Child 12500, Main Thread] WARNING: '!ipcActor->SendRecordDiscardedData(discardedData)', file /builds/worker/checkouts/gecko/toolkit/components/telemetry/core/ipc/TelemetryIPCAccumulator.cpp:298
[task 2021-07-11T10:41:52.647Z] 10:41:52 INFO - GECKO(12482) | MEMORY STAT | vsize 6631MB | residentFast 86MB | heapAllocated 7MB
[task 2021-07-11T10:41:55.411Z] 10:41:55 INFO - TEST-OK | toolkit/components/url-classifier/tests/mochitest/test_socialtracking_annotate.html | took 23798ms
[task 2021-07-11T10:41:58.748Z] 10:41:58 INFO - Error: Unable to restore focus, expect failures and timeouts.
[task 2021-07-11T10:41:58.755Z] 10:41:58 INFO - GECKO(12482) | [Parent 12482, Main Thread] WARNING: Receivedloadevent on unbridged BrowserParent!: file /builds/worker/checkouts/gecko/dom/ipc/BrowserParent.cpp:3898
[task 2021-07-11T10:41:58.756Z] 10:41:58 INFO - GECKO(12482) | [Parent 12482, Main Thread] WARNING: Receivedloadevent on unbridged BrowserParent!: file /builds/worker/checkouts/gecko/dom/ipc/BrowserParent.cpp:3898
[task 2021-07-11T10:41:58.759Z] 10:41:58 INFO - GECKO(12482) | [Child 12575, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/checkouts/gecko/widget/cocoa/nsAppShell.mm:764
[task 2021-07-11T10:41:58.774Z] 10:41:58 INFO - TEST-START | Shutdown
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root]
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Mozilla crash reason: MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Crash dump filename: /var/folders/dv/hnm8h5w55bv3mzsfk0_zk888000014/T/tmpybstp5m1.mozrunner/minidumps/D3A279EB-E43C-415D-9B28-8D583B7BE7E1.dmp
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Operating system: Mac OS X
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - 10.15.7 19H524
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - CPU: amd64
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - family 6 model 158 stepping 10
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - 12 CPUs
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO -
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - GPU: UNKNOWN
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO -
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Crash reason: EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Crash address: 0x0
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Process uptime: 291 seconds
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO -
[task 2021-07-11T10:58:59.738Z] 10:58:59 INFO - Thread 48 (crashed) - Shutdown Hang Terminator 0 XUL!mozilla::(anonymous namespace)::RunWatchdog(void*) [nsTerminator.cpp:e5591aaa2cdc8ace99686b1e09bcd5d4c7c39fa6 : 246 + 0x22]
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rax = 0x000000011cbee924 rdx = 0x0000000000000000
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rcx = 0x0000000106e5c3d8 rbx = 0x0000000000000276
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rsi = 0x00000000000120a8 rdi = 0x00007fff8e86aca8
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rbp = 0x0000700006c55f70 rsp = 0x0000700006c55f50
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r8 = 0x00000000000130a8 r9 = 0x0000000000000000
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r10 = 0x00007fff8e86acc8 r11 = 0x00007fff8e86acc0
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r12 = 0x0000000123fe9d90 r13 = 0x0000000000000000
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r14 = 0x000000010700f450 r15 = 0x0000000000000007
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rip = 0x000000011861936c
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - Found by: given as instruction pointer in context
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - 1 libnss3.dylib!_pt_root [ptthread.c:e5591aaa2cdc8ace99686b1e09bcd5d4c7c39fa6 : 201 + 0xa]
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rbx = 0x0000000000000276 rbp = 0x0000700006c55fb0
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rsp = 0x0000700006c55f80 r12 = 0x0000000123fe9d90
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r15 = 0x0000000000000007 rip = 0x00000001072883c9
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - Found by: call frame info
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - 2 libsystem_pthread.dylib!_pthread_start + 0x94
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rbx = 0x6e776f6474756853 rbp = 0x0000700006c55fd0
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - rsp = 0x0000700006c55fc0 r12 = 0x0000000123fe9d90
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-07-11T10:58:59.739Z] 10:58:59 INFO - r15 = 0x0000000a0000001f rip = 0x00007fff681c3109
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - Found by: call frame info
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - 3 libsystem_pthread.dylib!thread_start + 0xf
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - rbx = 0x6e776f6474756853 rbp = 0x0000700006c55ff0
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - rsp = 0x0000700006c55fe0 r12 = 0x0000000000000000
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - r15 = 0x0000000a0000001f rip = 0x00007fff681beb8b
[task 2021-07-11T10:58:59.740Z] 10:58:59 INFO - Found by: call frame info
Jim, can you please assign someone to take a look?
Flags: needinfo?(jmathies)
Whiteboard: [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 5•3 years ago
|
||
@ Nical, could these test hangs be caused by opaque pass bug 1717592, which landed on July 5?
This is a watchdog timeout after a test hang. The first test failures were on July 6. Did some big graphics changes land on July 5 or 6?
Here is the changelog from July 5 through July 6:
Linux semi-headless mode on July 6, but I don't think it's related to this regression because we see these test failures on both Linux and macOS.
Fission Milestone: --- → Future
status-firefox90:
--- → unaffected
status-firefox91:
--- → unaffected
status-firefox92:
--- → affected
status-firefox-esr78:
--- → unaffected
Flags: needinfo?(nical.bugzilla)
|
||
Comment 6•3 years ago
|
||
@ Nical, could these test hangs be caused by opaque pass bug 1717592, which landed on July 5?
No that patch only changed some details about how some things are drawn.
The CompositorThreadHolder::Shutdown stuff spins the event loop until some shutdown logic happening asynchronously in other threads is complete. The other threads all appear to be sitting idle in their event loops (no deadlock as far as I can see) so something is holding to a RefPtr<CompositorThreadHolder> (it's when the last one goes away that we let the rest of the main thread's shutdown continue).
Flags: needinfo?(nical.bugzilla)
|
||
Comment 7•3 years ago
|
||
Update:
There have been 37 failures within the last 7 days:
• 17 failures on Linux 18.04 x64 debug
• 5 failures on Linux 18.04 x64 WebRender debug
• 15 failures on OS X 10.15 WebRender debug
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=345587870&repo=autoland&lineNumber=4550
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 11•3 years ago
|
||
@ Emilio, do you think your PresShell active changes in bug 1717983 might cause intermittent compositor shutdown hangs in Fission tests on Linux and macOS? Nical's comment 6 has more details about the hanging threads.
100% of these test hangs are Fission tests.
The first failing changeset was e806428569824e8b75664663ad5289ad5c6a3c31, which hit five test hangs on autoland on 2021-07-06, but those changes don't look suspicious because they are related to Pocket and the New Tab:
Here is the mozilla-central pushlog that contains changeset e806428569824e8b75664663ad5289ad5c6a3c31 plus a few pushes before it:
Flags: needinfo?(emilio)
OS: Unspecified → All
Summary: Intermittent Fission OOM - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root] → Intermittent Fission OOM - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root] MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
|
||
Comment 12•3 years ago
|
||
I don't think so, my patch only propagates activeness to in-process subdocuments properly, and they don't keep the compositor around in any way that the parent document wouldn't afaict.
Flags: needinfo?(emilio)
|
|
||
Comment 13•3 years ago
|
||
Update:
There have been 38 failures within the last 7 days:
• 5 failures on Linux 18.04 x64 debug
• 23 failures on Linux 18.04 x64 WebRender debug
• 10 failures on OS X 10.15 WebRender debug
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=347009473&repo=mozilla-central&lineNumber=13408
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 15•3 years ago
•
|
||
Update:
There have been 39 failures within the last 7 days:
• 34 failures on Linux 18.04 x64 WebRender debug
• 5 failures on OS X 10.15 WebRender debug
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=347752013&repo=autoland&lineNumber=30807
<...>
[task 2021-08-07T17:31:04.031Z] 17:31:04 INFO - GECKO(15806) | [Child 16823, Main Thread] WARNING: '!ipcActor->SendRecordDiscardedData(discardedData)', file /builds/worker/checkouts/gecko/toolkit/components/telemetry/core/ipc/TelemetryIPCAccumulator.cpp:298
[task 2021-08-07T17:31:13.925Z] 17:31:13 INFO - GECKO(15806) | out of memory: 0x0000000028700000 bytes requested
[task 2021-08-07T17:31:13.925Z] 17:31:13 INFO - GECKO(15806) | Hit MOZ_CRASH() at /builds/worker/checkouts/gecko/memory/mozalloc/mozalloc_abort.cpp:33
[task 2021-08-07T17:31:14.009Z] 17:31:14 INFO - GECKO(15806) | #01: mozalloc_abort [memory/mozalloc/mozalloc_abort.cpp:33]
[task 2021-08-07T17:31:14.011Z] 17:31:14 INFO - GECKO(15806) | #02: mozalloc_handle_oom(unsigned long) [memory/mozalloc/mozalloc_oom.cpp:51]
[task 2021-08-07T17:31:14.012Z] 17:31:14 INFO - GECKO(15806) | #03: moz_xstrdup [memory/mozalloc/mozalloc.cpp:80]
[task 2021-08-07T17:31:14.012Z] 17:31:14 INFO - GECKO(15806) | #04: nsTArrayInfallibleAllocator::ResultTypeProxy nsTArray_base<nsTArrayInfallibleAllocator, nsTArray_RelocateUsingMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator>(unsigned long, unsigned long) [xpcom/ds/nsTArray-inl.h:223]
[task 2021-08-07T17:31:14.014Z] 17:31:14 INFO - GECKO(15806) | #05: mozilla::net::ChannelEventQueue::RunOrEnqueue(mozilla::net::ChannelEvent*, bool) [netwerk/ipc/ChannelEventQueue.h:218]
[task 2021-08-07T17:31:14.014Z] 17:31:14 INFO - GECKO(15806) | #06: mozilla::net::HttpChannelChild::DoNotifyListener() [netwerk/protocol/http/HttpChannelChild.cpp:1213]
[task 2021-08-07T17:31:14.014Z] 17:31:14 INFO - GECKO(15806) | #07: mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::net::HttpChannelChild>, void (mozilla::net::HttpChannelChild::*)(), true, (mozilla::RunnableKind)0, >::Run() [xpcom/threads/nsThreadUtils.h:1204]
[task 2021-08-07T17:31:14.015Z] 17:31:14 INFO - GECKO(15806) | #08: mozilla::RunnableTask::Run() [xpcom/threads/TaskController.cpp:503]
[task 2021-08-07T17:31:14.016Z] 17:31:14 INFO - GECKO(15806) | #09: mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [xpcom/threads/TaskController.cpp:805]
[task 2021-08-07T17:31:14.017Z] 17:31:14 INFO - GECKO(15806) | #10: mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [xpcom/threads/TaskController.cpp:641]
[task 2021-08-07T17:31:14.018Z] 17:31:14 INFO - GECKO(15806) | #11: mozilla::TaskController::ProcessPendingMTTask(bool) [xpcom/threads/TaskController.cpp:425]
[task 2021-08-07T17:31:14.020Z] 17:31:14 INFO - GECKO(15806) | #12: mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_0>::Run() [xpcom/threads/nsThreadUtils.h:533]
[task 2021-08-07T17:31:14.021Z] 17:31:14 INFO - GECKO(15806) | #13: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1152]
[task 2021-08-07T17:31:14.022Z] 17:31:14 INFO - GECKO(15806) | #14: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:466]
[task 2021-08-07T17:31:14.023Z] 17:31:14 INFO - GECKO(15806) | #15: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:85]
[task 2021-08-07T17:31:14.024Z] 17:31:14 INFO - GECKO(15806) | #16: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:331]
[task 2021-08-07T17:31:14.026Z] 17:31:14 INFO - GECKO(15806) | #17: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
[task 2021-08-07T17:31:14.027Z] 17:31:14 INFO - GECKO(15806) | #18: nsBaseAppShell::Run() [widget/nsBaseAppShell.cpp:139]
[task 2021-08-07T17:31:14.028Z] 17:31:14 INFO - GECKO(15806) | #19: XRE_RunAppShell() [toolkit/xre/nsEmbedFunctions.cpp:917]
[task 2021-08-07T17:31:14.030Z] 17:31:14 INFO - GECKO(15806) | #20: mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:235]
[task 2021-08-07T17:31:14.031Z] 17:31:14 INFO - GECKO(15806) | #21: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:331]
[task 2021-08-07T17:31:14.032Z] 17:31:14 INFO - GECKO(15806) | #22: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
[task 2021-08-07T17:31:14.034Z] 17:31:14 INFO - GECKO(15806) | #23: XRE_InitChildProcess(int, char**, XREChildData const*) [toolkit/xre/nsEmbedFunctions.cpp:749]
[task 2021-08-07T17:31:14.036Z] 17:31:14 INFO - GECKO(15806) | #24: content_process_main(mozilla::Bootstrap*, int, char**) [ipc/contentproc/plugin-container.cpp:58]
[task 2021-08-07T17:31:14.037Z] 17:31:14 INFO - GECKO(15806) | #25: main [browser/app/nsBrowserApp.cpp:327]
[task 2021-08-07T17:31:14.038Z] 17:31:14 INFO - GECKO(15806) | #26: __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6 + 0x21b97]
[task 2021-08-07T17:31:14.039Z] 17:31:14 INFO - GECKO(15806) | #27: ??? [/builds/worker/workspace/build/application/firefox/firefox-bin + 0x413b9]
[task 2021-08-07T17:31:14.040Z] 17:31:14 INFO - GECKO(15806) | #28: ??? (???:???)
[task 2021-08-07T17:31:14.277Z] 17:31:14 INFO - TEST-INFO | Main app process: exit 11
[task 2021-08-07T17:31:14.278Z] 17:31:14 INFO - TEST-INFO | Confirming we saw 110 DOCSHELL created and 109 destroyed log strings.
[task 2021-08-07T17:31:14.279Z] 17:31:14 INFO - TEST-INFO | Confirming we saw 298 DOMWINDOW created and 286 destroyed log strings.
[task 2021-08-07T17:31:14.279Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_bug1601563.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.280Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_bug399606.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.281Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_bug461710.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.282Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_bug646422.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.283Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_bug680727.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.283Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_double_redirect.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.284Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_favicon_privatebrowsing_perwindowpb.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.288Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_history_post.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.288Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_multi_redirect_frecency.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.288Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_notfound.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.289Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_onvisit_title_null_for_navigation.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.289Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_redirect.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.289Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_redirect_self.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.290Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_settitle.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.292Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_visited_notfound.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.292Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_visituri.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.292Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_visituri_nohistory.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.292Z] 17:31:14 INFO - TEST-INFO | toolkit/components/places/tests/browser/browser_visituri_privatebrowsing_perwindowpb.js | This test created 1 hidden docshell(s)
[task 2021-08-07T17:31:14.293Z] 17:31:14 INFO - Buffered messages finished
[task 2021-08-07T17:31:14.293Z] 17:31:14 ERROR - TEST-UNEXPECTED-FAIL | Last test finished | application terminated with exit code 11
[task 2021-08-07T17:31:14.293Z] 17:31:14 INFO - runtests.py | Application ran for: 0:14:21.776261
[task 2021-08-07T17:31:14.293Z] 17:31:14 INFO - zombiecheck | Reading PID log: /tmp/tmpe_p91aptpidlog
[task 2021-08-07T17:31:14.294Z] 17:31:14 INFO - ==> process 15806 launched child process 15824
<...>
[task 2021-08-07T17:31:14.348Z] 17:31:14 INFO - zombiecheck | Checking for orphan process with PID: 17023
[task 2021-08-07T17:31:14.349Z] 17:31:14 INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk /tmp/tmphpe6zosg.mozrunner/minidumps/4430e0d1-f608-70f4-ee10-b17200b1736d.dmp /builds/worker/workspace/build/symbols
[task 2021-08-07T17:31:20.653Z] 17:31:20 INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/4430e0d1-f608-70f4-ee10-b17200b1736d.dmp
[task 2021-08-07T17:31:20.655Z] 17:31:20 INFO - mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/4430e0d1-f608-70f4-ee10-b17200b1736d.extra
[task 2021-08-07T17:31:20.777Z] 17:31:20 INFO - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root]
[task 2021-08-07T17:31:20.778Z] 17:31:20 INFO - Mozilla crash reason: MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
[task 2021-08-07T17:31:20.778Z] 17:31:20 INFO - Crash dump filename: /tmp/tmphpe6zosg.mozrunner/minidumps/4430e0d1-f608-70f4-ee10-b17200b1736d.dmp
[task 2021-08-07T17:31:20.778Z] 17:31:20 INFO - Operating system: Linux
[task 2021-08-07T17:31:20.778Z] 17:31:20 INFO - 0.0.0 Linux 4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018 x86_64
[task 2021-08-07T17:31:20.779Z] 17:31:20 INFO - CPU: amd64
[task 2021-08-07T17:31:20.779Z] 17:31:20 INFO - family 6 model 85 stepping 7
[task 2021-08-07T17:31:20.780Z] 17:31:20 INFO - 2 CPUs
[task 2021-08-07T17:31:20.780Z] 17:31:20 INFO -
[task 2021-08-07T17:31:20.780Z] 17:31:20 INFO - GPU: UNKNOWN
[task 2021-08-07T17:31:20.781Z] 17:31:20 INFO -
[task 2021-08-07T17:31:20.781Z] 17:31:20 INFO - Crash reason: SIGSEGV /SEGV_MAPERR
[task 2021-08-07T17:31:20.781Z] 17:31:20 INFO - Crash address: 0x0
[task 2021-08-07T17:31:20.781Z] 17:31:20 INFO - Process uptime: not available
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO -
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO - Thread 37 (crashed) 0 libxul.so!mozilla::(anonymous namespace)::RunWatchdog(void*) [nsTerminator.cpp:a1d9ddb4fb7941b513a163f408f588c966515dc3 : 246 + 0x22]
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO - rax = 0x00007fa4c8ea48c0 rdx = 0x0000000000000000
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO - rcx = 0x0000561e42db3660 rbx = 0x0000000000000276
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO - rsi = 0x00007fa4dc9958b0 rdi = 0x00007fa4dc994680
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO - rbp = 0x00007fa4b79f7e60 rsp = 0x00007fa4b79f7e30
[task 2021-08-07T17:31:20.782Z] 17:31:20 INFO - r8 = 0x00007fa4dc9958b0 r9 = 0x00007fa4b79f8700
[task 2021-08-07T17:31:20.783Z] 17:31:20 INFO - r10 = 0x0000000000000000 r11 = 0x0000000000000000
[task 2021-08-07T17:31:20.783Z] 17:31:20 INFO - r12 = 0x00007fa4b79f8630 r13 = 0x0000000000000007
[task 2021-08-07T17:31:20.783Z] 17:31:20 INFO - r14 = 0x00007fa4b79f7e40 r15 = 0x00007fa4a61591c0
[task 2021-08-07T17:31:20.784Z] 17:31:20 INFO - rip = 0x00007fa4cfd07eb2
[task 2021-08-07T17:31:20.784Z] 17:31:20 INFO - Found by: given as instruction pointer in context
[task 2021-08-07T17:31:20.784Z] 17:31:20 INFO - 1 libnspr4.so!_pt_root [ptthread.c:a1d9ddb4fb7941b513a163f408f588c966515dc3 : 201 + 0x7]
[task 2021-08-07T17:31:20.784Z] 17:31:20 INFO - rbx = 0x00007fa49dd36dc0 rbp = 0x00007fa4b79f7eb0
[task 2021-08-07T17:31:20.785Z] 17:31:20 INFO - rsp = 0x00007fa4b79f7e70 r12 = 0x00007fa4b79f8630
[task 2021-08-07T17:31:20.785Z] 17:31:20 INFO - r13 = 0x0000000000000007 r14 = 0x00000000000042a6
[task 2021-08-07T17:31:20.785Z] 17:31:20 INFO - r15 = 0x00007fa4b79f8700 rip = 0x00007fa4dd8743f3
[task 2021-08-07T17:31:20.785Z] 17:31:20 INFO - Found by: call frame info
[task 2021-08-07T17:31:20.785Z] 17:31:20 INFO - 2 libpthread.so.0 + 0x76db
[task 2021-08-07T17:31:20.785Z] 17:31:20 INFO - rbx = 0x0000000000000000 rbp = 0x0000000000000000
[task 2021-08-07T17:31:20.786Z] 17:31:20 INFO - rsp = 0x00007fa4b79f7ec0 r12 = 0x00007fa4b79f7f80
[task 2021-08-07T17:31:20.786Z] 17:31:20 INFO - r13 = 0x0000000000000000 r14 = 0x00007fa49dd36dc0
[task 2021-08-07T17:31:20.786Z] 17:31:20 INFO - r15 = 0x00007ffc58a56d50 rip = 0x00007fa4dd4e36db
[task 2021-08-07T17:31:20.786Z] 17:31:20 INFO - Found by: call frame info
[task 2021-08-07T17:31:20.787Z] 17:31:20 INFO - 3 libc.so.6 + 0x121a3f
[task 2021-08-07T17:31:20.787Z] 17:31:20 INFO - rsp = 0x00007fa4b79f7f80 rip = 0x00007fa4dc6c9a3f
[task 2021-08-07T17:31:20.787Z] 17:31:20 INFO - Found by: stack scanning
[task 2021-08-07T17:31:20.787Z] 17:31:20 INFO -
<...>
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
status-firefox93:
--- → affected
status-firefox-esr91:
--- → unaffected
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 18•3 years ago
|
||
(In reply to Iulian Moraru from comment #15)
[task 2021-08-07T17:31:13.925Z] 17:31:13 INFO - GECKO(15806) | out of memory: 0x0000000028700000 bytes requested
That failure looks like there's more going on than just a shutdown hang.
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Flags: needinfo?(jmathies)
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
status-firefox94:
--- → affected
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 29•3 years ago
|
||
There are 122 total failures in the last 7 days on
- linux1804-64-ccov-qr opt
- linux1804-64-qr debug
- macosx1015-64-qr debug
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=352664086&repo=autoland&lineNumber=58777
[task 2021-09-24T15:10:08.190Z] 15:10:08 INFO - TEST-START | browser/components/extensions/test/browser/browser_ext_windows_remove.js
[task 2021-09-24T15:10:08.230Z] 15:10:08 INFO - GECKO(3110) | [Child 3115: Main Thread]: I/DocShellAndDOMWindowLeak ++DOCSHELL 10869cc00 == 7 [pid = 3115] [id = 966]
[task 2021-09-24T15:10:08.231Z] 15:10:08 INFO - GECKO(3110) | [Child 3115: Main Thread]: I/DocShellAndDOMWindowLeak ++DOMWINDOW == 45 (108679c80) [pid = 3115] [serial = 2761] [outer = 0]
[task 2021-09-24T15:10:08.231Z] 15:10:08 INFO - GECKO(3110) | [Child 3115: Main Thread]: I/DocShellAndDOMWindowLeak ++DOMWINDOW == 46 (1086a0c00) [pid = 3115] [serial = 2762] [outer = 108679c80]
[task 2021-09-24T15:10:08.237Z] 15:10:08 INFO - GECKO(3110) | [Child 3115, Main Thread] WARNING: Fallback to BasicLayerManager: file /builds/worker/checkouts/gecko/dom/ipc/BrowserChild.cpp:2864
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root]
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - Mozilla crash reason: MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - Crash dump filename: /var/folders/sf/kf7kx8y520g2kgmq0hhfxzf8000014/T/tmpi3_kc8gx.mozrunner/minidumps/6BC58CA4-DF96-4541-9C04-955EE72C1DF0.dmp
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - Operating system: Mac OS X
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - 10.15.7 19H524
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - CPU: amd64
[task 2021-09-24T15:27:01.306Z] 15:27:01 INFO - family 6 model 158 stepping 10
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - 12 CPUs
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO -
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - GPU: UNKNOWN
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO -
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - Crash reason: EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - Crash address: 0x0
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - Process uptime: 969 seconds
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO -
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - Thread 46 tid 555787 (crashed) - Shutdown Hang Terminator 0 XUL!mozilla::(anonymous namespace)::RunWatchdog(void*) [nsTerminator.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 246 + 0x22]
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rax = 0x000000011268280d rdx = 0x0000000000000000
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rcx = 0x0000000105b77458 rbx = 0x0000000000000276
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rsi = 0x00000000000120a8 rdi = 0x00007fff8ecb0ca8
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rbp = 0x00007000055aaf70 rsp = 0x00007000055aaf50
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - r8 = 0x00000000000130a8 r9 = 0x0000000000000000
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - r10 = 0x00007fff8ecb0cc8 r11 = 0x00007fff8ecb0cc0
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - r12 = 0x000000011c9d0c60 r13 = 0x0000000000000000
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - r14 = 0x00000001227a8f20 r15 = 0x0000000000000007
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rip = 0x000000010e3be0dc
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - Found by: given as instruction pointer in context
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - 1 libnss3.dylib!_pt_root [ptthread.c:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 201 + 0xa]
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rbx = 0x00007000055ab000 rbp = 0x00007000055aafb0
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - rsp = 0x00007000055aaf80 r12 = 0x000000011c9d0c60
[task 2021-09-24T15:27:01.307Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x00007000055ab000
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r15 = 0x0000000000000007 rip = 0x0000000106009939
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - 2 libsystem_pthread.dylib!_pthread_start + 0x94
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rbx = 0x00007000055ab000 rbp = 0x00007000055aafd0
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rsp = 0x00007000055aafc0 r12 = 0x0000000000000000
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r15 = 0x0000000000000000 rip = 0x00007fff68609109
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - 3 libsystem_pthread.dylib!thread_start + 0xf
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rbx = 0x0000000000000000 rbp = 0x00007000055aaff0
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rsp = 0x00007000055aafe0 r12 = 0x0000000000000000
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r15 = 0x0000000000000000 rip = 0x00007fff68604b8b
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO -
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - Thread 0 tid 775 - GeckoMain
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - 0 libsystem_kernel.dylib!__psynch_cvwait + 0xa
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rax = 0x0000000002000131 rdx = 0x0000000000005200
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rcx = 0x00007ffeea147f18 rbx = 0x0000000000000002
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rsi = 0x0000520100005300 rdi = 0x0000000117413530
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - rbp = 0x00007ffeea147fb0 rsp = 0x00007ffeea147f18
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r8 = 0x0000000000000000 r9 = 0x00000000000000a4
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r10 = 0x0000000000000000 r11 = 0x0000000000000202
[task 2021-09-24T15:27:01.308Z] 15:27:01 INFO - r12 = 0x0000000117413530 r13 = 0x0000000000000016
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r14 = 0x0000520100005300 r15 = 0x00000001087fcdc0
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rip = 0x00007fff68548882
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - Found by: given as instruction pointer in context
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - 1 libmozglue.dylib!mozilla::detail::ConditionVariableImpl::wait_for(mozilla::detail::MutexImpl&, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator> const&) [ConditionVariable_posix.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 115 + 0xb]
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rbp = 0x00007ffeea148000 rsp = 0x00007ffeea147fc0
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rip = 0x0000000105b2a0fd
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - Found by: previous frame's frame pointer
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - 2 XUL!mozilla::OffTheBooksCondVar::Wait(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>) [BlockingResourceBase.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 534 + 0x11]
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rbx = 0x0000000117413510 rbp = 0x00007ffeea148050
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rsp = 0x00007ffeea148010 r12 = 0x0000000000000000
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r13 = 0x0000000105d49120 r14 = 0x0000000105d49120
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r15 = 0x0000000117413501 rip = 0x00000001089b5c88
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - 3 XUL!mozilla::OffTheBooksCondVar::Wait() [BlockingResourceBase.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 514 + 0x5]
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rbx = 0x0000000117413400 rbp = 0x00007ffeea148060
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rsp = 0x00007ffeea148060 r12 = 0x0000000111f7359e
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r13 = 0x00007ffeea148078 r14 = 0xaaaaaaaaaaaaaaaa
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r15 = 0x0000000117413510 rip = 0x00000001089b5bb3
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - 4 XUL!mozilla::TaskController::GetRunnableForMTTask(bool) [TaskController.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 515 + 0x8]
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rbx = 0x0000000117413400 rbp = 0x00007ffeea1480b0
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - rsp = 0x00007ffeea148070 r12 = 0x0000000111f7359e
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r13 = 0x00007ffeea148078 r14 = 0xaaaaaaaaaaaaaaaa
[task 2021-09-24T15:27:01.309Z] 15:27:01 INFO - r15 = 0x0000000117413510 rip = 0x00000001089c6e88
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - 5 XUL!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 1082 + 0x19]
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rbx = 0x0000000000000001 rbp = 0x00007ffeea1481d0
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rsp = 0x00007ffeea1480c0 r12 = 0x0000000000000001
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r13 = 0x0000000105d21320 r14 = 0x000000011cb66b01
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r15 = 0x000000011e9a5601 rip = 0x00000001089d9f31
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - 6 XUL!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 466 + 0xd]
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rbx = 0x0000000000000001 rbp = 0x00007ffeea148200
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rsp = 0x00007ffeea1481e0 r12 = 0x000000011e9fd100
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x000000011cb66b60
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r15 = 0x000000011e9a5680 rip = 0x00000001089e0ddc
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - 7 XUL!mozilla::layers::CompositorThreadHolder::Shutdown() [CompositorThread.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 141 + 0x47]
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rbx = 0x0000000105d21320 rbp = 0x00007ffeea148250
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rsp = 0x00007ffeea148210 r12 = 0x000000011e9fd100
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x000000011cb66b60
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r15 = 0x000000011e9a5680 rip = 0x0000000109baeca7
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - 8 XUL!gfxPlatform::ShutdownLayersIPC() [gfxPlatform.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 1297 + 0x5]
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rbx = 0x00007ffeea1482b0 rbp = 0x00007ffeea148280
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rsp = 0x00007ffeea148260 r12 = 0x0000000105d8bc40
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000105d61908
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r15 = 0x00007ffeea148450 rip = 0x0000000109c6a09f
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - 9 XUL!mozilla::ShutdownXPCOM(nsIServiceManager*) [XPCOMInit.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 622 + 0x5]
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rbx = 0x00007ffeea1482b0 rbp = 0x00007ffeea148320
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rsp = 0x00007ffeea148290 r12 = 0x0000000105d8bc40
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000105d61908
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r15 = 0x00007ffeea148450 rip = 0x0000000108a2837d
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - 10 XUL!ScopedXPCOMStartup::~ScopedXPCOMStartup() [nsAppRunner.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 1683 + 0x8]
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rbx = 0xaaaaaaaaaaaaaaaa rbp = 0x00007ffeea1483b0
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - rsp = 0x00007ffeea148330 r12 = 0x0000000105d8bc40
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000105d16500
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - r15 = 0x00007ffeea148450 rip = 0x000000010e3f8829
[task 2021-09-24T15:27:01.310Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - 11 XUL!XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) [nsAppRunner.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 5493 + 0x19]
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rbx = 0x0000000105d16500 rbp = 0x00007ffeea148440
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rsp = 0x00007ffeea1483c0 r12 = 0x0000000105d8bc40
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r15 = 0x00007ffeea148450 rip = 0x000000010e4072a9
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - 12 XUL!XRE_main(int, char**, mozilla::BootstrapConfig const&) [nsAppRunner.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 5535 + 0x10]
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rbx = 0x0000000000000005 rbp = 0x00007ffeea148580
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rsp = 0x00007ffeea148450 r12 = 0x00007ffeea148450
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r13 = 0x00007ffeea148a18 r14 = 0x00007ffeea1485c0
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r15 = 0x00007ffeea148a18 rip = 0x000000010e407b25
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - 13 firefox!main [nsBrowserApp.cpp:4e8c68a872fe1601f9f05a2ddc43b1335413a383 : 386 + 0x29d]
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rbx = 0x0000000105ab9331 rbp = 0x00007ffeea1489f0
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rsp = 0x00007ffeea148590 r12 = 0x000002711f5b9e6d
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r13 = 0x00007ffeea148a18 r14 = 0x00007ffeea148d31
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r15 = 0x0000000000000005 rip = 0x0000000105ab72d7
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - 14 libdyld.dylib!start + 0x1
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rbx = 0x0000000000000000 rbp = 0x00007ffeea148a08
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rsp = 0x00007ffeea148a00 r12 = 0x0000000000000000
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r15 = 0x0000000000000000 rip = 0x00007fff68404cc9
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - Found by: call frame info
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - 15 libdyld.dylib!start + 0x1
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rbx = 0x0000000000000000 rbp = 0x00007ffeea148a08
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - rsp = 0x00007ffeea148a08 r12 = 0x0000000000000000
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000
[task 2021-09-24T15:27:01.311Z] 15:27:01 INFO - r15 = 0x0000000000000000 rip = 0x00007fff68404cc9
[task 2021-09-24T15:27:01.312Z] 15:27:01 INFO - Found by: call frame info
Jed, are you working on this?
Flags: needinfo?(jld)
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 31•3 years ago
|
||
(In reply to Andreea Pavel [:apavel] from comment #29)
Jed, are you working on this?
Based on Jed's comment 2 and comment 15, I think this is a bug for the Graphics team. A process seems to be OOM'ing and then deadlocking in CompositorThreadHolder::Shutdown().
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 37•3 years ago
|
||
(In reply to Intermittent Failures Robot from comment #36)
109 failures in 3376 pushes (0.032 failures/push) were associated with this bug in the last 7 days.
This is the #7 most frequent failure this week.
** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **
Jim, this intermittent test failure (a shutdown hang) looks like a graphics issue. It started in July and then the failure volume increased after some spikes in September.
status-firefox95:
--- → affected
Flags: needinfo?(jld) → needinfo?(jmathies)
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 47•3 years ago
|
||
There are 125 total failures in the last 7 days on
- macosx1015-64-qr debug
- linux1804-64-qr debug
- linux1804-64-ccov-qr opt
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=355054253&repo=mozilla-central&lineNumber=4239
[task 2021-10-17T02:12:32.655Z] 02:12:32 INFO - TEST-START | dom/fetch/tests/test_invalid_header_exception.html
[task 2021-10-17T02:12:32.696Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:32.868Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:32.924Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:32.948Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3353
[task 2021-10-17T02:12:32.970Z] 02:12:32 INFO - GECKO(2717) | [Parent 2717, Main Thread] WARNING: WebProgress Ignored: no longer current window global: file /builds/worker/checkouts/gecko/dom/ipc/BrowserParent.cpp:2976
[task 2021-10-17T02:12:32.973Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:32.987Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:32.990Z] 02:12:32 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:33.005Z] 02:12:33 INFO - GECKO(2717) | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to /tmp/tmpnyszzx14.mozrunner/runtests_leaks_tab_pid3010.log
[task 2021-10-17T02:12:33.008Z] 02:12:33 INFO - GECKO(2717) | [3010, Main Thread] WARNING: XPCOM_MEM_BLOAT_LOG is set, disabling native allocations.: file /builds/worker/checkouts/gecko/tools/profiler/core/platform.cpp:251
[task 2021-10-17T02:12:33.016Z] 02:12:33 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:33.030Z] 02:12:33 INFO - GECKO(2717) | [Child 2909, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4439
[task 2021-10-17T02:12:33.111Z] 02:12:33 INFO - GECKO(2717) | [Child 3010, Main Thread] WARNING: could not set real-time limit in CubebUtils::InitLibrary: file /builds/worker/checkouts/gecko/dom/media/CubebUtils.cpp:639
[task 2021-10-17T02:12:33.123Z] 02:12:33 INFO - GECKO(2717) | [Child 2940, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:33.124Z] 02:12:33 INFO - GECKO(2717) | [Child 2940, Main Thread] WARNING: '!ipcActor->SendRecordDiscardedData(discardedData)', file /builds/worker/checkouts/gecko/toolkit/components/telemetry/core/ipc/TelemetryIPCAccumulator.cpp:298
[task 2021-10-17T02:12:33.238Z] 02:12:33 INFO - GECKO(2717) | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to /tmp/tmpnyszzx14.mozrunner/runtests_leaks_tab_pid3016.log
[task 2021-10-17T02:12:33.265Z] 02:12:33 INFO - GECKO(2717) | [3016, Main Thread] WARNING: XPCOM_MEM_BLOAT_LOG is set, disabling native allocations.: file /builds/worker/checkouts/gecko/tools/profiler/core/platform.cpp:251
[task 2021-10-17T02:12:33.379Z] 02:12:33 INFO - GECKO(2717) | [Child 3016, Main Thread] WARNING: could not set real-time limit in CubebUtils::InitLibrary: file /builds/worker/checkouts/gecko/dom/media/CubebUtils.cpp:639
[task 2021-10-17T02:12:35.217Z] 02:12:35 INFO - GECKO(2717) | MEMORY STAT | vsize 2518MB | residentFast 136MB | heapAllocated 8MB
[task 2021-10-17T02:12:35.259Z] 02:12:35 INFO - TEST-OK | dom/fetch/tests/test_invalid_header_exception.html | took 2604ms
[task 2021-10-17T02:12:35.401Z] 02:12:35 INFO - GECKO(2717) | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[task 2021-10-17T02:12:35.403Z] 02:12:35 INFO - GECKO(2717) | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[task 2021-10-17T02:12:35.406Z] 02:12:35 INFO - GECKO(2717) | [Child 2943, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:35.472Z] 02:12:35 INFO - GECKO(2717) | [Child 2943, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:527
[task 2021-10-17T02:12:35.547Z] 02:12:35 INFO - TEST-START | Shutdown
[task 2021-10-17T02:25:35.664Z] 02:25:35 INFO - GECKO(2717) | out of memory: 0x0000000028700000 bytes requested
[task 2021-10-17T02:25:35.665Z] 02:25:35 INFO - GECKO(2717) | Hit MOZ_CRASH() at /builds/worker/checkouts/gecko/memory/mozalloc/mozalloc_abort.cpp:33
[task 2021-10-17T02:25:35.845Z] 02:25:35 INFO - GECKO(2717) | #01: mozalloc_abort [memory/mozalloc/mozalloc_abort.cpp:33]
[task 2021-10-17T02:25:35.846Z] 02:25:35 INFO - GECKO(2717) | #02: mozalloc_handle_oom(unsigned long) [memory/mozalloc/mozalloc_oom.cpp:51]
[task 2021-10-17T02:25:35.848Z] 02:25:35 INFO - GECKO(2717) | #03: moz_xstrdup [memory/mozalloc/mozalloc.cpp:80]
[task 2021-10-17T02:25:35.849Z] 02:25:35 INFO - GECKO(2717) | #04: nsTArrayInfallibleAllocator::ResultTypeProxy nsTArray_base<nsTArrayInfallibleAllocator, nsTArray_RelocateUsingMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator>(unsigned long, unsigned long) [xpcom/ds/nsTArray-inl.h:223]
[task 2021-10-17T02:25:35.851Z] 02:25:35 INFO - GECKO(2717) | #05: mozilla::net::ChannelEventQueue::RunOrEnqueue(mozilla::net::ChannelEvent*, bool) [netwerk/ipc/ChannelEventQueue.h:218]
[task 2021-10-17T02:25:35.853Z] 02:25:35 INFO - GECKO(2717) | #06: mozilla::net::HttpChannelChild::DoNotifyListener() [netwerk/protocol/http/HttpChannelChild.cpp:1218]
[task 2021-10-17T02:25:35.854Z] 02:25:35 INFO - GECKO(2717) | #07: mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::net::HttpChannelChild>, void (mozilla::net::HttpChannelChild::*)(), true, (mozilla::RunnableKind)0, >::Run() [xpcom/threads/nsThreadUtils.h:1203]
[task 2021-10-17T02:25:35.855Z] 02:25:35 INFO - GECKO(2717) | #08: mozilla::RunnableTask::Run() [xpcom/threads/TaskController.cpp:468]
[task 2021-10-17T02:25:35.856Z] 02:25:35 INFO - GECKO(2717) | #09: mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [xpcom/threads/TaskController.cpp:770]
[task 2021-10-17T02:25:35.857Z] 02:25:35 INFO - GECKO(2717) | #10: mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [xpcom/threads/TaskController.cpp:606]
[task 2021-10-17T02:25:35.858Z] 02:25:35 INFO - GECKO(2717) | #11: mozilla::TaskController::ProcessPendingMTTask(bool) [xpcom/threads/TaskController.cpp:390]
[task 2021-10-17T02:25:35.859Z] 02:25:35 INFO - GECKO(2717) | #12: mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_0>::Run() [xpcom/threads/nsThreadUtils.h:532]
[task 2021-10-17T02:25:35.859Z] 02:25:35 INFO - GECKO(2717) | #13: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1155]
[task 2021-10-17T02:25:35.860Z] 02:25:35 INFO - GECKO(2717) | #14: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:467]
[task 2021-10-17T02:25:35.861Z] 02:25:35 INFO - GECKO(2717) | #15: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:85]
[task 2021-10-17T02:25:35.862Z] 02:25:35 INFO - GECKO(2717) | #16: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:331]
[task 2021-10-17T02:25:35.863Z] 02:25:35 INFO - GECKO(2717) | #17: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
[task 2021-10-17T02:25:35.863Z] 02:25:35 INFO - GECKO(2717) | #18: nsBaseAppShell::Run() [widget/nsBaseAppShell.cpp:139]
[task 2021-10-17T02:25:35.864Z] 02:25:35 INFO - GECKO(2717) | #19: XRE_RunAppShell() [toolkit/xre/nsEmbedFunctions.cpp:917]
[task 2021-10-17T02:25:35.865Z] 02:25:35 INFO - GECKO(2717) | #20: mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:235]
[task 2021-10-17T02:25:35.866Z] 02:25:35 INFO - GECKO(2717) | #21: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:331]
[task 2021-10-17T02:25:35.866Z] 02:25:35 INFO - GECKO(2717) | #22: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
[task 2021-10-17T02:25:35.867Z] 02:25:35 INFO - GECKO(2717) | #23: XRE_InitChildProcess(int, char**, XREChildData const*) [toolkit/xre/nsEmbedFunctions.cpp:749]
[task 2021-10-17T02:25:35.868Z] 02:25:35 INFO - GECKO(2717) | #24: content_process_main(mozilla::Bootstrap*, int, char**) [ipc/contentproc/plugin-container.cpp:58]
[task 2021-10-17T02:25:35.869Z] 02:25:35 INFO - GECKO(2717) | #25: main [browser/app/nsBrowserApp.cpp:327]
[task 2021-10-17T02:25:35.869Z] 02:25:35 INFO - GECKO(2717) | #26: __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6 + 0x21b97]
[task 2021-10-17T02:25:35.870Z] 02:25:35 INFO - GECKO(2717) | #27: ??? [/builds/worker/workspace/build/application/firefox/firefox-bin + 0x42089]
[task 2021-10-17T02:25:35.871Z] 02:25:35 INFO - GECKO(2717) | #28: ??? (???:???)
[task 2021-10-17T02:25:36.086Z] 02:25:36 INFO - TEST-INFO | Main app process: exit 11
[task 2021-10-17T02:25:36.089Z] 02:25:36 INFO - Buffered messages finished
[task 2021-10-17T02:25:36.090Z] 02:25:36 ERROR - TEST-UNEXPECTED-FAIL | Last test finished | application terminated with exit code 11
[task 2021-10-17T02:25:36.094Z] 02:25:36 INFO - runtests.py | Application ran for: 0:13:18.845999
[task 2021-10-17T02:25:45.863Z] 02:25:45 INFO - PROCESS-CRASH | Last test finished | application crashed [@ _pt_root]
[task 2021-10-17T02:25:45.863Z] 02:25:45 INFO - Mozilla crash reason: MOZ_CRASH(Shutdown hanging after all known phases and workers finished.)
[task 2021-10-17T02:25:45.863Z] 02:25:45 INFO - Crash dump filename: /tmp/tmpnyszzx14.mozrunner/minidumps/4aab7ed2-6256-f409-b029-066d41325728.dmp
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Operating system: Linux
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - 0.0.0 Linux 4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018 x86_64
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - CPU: amd64
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - family 6 model 85 stepping 4
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - 2 CPUs
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO -
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - GPU: UNKNOWN
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO -
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Crash reason: SIGSEGV / SEGV_MAPERR
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Crash address: 0x0
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Process uptime: not available
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO -
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Thread 39 tid 3051 (crashed) 0 libxul.so!mozilla::(anonymous namespace)::RunWatchdog(void*) [nsTerminator.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 256 + 0x22]
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rax = 0x00007f50d080b810 rdx = 0x0000000000000000
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rcx = 0x00005574bea3c780 rbx = 0x00007f50a02fee20
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rsi = 0x00007f50e44f88b0 rdi = 0x00007f50e44f7680
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rbp = 0x00007f50a02fee60 rsp = 0x00007f50a02fee20
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r8 = 0x00007f50e44f88b0 r9 = 0x00007f50a02ff700
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r10 = 0x0000000000000000 r11 = 0x0000000000000000
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r12 = 0x00007f50a02ff630 r13 = 0x0000000000000007
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r14 = 0x00007f50a02fee40 r15 = 0x00007f50a28c50b0
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rip = 0x00007f50d780147a
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Found by: given as instruction pointer in context
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - 1 libnspr4.so!_pt_root [ptthread.c:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 201 + 0x7]
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rbx = 0x00007f50a377b040 rbp = 0x00007f50a02feeb0
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rsp = 0x00007f50a02fee70 r12 = 0x00007f50a02ff630
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r13 = 0x0000000000000007 r14 = 0x0000000000000beb
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r15 = 0x00007f50a02ff700 rip = 0x00007f50e53d73f3
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Found by: call frame info
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - 2 libpthread.so.0 + 0x76db
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rbx = 0x0000000000000000 rbp = 0x0000000000000000
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rsp = 0x00007f50a02feec0 r12 = 0x00007f50a02fef80
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r13 = 0x0000000000000000 r14 = 0x00007f50a377b040
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - r15 = 0x00007fffcbb462f0 rip = 0x00007f50e50466db
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Found by: call frame info
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - 3 libc.so.6 + 0x121a3f
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rsp = 0x00007f50a02fef80 rip = 0x00007f50e422ca3f
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - 0 libpthread.so.0 + 0xd9f3
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rax = 0xfffffffffffffe00 rdx = 0x0000000000000000
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rcx = 0x00007f50e504c9f3 rbx = 0x00007f50cd55eb00
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rsi = 0x0000000000000080 rdi = 0x00007f50cd55eb2c
[task 2021-10-17T02:25:45.864Z] 02:25:45 INFO - rbp = 0x00007f50cd55eb24 rsp = 0x00007fffcbb495f0
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - r8 = 0x0000000000000001 r9 = 0x0000000000000000
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - r10 = 0x0000000000000000 r11 = 0x0000000000000246
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - r12 = 0x00007f50cd55eb2c r13 = 0x0000000000000000
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - r14 = 0x00007f50cd55ea00 r15 = 0x00000000000000c3
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - rip = 0x00007f50e504c9f3
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - Found by: given as instruction pointer in context
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - 1 libgdk-3.so.0 + 0x67fd6
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - rsp = 0x00007fffcbb49630 rip = 0x00007f50e2775fd6
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - 2 libpthread.so.0 + 0xd690
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - rsp = 0x00007fffcbb49638 rip = 0x00007f50e504c690
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - 3 firefox-bin!mozilla::detail::ConditionVariableImpl::wait(mozilla::detail::MutexImpl&) [ConditionVariable_posix.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 108 + 0xb]
[task 2021-10-17T02:25:45.865Z] 02:25:45 INFO - rsp = 0x00007fffcbb496c0 rip = 0x00005574bea03089
[task 2021-10-17T02:25:45.866Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.866Z] 02:25:45 INFO - 4 firefox-bin!mozilla::detail::ConditionVariableImpl::wait_for(mozilla::detail::MutexImpl&, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator> const&) [ConditionVariable_posix.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 115 + 0x8]
[task 2021-10-17T02:25:45.866Z] 02:25:45 INFO - rsp = 0x00007fffcbb496e0 rip = 0x00005574bea03104
[task 2021-10-17T02:25:45.867Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.867Z] 02:25:45 INFO - 5 ld-linux-x86-64.so.2 + 0x138cc
[task 2021-10-17T02:25:45.867Z] 02:25:45 INFO - rsp = 0x00007fffcbb49710 rip = 0x00007f50e52718cc
[task 2021-10-17T02:25:45.867Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.867Z] 02:25:45 INFO - 6 libxul.so!mozilla::OffTheBooksCondVar::Wait(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>) [BlockingResourceBase.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 534 + 0x10]
[task 2021-10-17T02:25:45.867Z] 02:25:45 INFO - rsp = 0x00007fffcbb49750 rip = 0x00007f50d35c118a
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - 7 libxul.so!mozilla::OffTheBooksCondVar::Wait() [BlockingResourceBase.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 514 + 0x5]
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - rsp = 0x00007fffcbb497a0 rip = 0x00007f50d35c10df
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - 8 libxul.so!mozilla::TaskController::GetRunnableForMTTask(bool) [TaskController.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 515 + 0x8]
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - rsp = 0x00007fffcbb497b0 rip = 0x00007f50d35cf0d7
[task 2021-10-17T02:25:45.868Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - 9 libxul.so!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 1085 + 0x11]
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - rsp = 0x00007fffcbb49800 rip = 0x00007f50d35dca71
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - 10 libxul.so!<.plt ELF section in libxul.so> + 0x56594a
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - rsp = 0x00007fffcbb49830 rip = 0x00007f50da4a00aa
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.869Z] 02:25:45 INFO - 11 ld-linux-x86-64.so.2 + 0x138cc
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - rsp = 0x00007fffcbb498c0 rip = 0x00007f50e52718cc
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - 12 ld-linux-x86-64.so.2 + 0x19a58
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - rsp = 0x00007fffcbb498d0 rip = 0x00007f50e5277a58
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - 13 libxul.so!NS_IsMainThread [nsThreadManager.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 224 + 0x10]
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - rsp = 0x00007fffcbb498e0 rip = 0x00007f50d35ceb2c
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - 14 libxul.so!nsIXPConnect::XPConnect() [nsXPConnect.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 926 + 0x5]
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - rsp = 0x00007fffcbb498f0 rip = 0x00007f50d406d37f
[task 2021-10-17T02:25:45.870Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - 15 libxul.so!XPCJSContext::RecordScriptActivity(bool) [XPCJSContext.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 551 + 0x5]
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - rsp = 0x00007fffcbb49900 rip = 0x00007f50d4040ec0
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - 16 libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 467 + 0xc]
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - rsp = 0x00007fffcbb49940 rip = 0x00007f50d35e1850
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - Found by: stack scanning
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - 17 libxul.so!mozilla::layers::CompositorThreadHolder::Shutdown() [CompositorThread.cpp:e361209af61f51a3fa1dbf9ea8a2d276e22cc5bf : 141 + 0x1fa]
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - rsp = 0x00007fffcbb49970 rip = 0x00007f50d4367688
[task 2021-10-17T02:25:45.871Z] 02:25:45 INFO - Found by: stack scanning
Jim, are there any updates here?
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 53•3 years ago
|
||
There have been 112 total failures in the last 7 days, recent failure log.
Affected platforms are:
- macosx1015-64-qr
- linux1804-64-qr
- linux1804-64-ccov-qr
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 61•3 years ago
|
||
There have been 54 total failures in the last 7 days, recent failure log.
Affected platforms are:
- linux1804-64-ccov-qr
- linux1804-64-qr
- macosx1015-64-qr
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Updated•3 years ago
|
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 67•3 years ago
|
||
Andrew, any thoughts here? Some sort of failure in proper shutdown after an OOM.
|
||
Comment 68•3 years ago
•
|
||
This appears to only happen on Linux and OSX. Not on Windows (or Android). Possibly related to not having a GPU process. Doesn't matter whether it is a debug build or opt although more seem common on debug builds. We don't explicitly close CompositorManagerParent protocols on debug builds, but I don't remember why:
https://searchfox.org/mozilla-central/rev/84c1031f19b2cf8221da0850983f8d0269f6efd1/gfx/layers/ipc/CompositorManagerParent.h#25
https://searchfox.org/mozilla-central/rev/84c1031f19b2cf8221da0850983f8d0269f6efd1/gfx/layers/ipc/CompositorManagerParent.cpp#206
|
|
||
Comment 69•3 years ago
•
|
||
I was able to reproduce quite readily in try. I'll add some debug prints and see if that helps to identify which is the culprit:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f612e6f46160733559411851ea851fd34eea8fb0
|
|
||
Updated•3 years ago
|
No longer blocks: gfx-triage
You need to log in
before you can comment on or make changes to this bug.
Description
•