Closed Bug 1743092 Opened 2 years ago Closed 1 year ago

Intermittent LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator

Categories

(Core :: DOM: Workers, defect)

defect

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disable-recommended])

Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=359336486&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/eBEl2D2gSTWfy41CV-zOOw/runs/0/artifacts/public/logs/live_backing.log


INFO - TEST-START | /html/infrastructure/urls/resolving-urls/query-encoding/navigation.sub.html?encoding=utf8
[task 2021-11-26T10:07:53.752Z] 10:07:53     INFO - PID 6429 |     #21 0x7fd95b472a78 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20
[task 2021-11-26T10:07:53.753Z] 10:07:53     INFO - PID 6429 |     #22 0x7fd95b37df41 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:331:10
[task 2021-11-26T10:07:53.753Z] 10:07:53     INFO - PID 6429 |     #23 0x7fd95b37df41 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:324:3
[task 2021-11-26T10:07:53.754Z] 10:07:53     INFO - PID 6429 |     #24 0x7fd95b37df41 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:306:3
[task 2021-11-26T10:07:53.755Z] 10:07:53     INFO - PID 6429 |     #25 0x7fd95a2db189 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:391:10
[task 2021-11-26T10:07:53.755Z] 10:07:53     INFO - PID 6429 |     #26 0x7fd97b8d8ade in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2021-11-26T10:07:53.756Z] 10:07:53     INFO - PID 6429 |     #27 0x7fd97d69d6da in start_thread /tmp/glibc/nptl/pthread_create.c:463
....
[task 2021-11-26T10:07:55.219Z] 10:07:55     INFO - Browser exited with return code 0
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - ERROR | LeakSanitizer | SUMMARY: AddressSanitizer: 280 byte(s) leaked in 3 allocation(s).
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - Allowed depth was 4
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::AddStaticElement, InitializeStaticHeaders, mozilla::net::nvFIFO::nvFIFO, mozilla::net::Http2BaseCompressor::Http2BaseCompressor
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - INFO | LeakSanitizer | Frame mozilla::net::AddStaticElement matched a expected leak
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived, mozilla::ipc::MessageChannel::DispatchAsyncMessage, mozilla::ipc::MessageChannel::DispatchMessage
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - TEST-FAIL | LeakSanitizer | leak at mozilla::net::AddStaticElement, AddStaticElement, InitializeStaticHeaders, mozilla::net::nvFIFO::nvFIFO
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - INFO | LeakSanitizer | Frame mozilla::net::AddStaticElement matched a expected leak
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::SupportsThreadSafeWeakPtr, mozilla::dom::RemoteWorkerChild::RemoteWorkerChild, mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - Closing logging queue
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - queue closed
[task 2021-11-26T10:07:55.227Z] 10:07:55     INFO - STDERR: Traceback (most recent call last):
[task 2021-11-26T10:07:55.228Z] 10:07:55     INFO - STDERR:   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/mozprocess/processhandler.py", line 1239, in _read
[task 2021-11-26T10:07:55.228Z] 10:07:55     INFO -     callback(line.rstrip())
[task 2021-11-26T10:07:55.231Z] 10:07:55     INFO - STDERR:   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/mozprocess/processhandler.py", line 1136, in __call__
[task 2021-11-26T10:07:55.231Z] 10:07:55     INFO -     e(*args, **kwargs)
[task 2021-11-26T10:07:55.232Z] 10:07:55     INFO - STDERR:   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/browsers/firefox.py", line 604, in __call__
[task 2021-11-26T10:07:55.232Z] 10:07:55     INFO -     data = self.lsan_handler.log(data)
[task 2021-11-26T10:07:55.233Z] 10:07:55     INFO - STDERR:   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/mozleak/lsan.py", line 116, in log
[task 2021-11-26T10:07:55.233Z] 10:07:55     INFO -     assert self.summaryData is None
[task 2021-11-26T10:07:55.234Z] 10:07:55     INFO - STDERR: AssertionError
[task 2021-11-26T10:07:55.235Z] 10:07:55     INFO - INFO | runtests.py | ASan using symbolizer at /builds/worker/workspace/build/application/firefox/llvm-symbolizer
[task 2021-11-26T10:07:55.236Z] 10:07:55     INFO - LSan enabled.
[task 2021-11-26T10:07:55.237Z] 10:07:55     INFO - INFO | runtests.py | ASan running in default memory configuration
[task 2021-11-26T10:07:55.238Z] 10:07:55     INFO - UBSan enabled.
[task 2021-11-26T10:07:55.240Z] 10:07:55     INFO - Application command: /builds/worker/workspace/build/application/firefox/firefox --marionette about:blank -profile /tmp/tmpn0_dmwx1
[task 2021-11-26T10:07:55.257Z] 10:07:55     INFO - ......
[task 2021-11-26T10:07:55.257Z] 10:07:55     INFO - TEST-OK | /html/infrastructure/urls/resolving-urls/query-encoding/navigation.sub.html?encoding=utf8 | took 1595ms
Component: DOM: Core & HTML → Networking
Blocks: LSan

Update:
There have been 29 failures within the last 7 days, all on Linux 18.04 x64 WebRender asan opt.
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=370067956&repo=autoland&lineNumber=3129

[task 2022-03-05T04:18:41.345Z] 04:18:41     INFO - TEST-START | dom/serviceworkers/test/test_xslt.html
<...>
[task 2022-03-05T04:18:45.115Z] 04:18:45     INFO - GECKO(3034) | 1646453925106	Marionette	DEBUG	Marionette stopped listening
[task 2022-03-05T04:18:45.665Z] 04:18:45     INFO - GECKO(3034) | ###!!! [Parent][PCompositorManagerParent] Error: RunMessage(msgname=PCompositorBridge::Msg___delete__) Channel closing: too late to send/recv, messages will be lost
[task 2022-03-05T04:18:45.752Z] 04:18:45     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:45.754Z] 04:18:45     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:45.755Z] 04:18:45     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:45.755Z] 04:18:45     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:45.757Z] 04:18:45     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:45.758Z] 04:18:45     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:46.004Z] 04:18:46     INFO - GECKO(3034) | ###!!! [Parent][PCompositorManagerParent] Error: RunMessage(msgname=PCompositorBridge::Msg___delete__) Channel closing: too late to send/recv, messages will be lost
[task 2022-03-05T04:18:46.015Z] 04:18:46     INFO - GECKO(3034) | ###!!! [Parent][PCompositorManagerParent] Error: RunMessage(msgname=PCompositorBridge::Msg___delete__) Channel closing: too late to send/recv, messages will be lost
[task 2022-03-05T04:18:46.024Z] 04:18:46     INFO - GECKO(3034) | ###!!! [Parent][PCompositorManagerParent] Error: RunMessage(msgname=PCompositorBridge::Msg___delete__) Channel closing: too late to send/recv, messages will be lost
[task 2022-03-05T04:18:47.638Z] 04:18:47     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:47.638Z] 04:18:47     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:47.638Z] 04:18:47     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:47.639Z] 04:18:47     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:47.640Z] 04:18:47     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:47.640Z] 04:18:47     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:50.822Z] 04:18:50     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:50.824Z] 04:18:50     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:50.825Z] 04:18:50     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:50.826Z] 04:18:50     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:50.827Z] 04:18:50     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:50.828Z] 04:18:50     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:50.921Z] 04:18:50     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:50.925Z] 04:18:50     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:50.927Z] 04:18:50     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:50.931Z] 04:18:50     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:50.932Z] 04:18:50     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:50.933Z] 04:18:50     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.217Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.220Z] 04:18:51     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:51.223Z] 04:18:51     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:51.227Z] 04:18:51     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:51.229Z] 04:18:51     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:51.230Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.329Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.330Z] 04:18:51     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:51.331Z] 04:18:51     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:51.333Z] 04:18:51     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:51.334Z] 04:18:51     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:51.334Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.392Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.394Z] 04:18:51     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:51.395Z] 04:18:51     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:51.396Z] 04:18:51     INFO - GECKO(3034) |      13        416 nsComponentManagerImpl
[task 2022-03-05T04:18:51.397Z] 04:18:51     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:51.398Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.540Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.544Z] 04:18:51     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:51.547Z] 04:18:51     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:51.548Z] 04:18:51     INFO - GECKO(3034) |      55       1648 nsComponentManagerImpl
[task 2022-03-05T04:18:51.550Z] 04:18:51     INFO - GECKO(3034) |       2        288 libfontconfig.so
[task 2022-03-05T04:18:51.551Z] 04:18:51     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:51.566Z] 04:18:51     INFO - GECKO(3034) | Waiting in WillDestroyCurrentMessageLoop for pid 3148
[task 2022-03-05T04:18:55.831Z] 04:18:55     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:55.832Z] 04:18:55     INFO - GECKO(3034) | Suppressions used:
[task 2022-03-05T04:18:55.833Z] 04:18:55     INFO - GECKO(3034) |   count      bytes template
[task 2022-03-05T04:18:55.834Z] 04:18:55     INFO - GECKO(3034) |      10        312 nsComponentManagerImpl
[task 2022-03-05T04:18:55.835Z] 04:18:55     INFO - GECKO(3034) |     633      18115 libfontconfig.so
[task 2022-03-05T04:18:55.836Z] 04:18:55     INFO - GECKO(3034) |       1          9 libglib-2.0.so
[task 2022-03-05T04:18:55.836Z] 04:18:55     INFO - GECKO(3034) |       2        416 mozJSComponentLoader
[task 2022-03-05T04:18:55.836Z] 04:18:55     INFO - GECKO(3034) |       2         48 XPCWrappedNativeJSOps
[task 2022-03-05T04:18:55.837Z] 04:18:55     INFO - GECKO(3034) | -----------------------------------------------------
[task 2022-03-05T04:18:56.144Z] 04:18:56     INFO - TEST-INFO | Main app process: exit 0
[task 2022-03-05T04:18:56.145Z] 04:18:56     INFO - TEST-INFO | LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2022-03-05T04:18:56.145Z] 04:18:56     INFO - TEST-INFO | LeakSanitizer | This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2022-03-05T04:18:56.146Z] 04:18:56    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator
[task 2022-03-05T04:18:56.146Z] 04:18:56    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::SupportsThreadSafeWeakPtr, mozilla::dom::RemoteWorkerChild::RemoteWorkerChild, mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived
[task 2022-03-05T04:18:56.146Z] 04:18:56    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at NS_NewRunnableFunction, mozilla::dom::RemoteWorkerChild::ShutdownOnWorker, operator, std::_Function_handler
[task 2022-03-05T04:18:56.147Z] 04:18:56    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived, mozilla::ipc::MessageChannel::DispatchAsyncMessage, mozilla::ipc::MessageChannel::DispatchMessage
[task 2022-03-05T04:18:56.147Z] 04:18:56     INFO - runtests.py | Application ran for: 0:04:41.952776
Whiteboard: [stockwell needswork:owner]

Hi Dragana! Can you please take a look at this?
Thank you!

Flags: needinfo?(dd.mozilla)

This looks like another variation of the RemoteWorkerChild proxy leak.

Component: Networking → DOM: Workers
Flags: needinfo?(dd.mozilla) → needinfo?(jstutte)
Flags: needinfo?(jstutte)
See Also: → 1749068

Update:
There have been 37 failures within the last 7 days on Linux 18.04 x64 WebRender asan opt.

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=378854460&repo=autoland&lineNumber=2439

[task 2022-05-21T10:32:34.722Z] 10:32:34     INFO - TEST-START | /content-security-policy/inside-worker/sharedworker-script-src.sub.html
[task 2022-05-21T10:32:34.726Z] 10:32:34     INFO - Closing window d4b8455f-4405-4524-a198-86ddd874ca35
[task 2022-05-21T10:32:36.498Z] 10:32:36     INFO - ............
[task 2022-05-21T10:32:36.498Z] 10:32:36     INFO - TEST-OK | /content-security-policy/inside-worker/sharedworker-script-src.sub.html | took 1772ms
[task 2022-05-21T10:32:36.498Z] 10:32:36     INFO - Restarting browser for new test group
[task 2022-05-21T10:32:36.500Z] 10:32:36     INFO - PID 1930 | 1653129156495	Marionette	INFO	Stopped listening on port 55673
[task 2022-05-21T10:32:38.960Z] 10:32:38     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:38.961Z] 10:32:38     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:38.962Z] 10:32:38     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:38.965Z] 10:32:38     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:38.966Z] 10:32:38     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:38.967Z] 10:32:38     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.036Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.037Z] 10:32:39     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:39.038Z] 10:32:39     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:39.041Z] 10:32:39     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:39.042Z] 10:32:39     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:39.043Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.341Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.341Z] 10:32:39     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:39.341Z] 10:32:39     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:39.341Z] 10:32:39     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:39.342Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.370Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.370Z] 10:32:39     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:39.371Z] 10:32:39     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:39.371Z] 10:32:39     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:39.371Z] 10:32:39     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:39.371Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.466Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.467Z] 10:32:39     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:39.467Z] 10:32:39     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:39.467Z] 10:32:39     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:39.467Z] 10:32:39     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:39.467Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.571Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:39.574Z] 10:32:39     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:39.575Z] 10:32:39     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:39.576Z] 10:32:39     INFO - PID 1930 |      12        384 nsComponentManagerImpl
[task 2022-05-21T10:32:39.578Z] 10:32:39     INFO - PID 1930 |       2        288 libfontconfig.so
[task 2022-05-21T10:32:39.578Z] 10:32:39     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:40.649Z] 10:32:40     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:40.652Z] 10:32:40     INFO - PID 1930 | Suppressions used:
[task 2022-05-21T10:32:40.653Z] 10:32:40     INFO - PID 1930 |   count      bytes template
[task 2022-05-21T10:32:40.653Z] 10:32:40     INFO - PID 1930 |       9        280 nsComponentManagerImpl
[task 2022-05-21T10:32:40.653Z] 10:32:40     INFO - PID 1930 |     633      18115 libfontconfig.so
[task 2022-05-21T10:32:40.653Z] 10:32:40     INFO - PID 1930 |      31        189 libglib-2.0.so
[task 2022-05-21T10:32:40.654Z] 10:32:40     INFO - PID 1930 |       2        416 mozJSComponentLoader
[task 2022-05-21T10:32:40.654Z] 10:32:40     INFO - PID 1930 |       1         32 XPCWrappedNativeJSOps
[task 2022-05-21T10:32:40.654Z] 10:32:40     INFO - PID 1930 | -----------------------------------------------------
[task 2022-05-21T10:32:40.738Z] 10:32:40     INFO - Browser exited with return code 0
[task 2022-05-21T10:32:40.738Z] 10:32:40     INFO - ERROR | LeakSanitizer | SUMMARY: AddressSanitizer: 328 byte(s) leaked in 4 allocation(s).
[task 2022-05-21T10:32:40.738Z] 10:32:40     INFO - LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - Allowed depth was 4
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at NS_NewRunnableFunction, mozilla::dom::RemoteWorkerChild::ShutdownOnWorker, operator, std::_Function_handler
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::SupportsThreadSafeWeakPtr, mozilla::dom::RemoteWorkerChild::RemoteWorkerChild, mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived, mozilla::ipc::MessageChannel::DispatchAsyncMessage, mozilla::ipc::MessageChannel::DispatchMessage
[task 2022-05-21T10:32:40.739Z] 10:32:40     INFO - Closing logging queue
[task 2022-05-21T10:32:40.740Z] 10:32:40     INFO - queue closed

Hi Jens. Could you take a look or assign this to someone?
Thank you.

Whiteboard: [stockwell unknown][stockwell needswork:owner] → [stockwell needswork:owner]

There have been 38 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr
Flags: needinfo?(jstutte)
Flags: needinfo?(jstutte) → needinfo?(bugmail)

There have been 33 failures in the last 7 days.

Happens on:

  • linux1804-64-asan-qr opt.

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=381435126&repo=autoland&lineNumber=3709

There have been 38 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr
Flags: needinfo?(bugmail)

Hi Andrew. Did you get a chance to look into this?

Flags: needinfo?(bugmail)

Update:

There have been 38 failures within the last 7 days, all of them on Linux 18.04 x64 WebRender asan opt.
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=384561291&repo=autoland&lineNumber=6045

Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]

I checked the latest 10 logs and the leaks are under different tests so we cannot disable it to reduce the noise.
Andrew, did you get a chance to take a look at this?

Flags: needinfo?(bugmail)
Flags: needinfo?(bugmail)

Update:

There have been 31 failures within the last 7 days on Linux 18.04 x64 WebRender asan opt.

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=387553662&repo=autoland&lineNumber=6048

There are 27 total failures in the last 7 days on linux1804-64-asan-qr opt.
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=389535326&repo=autoland&lineNumber=3914

[task 2022-09-05T22:21:06.770Z] 22:21:06     INFO - TEST-START | dom/serviceworkers/test/test_match_all_client_properties.html
[task 2022-09-05T22:21:07.241Z] 22:21:07     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:21:07.242Z] 22:21:07     INFO - GECKO(2548) | Suppressions used:
[task 2022-09-05T22:21:07.243Z] 22:21:07     INFO - GECKO(2548) |   count      bytes template
[task 2022-09-05T22:21:07.244Z] 22:21:07     INFO - GECKO(2548) |       2        288 libfontconfig.so
[task 2022-09-05T22:21:07.246Z] 22:21:07     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:21:07.392Z] 22:21:07     INFO - GECKO(2548) | =================================================================
[task 2022-09-05T22:21:07.396Z] 22:21:07    ERROR - GECKO(2548) | ==3366==ERROR: LeakSanitizer: detected memory leaks
[task 2022-09-05T22:21:07.400Z] 22:21:07     INFO - GECKO(2548) | Direct leak of 64 byte(s) in 1 object(s) allocated from:
[task 2022-09-05T22:21:07.409Z] 22:21:07     INFO - GECKO(2548) |     #0 0x5650c6d4e19e in malloc /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
[task 2022-09-05T22:21:07.410Z] 22:21:07     INFO - GECKO(2548) |     #1 0x5650c6d92cb5 in moz_xmalloc /builds/worker/checkouts/gecko/memory/mozalloc/mozalloc.cpp:52:15
[task 2022-09-05T22:21:07.411Z] 22:21:07     INFO - GECKO(2548) |     #2 0x7f6bbaa904d3 in operator new /builds/worker/workspace/obj-build/dist/include/mozilla/cxxalloc.h:33:10
[task 2022-09-05T22:21:07.415Z] 22:21:07     INFO - GECKO(2548) |     #3 0x7f6bbaa904d3 in nsresult detail::ProxyRelease<mozilla::dom::RemoteWorkerChild>(char const*, nsIEventTarget*, already_AddRefed<mozilla::dom::RemoteWorkerChild>, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsProxyRelease.h:77:30
[task 2022-09-05T22:21:07.416Z] 22:21:07     INFO - GECKO(2548) |     #4 0x7f6bbaac1c21 in ProxyRelease<mozilla::dom::RemoteWorkerChild> /builds/worker/workspace/obj-build/dist/include/nsProxyRelease.h:93:12
[task 2022-09-05T22:21:07.417Z] 22:21:07     INFO - GECKO(2548) |     #5 0x7f6bbaac1c21 in NS_ProxyRelease<mozilla::dom::RemoteWorkerChild> /builds/worker/workspace/obj-build/dist/include/nsProxyRelease.h:143:10
[task 2022-09-05T22:21:07.418Z] 22:21:07     INFO - GECKO(2548) |     #6 0x7f6bbaac1c21 in operator() /builds/worker/checkouts/gecko/dom/workers/remoteworkers/RemoteWorkerChild.cpp:570:9
[task 2022-09-05T22:21:07.419Z] 22:21:07     INFO - GECKO(2548) |     #7 0x7f6bbaac1c21 in decltype(std::forward<mozilla::dom::RemoteWorkerChild::InitializeOnWorker()::$_0&>(fp)()) fu2::abi_400::detail::invocation::invoke<mozilla::dom::RemoteWorkerChild::InitializeOnWorker()::$_0&>(mozilla::dom::RemoteWorkerChild::InitializeOnWorker()::$_0&) /builds/worker/workspace/obj-build/dist/include/function2/function2.hpp:222:10
[task 2022-09-05T22:21:07.428Z] 22:21:07     INFO - GECKO(2548) |     #8 0x7f6bbaa73c25 in invoke<0UL, fu2::abi_400::detail::type_erasure::erasure<true, fu2::abi_400::detail::config<true, false, fu2::capacity_fixed<16UL, 8UL> >, fu2::abi_400::detail::property<false, false, void ()> > &> /builds/worker/workspace/obj-build/dist/include/function2/function2.hpp:1258:37
[task 2022-09-05T22:21:07.430Z] 22:21:07     INFO - GECKO(2548) |     #9 0x7f6bbaa73c25 in operator() /builds/worker/workspace/obj-build/dist/include/function2/function2.hpp:816:1
[task 2022-09-05T22:21:07.431Z] 22:21:07     INFO - GECKO(2548) |     #10 0x7f6bbaa73c25 in mozilla::dom::WorkerRef::Notify() /builds/worker/checkouts/gecko/dom/workers/WorkerRef.cpp:97:3
[task 2022-09-05T22:21:07.432Z] 22:21:07     INFO - GECKO(2548) |     #11 0x7f6bbaa7410d in mozilla::dom::WeakWorkerRef::Notify() /builds/worker/checkouts/gecko/dom/workers/WorkerRef.cpp:132:14
[task 2022-09-05T22:21:07.433Z] 22:21:07     INFO - GECKO(2548) |     #12 0x7f6bbaa5d1af in mozilla::dom::WorkerPrivate::NotifyWorkerRefs(mozilla::dom::WorkerStatus) /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:4030:16
[task 2022-09-05T22:21:07.434Z] 22:21:07     INFO - GECKO(2548) |     #13 0x7f6bbaa5f370 in mozilla::dom::WorkerPrivate::NotifyInternal(mozilla::dom::WorkerStatus) /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:4647:5
[task 2022-09-05T22:21:07.435Z] 22:21:07     INFO - GECKO(2548) |     #14 0x7f6bbaa75c6e in mozilla::dom::WorkerRunnable::Run() /builds/worker/checkouts/gecko/dom/workers/WorkerRunnable.cpp:377:12
[task 2022-09-05T22:21:07.436Z] 22:21:07     INFO - GECKO(2548) |     #15 0x7f6bbaa5ef57 in mozilla::dom::WorkerPrivate::ProcessAllControlRunnablesLocked() /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:3782:9
[task 2022-09-05T22:21:07.437Z] 22:21:07     INFO - GECKO(2548) |     #16 0x7f6bbaa5da84 in mozilla::dom::WorkerPrivate::DoRunLoop(JSContext*) /builds/worker/checkouts/gecko/dom/workers/WorkerPrivate.cpp:3103:21
[task 2022-09-05T22:21:07.438Z] 22:21:07     INFO - GECKO(2548) |     #17 0x7f6bbaa351ee in mozilla::dom::workerinternals::(anonymous namespace)::WorkerThreadPrimaryRunnable::Run() /builds/worker/checkouts/gecko/dom/workers/RuntimeService.cpp:2042:42
[task 2022-09-05T22:21:07.439Z] 22:21:07     INFO - GECKO(2548) |     #18 0x7f6bb33cd8ce in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1199:16
[task 2022-09-05T22:21:07.440Z] 22:21:07     INFO - GECKO(2548) |     #19 0x7f6bb33d7524 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465:10
[task 2022-09-05T22:21:07.475Z] 22:21:07     INFO - GECKO(2548) |     #20 0x7f6bb49d74c8 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20
[task 2022-09-05T22:21:07.480Z] 22:21:07     INFO - GECKO(2548) |     #21 0x7f6bb4873151 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381:10
[task 2022-09-05T22:21:07.483Z] 22:21:07     INFO - GECKO(2548) |     #22 0x7f6bb4873151 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374:3
[task 2022-09-05T22:21:07.487Z] 22:21:07     INFO - GECKO(2548) |     #23 0x7f6bb4873151 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356:3
[task 2022-09-05T22:21:07.506Z] 22:21:07     INFO - GECKO(2548) |     #24 0x7f6bb33c4a24 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:384:10
[task 2022-09-05T22:21:07.509Z] 22:21:07     INFO - GECKO(2548) |     #25 0x7f6bdb884c0e in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2022-09-05T22:21:07.511Z] 22:21:07     INFO - GECKO(2548) |     #26 0x7f6bdb5b76da in start_thread /tmp/glibc/nptl/pthread_create.c:463

task 2022-09-05T22:21:07.615Z] 22:21:07     INFO - GECKO(2548) |     #17 0x7f6bdb5b76da in start_thread /tmp/glibc/nptl/pthread_create.c:463
[task 2022-09-05T22:21:07.616Z] 22:21:07     INFO - GECKO(2548) | Indirect leak of 24 byte(s) in 1 object(s) allocated from:
[task 2022-09-05T22:21:07.618Z] 22:21:07     INFO - GECKO(2548) |     #0 0x5650c6d4e19e in malloc /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
[task 2022-09-05T22:21:07.628Z] 22:21:07     INFO - GECKO(2548) |     #1 0x5650c6d92cb5 in moz_xmalloc /builds/worker/checkouts/gecko/memory/mozalloc/mozalloc.cpp:52:15
[task 2022-09-05T22:21:07.634Z] 22:21:07     INFO - GECKO(2548) |     #2 0x7f6bbaa91992 in operator new /builds/worker/workspace/obj-build/dist/include/mozilla/cxxalloc.h:33:10
[task 2022-09-05T22:21:07.636Z] 22:21:07     INFO - GECKO(2548) |     #3 0x7f6bbaa91992 in mozilla::SupportsThreadSafeWeakPtr<mozilla::dom::RemoteWorkerChild>::SupportsThreadSafeWeakPtr() /builds/worker/workspace/obj-build/dist/include/mozilla/ThreadSafeWeakPtr.h:148:42
[task 2022-09-05T22:21:07.638Z] 22:21:07     INFO - GECKO(2548) |     #4 0x7f6bbaa91560 in mozilla::dom::RemoteWorkerChild::RemoteWorkerChild(mozilla::dom::RemoteWorkerData const&) /builds/worker/checkouts/gecko/dom/workers/remoteworkers/RemoteWorkerChild.cpp:260:20
[task 2022-09-05T22:21:07.640Z] 22:21:07     INFO - GECKO(2548) |     #5 0x7f6bb4952be1 in mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild(mozilla::dom::RemoteWorkerData const&) /builds/worker/checkouts/gecko/ipc/glue/BackgroundChildImpl.cpp:285:46
[task 2022-09-05T22:21:07.642Z] 22:21:07     INFO - GECKO(2548) |     #6 0x7f6bb4a549b0 in mozilla::ipc::PBackgroundChild::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PBackgroundChild.cpp:6834:49
[task 2022-09-05T22:21:07.643Z] 22:21:07     INFO - GECKO(2548) |     #7 0x7f6bb49d027d in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) /builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp:1755:25
[task 2022-09-05T22:21:07.645Z] 22:21:07     INFO - GECKO(2548) |     #8 0x7f6bb49cdd50 in mozilla::ipc::MessageChannel::DispatchMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::UniquePtr<IPC::Message, mozilla::DefaultDelete<IPC::Message> >) /builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp:1680:9
[task 2022-09-05T22:21:07.647Z] 22:21:07     INFO - GECKO(2548) |     #9 0x7f6bb49ce6f2 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::ipc::MessageChannel::MessageTask&) /builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp:1480:3
[task 2022-09-05T22:21:07.649Z] 22:21:07     INFO - GECKO(2548) |     #10 0x7f6bb49cf46e in mozilla::ipc::MessageChannel::MessageTask::Run() /builds/worker/checkouts/gecko/ipc/glue/MessageChannel.cpp:1578:14
[task 2022-09-05T22:21:07.651Z] 22:21:07     INFO - GECKO(2548) |     #11 0x7f6bb33cd8ce in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1199:16
[task 2022-09-05T22:21:07.655Z] 22:21:07     INFO - GECKO(2548) |     #12 0x7f6bb33d7524 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465:10
[task 2022-09-05T22:21:07.657Z] 22:21:07     INFO - GECKO(2548) |     #13 0x7f6bb49d74c8 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20
[task 2022-09-05T22:21:07.658Z] 22:21:07     INFO - GECKO(2548) |     #14 0x7f6bb4873151 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381:10
[task 2022-09-05T22:21:07.660Z] 22:21:07     INFO - GECKO(2548) |     #15 0x7f6bb4873151 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374:3
[task 2022-09-05T22:21:07.661Z] 22:21:07     INFO - GECKO(2548) |     #16 0x7f6bb4873151 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356:3
[task 2022-09-05T22:21:07.663Z] 22:21:07     INFO - GECKO(2548) |     #17 0x7f6bb33c4a24 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:384:10
[task 2022-09-05T22:21:07.664Z] 22:21:07     INFO - GECKO(2548) |     #18 0x7f6bdb884c0e in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2022-09-05T22:21:07.668Z] 22:21:07     INFO - GECKO(2548) |     #19 0x7f6bdb5b76da in start_thread /tmp/glibc/nptl/pthread_create.c:463
[task 2022-09-05T22:21:07.670Z] 22:21:07     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:21:07.671Z] 22:21:07     INFO - GECKO(2548) | Suppressions used:
[task 2022-09-05T22:21:07.673Z] 22:21:07     INFO - GECKO(2548) |   count      bytes template
[task 2022-09-05T22:21:07.674Z] 22:21:07     INFO - GECKO(2548) |       5        832 libc.so
[task 2022-09-05T22:21:07.675Z] 22:21:07     INFO - GECKO(2548) |       2        288 libfontconfig.so
[task 2022-09-05T22:21:07.677Z] 22:21:07     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:21:07.678Z] 22:21:07     INFO - GECKO(2548) | SUMMARY: AddressSanitizer: 328 byte(s) leaked in 4 allocation(s).
[task 2022-09-05T22:21:08.037Z] 22:21:08     INFO - GECKO(2548) | MatchAllPropertiesWorker:Start
[task 2022-09-05T22:21:08.138Z] 22:21:08     INFO - GECKO(2548) | JavaScript error: resource://gre/actors/BrowserElementParent.jsm, line 24: TypeError: can't access property "ownerGlobal", browser is null
[task 2022-09-05T22:21:08.336Z] 22:21:08     INFO - GECKO(2548) | MatchAllPropertiesWorker:Start
[task 2022-09-05T22:21:08.509Z] 22:21:08     INFO - GECKO(2548) | MEMORY STAT | vsize 20974620MB | residentFast 795MB
[task 2022-09-05T22:21:08.600Z] 22:21:08     INFO - TEST-OK | dom/serviceworkers/test/test_match_all_client_properties.html | took 1829ms

[task 2022-09-05T22:22:57.954Z] 22:22:57     INFO - GECKO(2548) | Suppressions used:
[task 2022-09-05T22:22:57.954Z] 22:22:57     INFO - GECKO(2548) |   count      bytes template
[task 2022-09-05T22:22:57.955Z] 22:22:57     INFO - GECKO(2548) |       2        288 libfontconfig.so
[task 2022-09-05T22:22:57.955Z] 22:22:57     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:22:58.078Z] 22:22:58     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:22:58.078Z] 22:22:58     INFO - GECKO(2548) | Suppressions used:
[task 2022-09-05T22:22:58.078Z] 22:22:58     INFO - GECKO(2548) |   count      bytes template
[task 2022-09-05T22:22:58.078Z] 22:22:58     INFO - GECKO(2548) |       2        288 libfontconfig.so
[task 2022-09-05T22:22:58.078Z] 22:22:58     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:22:59.313Z] 22:22:59     INFO - GECKO(2548) | Waiting in WillDestroyCurrentMessageLoop for pid 2667
[task 2022-09-05T22:22:59.327Z] 22:22:59     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:22:59.328Z] 22:22:59     INFO - GECKO(2548) | Suppressions used:
[task 2022-09-05T22:22:59.329Z] 22:22:59     INFO - GECKO(2548) |   count      bytes template
[task 2022-09-05T22:22:59.330Z] 22:22:59     INFO - GECKO(2548) |      14        448 nsComponentManagerImpl
[task 2022-09-05T22:22:59.330Z] 22:22:59     INFO - GECKO(2548) |       2        288 libfontconfig.so
[task 2022-09-05T22:22:59.331Z] 22:22:59     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:23:00.600Z] 22:23:00     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:23:00.601Z] 22:23:00     INFO - GECKO(2548) | Suppressions used:
[task 2022-09-05T22:23:00.601Z] 22:23:00     INFO - GECKO(2548) |   count      bytes template
[task 2022-09-05T22:23:00.602Z] 22:23:00     INFO - GECKO(2548) |       1         32 nsComponentManagerImpl
[task 2022-09-05T22:23:00.603Z] 22:23:00     INFO - GECKO(2548) |     633      18115 libfontconfig.so
[task 2022-09-05T22:23:00.604Z] 22:23:00     INFO - GECKO(2548) |       2      16393 libglib-2.0.so
[task 2022-09-05T22:23:00.605Z] 22:23:00     INFO - GECKO(2548) | -----------------------------------------------------
[task 2022-09-05T22:23:00.705Z] 22:23:00     INFO - TEST-INFO | Main app process: exit 0
[task 2022-09-05T22:23:00.705Z] 22:23:00     INFO - TEST-INFO | LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2022-09-05T22:23:00.705Z] 22:23:00     INFO - TEST-INFO | LeakSanitizer | This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2022-09-05T22:23:00.705Z] 22:23:00    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::SupportsThreadSafeWeakPtr, mozilla::dom::RemoteWorkerChild::RemoteWorkerChild, mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived
[task 2022-09-05T22:23:00.705Z] 22:23:00    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at already_AddRefed, mozilla::dom::RemoteWorkerChild::ShutdownOnWorker, operator, decltype
[task 2022-09-05T22:23:00.705Z] 22:23:00    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived, mozilla::ipc::MessageChannel::DispatchAsyncMessage, mozilla::ipc::MessageChannel::DispatchMessage
[task 2022-09-05T22:23:00.705Z] 22:23:00    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator
[task 2022-09-05T22:23:00.705Z] 22:23:00     INFO - runtests.py | Application ran for: 0:02:55.700798
[task 2022-09-05T22:23:00.705Z] 22:23:00     INFO - zombiecheck | Reading PID log: /tmp/tmpwkftlev5pidlog
[task 2022-09-05T22:23:00.705Z] 22:23:00     INFO - ==> process 2548 launched child process 2565
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2617
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2639
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2666
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2667
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2669
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2765
[task 2022-09-05T22:23:00.709Z] 22:23:00     INFO - ==> process 2548 launched child process 2766

Andrew, can you please take a look or assign it to someone?
Thank you!

There have been 35 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr

There have been 39 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr
[task 2022-12-16T04:00:00.590Z] 04:00:00     INFO - GECKO(2943) | observe@resource://gre/modules/AsyncShutdown.jsm:572:16
[task 2022-12-16T04:00:00.591Z] 04:00:00     INFO - GECKO(2943) | JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2022-12-16T04:00:00.592Z] 04:00:00     INFO - GECKO(2943) | JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2022-12-16T04:00:00.594Z] 04:00:00     INFO - GECKO(2943) | JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2022-12-16T04:00:00.595Z] 04:00:00     INFO - GECKO(2943) | JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2022-12-16T04:00:00.597Z] 04:00:00     INFO - GECKO(2943) | JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2022-12-16T04:00:01.703Z] 04:00:01     INFO - GECKO(2943) | -----------------------------------------------------
[task 2022-12-16T04:00:01.705Z] 04:00:01     INFO - GECKO(2943) | Suppressions used:
[task 2022-12-16T04:00:01.706Z] 04:00:01     INFO - GECKO(2943) |   count      bytes template
[task 2022-12-16T04:00:01.707Z] 04:00:01     INFO - GECKO(2943) |      14        448 nsComponentManagerImpl
[task 2022-12-16T04:00:01.708Z] 04:00:01     INFO - GECKO(2943) |       2        288 libfontconfig.so
[task 2022-12-16T04:00:01.709Z] 04:00:01     INFO - GECKO(2943) | -----------------------------------------------------
[task 2022-12-16T04:00:01.758Z] 04:00:01     INFO - GECKO(2943) | [Parent 2943, IPC I/O Parent] WARNING: Process 3075 may be hanging at shutdown; will wait for up to 8000ms: file /builds/worker/checkouts/gecko/ipc/chromium/src/chrome/common/process_watcher_posix_sigchld.cc:175
[task 2022-12-16T04:00:03.182Z] 04:00:03     INFO - GECKO(2943) | -----------------------------------------------------
[task 2022-12-16T04:00:03.183Z] 04:00:03     INFO - GECKO(2943) | Suppressions used:
[task 2022-12-16T04:00:03.184Z] 04:00:03     INFO - GECKO(2943) |   count      bytes template
[task 2022-12-16T04:00:03.184Z] 04:00:03     INFO - GECKO(2943) |       1         32 nsComponentManagerImpl
[task 2022-12-16T04:00:03.185Z] 04:00:03     INFO - GECKO(2943) |     633      18115 libfontconfig.so
[task 2022-12-16T04:00:03.185Z] 04:00:03     INFO - GECKO(2943) |       2      16393 libglib-2.0.so
[task 2022-12-16T04:00:03.186Z] 04:00:03     INFO - GECKO(2943) | -----------------------------------------------------
[task 2022-12-16T04:00:03.271Z] 04:00:03     INFO - TEST-INFO | Main app process: exit 0
[task 2022-12-16T04:00:03.272Z] 04:00:03     INFO - TEST-INFO | LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2022-12-16T04:00:03.272Z] 04:00:03     INFO - TEST-INFO | LeakSanitizer | This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2022-12-16T04:00:03.272Z] 04:00:03    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator
[task 2022-12-16T04:00:03.272Z] 04:00:03    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::SupportsThreadSafeWeakPtr, mozilla::dom::RemoteWorkerChild::RemoteWorkerChild, mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived
[task 2022-12-16T04:00:03.272Z] 04:00:03    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived, mozilla::ipc::MessageChannel::DispatchAsyncMessage, mozilla::ipc::MessageChannel::DispatchMessage
[task 2022-12-16T04:00:03.272Z] 04:00:03     INFO - runtests.py | Application ran for: 0:02:38.166359

There have been 48 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr
[task 2023-01-20T01:33:33.435Z] 01:33:33     INFO - GECKO(5517) | _checkPendingPings@resource://gre/modules/TelemetrySend.sys.mjs:863:17
[task 2023-01-20T01:33:33.436Z] 01:33:33     INFO - GECKO(5517) | setup@resource://gre/modules/TelemetrySend.sys.mjs:803:18
[task 2023-01-20T01:33:33.437Z] 01:33:33     INFO - GECKO(5517) | setup@resource://gre/modules/TelemetrySend.sys.mjs:241:30
[task 2023-01-20T01:33:33.438Z] 01:33:33     INFO - GECKO(5517) | setupTelemetry/this._delayedInitTask<@resource://gre/modules/TelemetryControllerParent.sys.mjs:829:36
[task 2023-01-20T01:33:33.439Z] 01:33:33     INFO - GECKO(5517) | observe@resource://gre/modules/AsyncShutdown.sys.mjs:576:16
[task 2023-01-20T01:33:33.440Z] 01:33:33     INFO - GECKO(5517) | JavaScript error: resource://gre/modules/AsyncShutdown.sys.mjs, line 727: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2023-01-20T01:33:33.441Z] 01:33:33     INFO - GECKO(5517) | JavaScript error: resource://gre/modules/AsyncShutdown.sys.mjs, line 727: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2023-01-20T01:33:33.442Z] 01:33:33     INFO - GECKO(5517) | JavaScript error: resource://gre/modules/AsyncShutdown.sys.mjs, line 727: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2023-01-20T01:33:33.445Z] 01:33:33     INFO - GECKO(5517) | JavaScript error: resource://gre/modules/AsyncShutdown.sys.mjs, line 727: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2023-01-20T01:33:33.446Z] 01:33:33     INFO - GECKO(5517) | JavaScript error: resource://gre/modules/AsyncShutdown.sys.mjs, line 727: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
[task 2023-01-20T01:33:33.503Z] 01:33:33     INFO - GECKO(5517) | -----------------------------------------------------
[task 2023-01-20T01:33:33.503Z] 01:33:33     INFO - GECKO(5517) | Suppressions used:
[task 2023-01-20T01:33:33.503Z] 01:33:33     INFO - GECKO(5517) |   count      bytes template
[task 2023-01-20T01:33:33.503Z] 01:33:33     INFO - GECKO(5517) |       2        288 libfontconfig.so
[task 2023-01-20T01:33:33.504Z] 01:33:33     INFO - GECKO(5517) | -----------------------------------------------------
[task 2023-01-20T01:33:34.682Z] 01:33:34     INFO - GECKO(5517) | -----------------------------------------------------
[task 2023-01-20T01:33:34.682Z] 01:33:34     INFO - GECKO(5517) | Suppressions used:
[task 2023-01-20T01:33:34.682Z] 01:33:34     INFO - GECKO(5517) |   count      bytes template
[task 2023-01-20T01:33:34.682Z] 01:33:34     INFO - GECKO(5517) |      14        448 nsComponentManagerImpl
[task 2023-01-20T01:33:34.682Z] 01:33:34     INFO - GECKO(5517) |       2        288 libfontconfig.so
[task 2023-01-20T01:33:34.682Z] 01:33:34     INFO - GECKO(5517) | -----------------------------------------------------
[task 2023-01-20T01:33:35.952Z] 01:33:35     INFO - GECKO(5517) | -----------------------------------------------------
[task 2023-01-20T01:33:35.954Z] 01:33:35     INFO - GECKO(5517) | Suppressions used:
[task 2023-01-20T01:33:35.955Z] 01:33:35     INFO - GECKO(5517) |   count      bytes template
[task 2023-01-20T01:33:35.956Z] 01:33:35     INFO - GECKO(5517) |       1         32 nsComponentManagerImpl
[task 2023-01-20T01:33:35.957Z] 01:33:35     INFO - GECKO(5517) |     633      18115 libfontconfig.so
[task 2023-01-20T01:33:35.958Z] 01:33:35     INFO - GECKO(5517) |       2      16393 libglib-2.0.so
[task 2023-01-20T01:33:35.958Z] 01:33:35     INFO - GECKO(5517) | -----------------------------------------------------
[task 2023-01-20T01:33:36.045Z] 01:33:36     INFO - TEST-INFO | Main app process: exit 0
[task 2023-01-20T01:33:36.045Z] 01:33:36     INFO - TEST-INFO | LeakSanitizer | To show the addresses of leaked objects add report_objects=1 to LSAN_OPTIONS
[task 2023-01-20T01:33:36.046Z] 01:33:36     INFO - TEST-INFO | LeakSanitizer | This can be done in testing/mozbase/mozrunner/mozrunner/utils.py
[task 2023-01-20T01:33:36.047Z] 01:33:36    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at detail::ProxyRelease, ProxyRelease, NS_ProxyRelease, operator
[task 2023-01-20T01:33:36.048Z] 01:33:36    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived, mozilla::ipc::MessageChannel::DispatchAsyncMessage, mozilla::ipc::MessageChannel::DispatchMessage
[task 2023-01-20T01:33:36.048Z] 01:33:36    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at mozilla::SupportsThreadSafeWeakPtr, mozilla::dom::RemoteWorkerChild::RemoteWorkerChild, mozilla::ipc::BackgroundChildImpl::AllocPRemoteWorkerChild, mozilla::ipc::PBackgroundChild::OnMessageReceived
[task 2023-01-20T01:33:36.049Z] 01:33:36    ERROR - TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at NS_NewRunnableFunction, mozilla::dom::RemoteWorkerChild::ShutdownOnWorker, operator, decltype
[task 2023-01-20T01:33:36.049Z] 01:33:36     INFO - runtests.py | Application ran for: 0:02:40.350680

There have been 43 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr

Jan, could you have a look over these frequent leaks? There are 163 total failures in the last 30 days and it's on the disable recommended list: https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2023-01-04&endday=2023-02-03&tree=trunk&failurehash=all&bug=1743092
Same is the case for Bug 1717657 and Bug 1703952.

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=404449667&repo=autoland

Flags: needinfo?(jvarga)

There have been 43 total failures in the last 7 days, recent failure log.
Affected platforms are:

  • linux1804-64-asan-qr

Jari, could you have a look over or redirect these frequent leaks? Together with Bug 1703952 and Bug 1717657.

Flags: needinfo?(jvarga)
Flags: needinfo?(jjalkanen)
Flags: needinfo?(bugmail)

Update:

There have been 42 failures within the last 7 days, all of them on Linux 18.04 x64 WebRender asan opt.

Recent log: https://treeherder.mozilla.org/logviewer?job_id=406898012&repo=autoland&lineNumber=6759

See Also: → 1818895
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Depends on: 1775784
Whiteboard: [stockwell disable-recommended]
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]

Update:

There have been 75 failures within the last 7 days, all of them on Linux 18.04 x64 WebRender asan opt.

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=408963709&repo=mozilla-central&lineNumber=7786

This should go away after Bug 1775784 landed.

Flags: needinfo?(jjalkanen)
Whiteboard: [stockwell disable-recommended]

(In reply to Natalia Csoregi [:nataliaCs] from comment #96)

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=408963709&repo=mozilla-central&lineNumber=7786

This log at least is definitely one that was fixed by the bug 1775784 landing as it's a leak from RemoteWorkerChild::ShutdownOnWorker which is a method that no longer exists, so ideally that should take care of all of the signatures.

Whiteboard: [stockwell disable-recommended]

(In reply to Cosmin Sabou [:CosminS] from comment #97)

This should go away after Bug 1775784 landed.

Seems confirmed by the data.

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.