1791335 - Intermittent SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:261:3 in MOZ_Crash | Waiting in WillDestroyCurrentMessageLoop

Status: RESOLVED WORKSFORME

(Core :: DOM: Core & HTML, defect)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=388528983&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/dhqZuQ4JQwqHzQeEaiRFDw/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/dhqZuQ4JQwqHzQeEaiRFDw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2022-08-25T23:34:21.631Z] 23:34:21     INFO - REFTEST INFO | Slowest test took 38301ms (file:///builds/worker/workspace/build/tests/reftest/tests/dom/svg/crashtests/1507961-1.html)
[task 2022-08-25T23:34:21.632Z] 23:34:21     INFO - REFTEST INFO | Total canvas count = 0
[task 2022-08-25T23:34:21.682Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619000eeb680 - BlockShutdown: NotifyImpendingShutdown.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3627
[task 2022-08-25T23:34:21.682Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619000eeb680 - BlockShutdown: RequestContentJSInterrupt.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3633
[task 2022-08-25T23:34:21.688Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619001259380 - BlockShutdown: NotifyImpendingShutdown.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3627
[task 2022-08-25T23:34:21.690Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619001259380 - BlockShutdown: RequestContentJSInterrupt.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3633
[task 2022-08-25T23:34:21.691Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619000e89e80 - BlockShutdown: NotifyImpendingShutdown.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3627
[task 2022-08-25T23:34:21.692Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619000e89e80 - BlockShutdown: RequestContentJSInterrupt.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3633
[task 2022-08-25T23:34:21.694Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=6190050b9a80 - BlockShutdown: NotifyImpendingShutdown.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3627
[task 2022-08-25T23:34:21.695Z] 23:34:21     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=6190050b9a80 - BlockShutdown: RequestContentJSInterrupt.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3633
[task 2022-08-25T23:34:22.152Z] 23:34:22     INFO - 1661470462151	Marionette	TRACE	Received observer notification quit-application
[task 2022-08-25T23:34:22.155Z] 23:34:22     INFO - 1661470462153	Marionette	INFO	Stopped listening on port 2828
[task 2022-08-25T23:34:22.161Z] 23:34:22     INFO - 1661470462160	Marionette	DEBUG	Marionette stopped listening
[task 2022-08-25T23:34:24.112Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619001259380 - BlockShutdown: CanSend.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3672
[task 2022-08-25T23:34:24.117Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619001259380 - ShutDownProcess: Sent shutdown message.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1804
[task 2022-08-25T23:34:24.132Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=6190050b9a80 - BlockShutdown: CanSend.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3672
[task 2022-08-25T23:34:24.137Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=6190050b9a80 - ShutDownProcess: Sent shutdown message.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1804
[task 2022-08-25T23:34:24.187Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619001259380 - ShutDownProcess: Closing channel.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1844
[task 2022-08-25T23:34:24.188Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=619001259380 - RemoveShutdownBlockers: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3787
[task 2022-08-25T23:34:24.199Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=6190050b9a80 - ShutDownProcess: Closing channel.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1844
[task 2022-08-25T23:34:24.200Z] 23:34:24     INFO - [Parent 1535, Main Thread] WARNING: ContentParent: id=6190050b9a80 - RemoveShutdownBlockers: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3787
[task 2022-08-25T23:34:37.841Z] 23:34:37     INFO - Waiting in WillDestroyCurrentMessageLoop for pid 1781
[task 2022-08-25T23:39:41.077Z] 23:39:41     INFO - RunWatchdog: Mainthread nested event loops during hang:
[task 2022-08-25T23:39:41.078Z] 23:39:41     INFO -  --- (no nested event loop active)
[task 2022-08-25T23:39:41.079Z] 23:39:41     INFO - AddressSanitizer:DEADLYSIGNAL
[task 2022-08-25T23:39:41.081Z] 23:39:41     INFO - =================================================================
[task 2022-08-25T23:39:41.082Z] 23:39:41    ERROR - ==1535==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x7f7492f6a9b0 bp 0x7f73d4dc8e10 sp 0x7f73d4dc8d40 T302)
[task 2022-08-25T23:39:41.082Z] 23:39:41     INFO - ==1535==The signal is caused by a WRITE memory access.
[task 2022-08-25T23:39:41.083Z] 23:39:41     INFO - ==1535==Hint: address points to the zero page.
[task 2022-08-25T23:39:42.537Z] 23:39:42     INFO -     #0 0x7f7492f6a9b0 in MOZ_Crash /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:261:3
[task 2022-08-25T23:39:42.539Z] 23:39:42     INFO -     #1 0x7f7492f6a9b0 in mozilla::(anonymous namespace)::RunWatchdog(void*) /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:244:5
[task 2022-08-25T23:39:42.542Z] 23:39:42     INFO -     #2 0x7f74adcf1c0e in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2022-08-25T23:39:42.544Z] 23:39:42     INFO -     #3 0x7f74ada236da in start_thread /tmp/glibc/nptl/pthread_create.c:463
[task 2022-08-25T23:39:42.545Z] 23:39:42     INFO -     #4 0x7f74aca01a3e in __clone /tmp/glibc/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
[task 2022-08-25T23:39:42.547Z] 23:39:42     INFO - AddressSanitizer can not provide additional info.
[task 2022-08-25T23:39:42.548Z] 23:39:42     INFO - SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:261:3 in MOZ_Crash
[task 2022-08-25T23:39:42.548Z] 23:39:42     INFO - Thread T302 (Shutdow~minator) created by T0 here:
[task 2022-08-25T23:39:42.673Z] 23:39:42     INFO -     #0 0x55f27d9026fc in __interceptor_pthread_create /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:208:3
[task 2022-08-25T23:39:42.673Z] 23:39:42     INFO -     #1 0x7f74adce1cbc in _PR_CreateThread /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:458:14
[task 2022-08-25T23:39:42.673Z] 23:39:42     INFO -     #2 0x7f74adcd305e in PR_CreateThread /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:533:12
[task 2022-08-25T23:39:42.673Z] 23:39:42     INFO -     #3 0x7f7492f69eb2 in CreateSystemThread /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:113:7
[task 2022-08-25T23:39:42.673Z] 23:39:42     INFO -     #4 0x7f7492f69eb2 in mozilla::nsTerminator::StartWatchdog() /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:469:7
[task 2022-08-25T23:39:42.673Z] 23:39:42     INFO -     #5 0x7f7492f6a2f0 in Start /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:381:3
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #6 0x7f7492f6a2f0 in mozilla::nsTerminator::AdvancePhase(mozilla::ShutdownPhase) /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:530:5
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #7 0x7f74860e4b39 in mozilla::AppShutdown::AdvanceShutdownPhaseInternal(mozilla::ShutdownPhase, bool, char16_t const*, nsCOMPtr<nsISupports> const&) /builds/worker/checkouts/gecko/xpcom/base/AppShutdown.cpp:354:18
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #8 0x7f7492d9b5d6 in nsAppStartup::Quit(unsigned int, int, bool*) /builds/worker/checkouts/gecko/toolkit/components/startup/nsAppStartup.cpp:484:5
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #9 0x7f7486387bf5 in NS_InvokeByIndex /builds/worker/checkouts/gecko/xpcom/reflect/xptcall/md/unix/xptcinvoke_asm_x86_64_unix.S:101
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #10 0x7f7487c411b1 in Invoke /builds/worker/checkouts/gecko/js/xpconnect/src/XPCWrappedNative.cpp:1626:10
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #11 0x7f7487c411b1 in Call /builds/worker/checkouts/gecko/js/xpconnect/src/XPCWrappedNative.cpp:1179:19
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #12 0x7f7487c411b1 in XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) /builds/worker/checkouts/gecko/js/xpconnect/src/XPCWrappedNative.cpp:1125:23
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #13 0x7f7487c461ac in XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) /builds/worker/checkouts/gecko/js/xpconnect/src/XPCWrappedNativeJSOps.cpp:965:10
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #14 0x7f7494a73d03 in CallJSNative /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:458:13
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #15 0x7f7494a73d03 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:546:12
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #16 0x7f7494a626bc in InternalCall /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:613:10
[task 2022-08-25T23:39:42.674Z] 23:39:42     INFO -     #17 0x7f7494a626bc in CallFromStack /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:618:10
[task 2022-08-25T23:39:42.677Z] 23:39:42     INFO -     #18 0x7f7494a626bc in Interpret(JSContext*, js::RunState&) /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:3374:16
[task 2022-08-25T23:39:42.679Z] 23:39:42     INFO -     #19 0x7f7494a47cbe in js::RunScript(JSContext*, js::RunState&) /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:430:13
[task 2022-08-25T23:39:42.681Z] 23:39:42     INFO -     #20 0x7f7494a73e25 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason) /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:578:13
[task 2022-08-25T23:39:42.683Z] 23:39:42     INFO -     #21 0x7f7494a758ce in InternalCall /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:613:10
[task 2022-08-25T23:39:42.684Z] 23:39:42     INFO -     #22 0x7f7494a758ce in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason) /builds/worker/checkouts/gecko/js/src/vm/Interpreter.cpp:645:8
[task 2022-08-25T23:39:42.686Z] 23:39:42     INFO -     #23 0x7f749343ef34 in JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) /builds/worker/checkouts/gecko/js/src/vm/CallAndConstruct.cpp:53:10
[task 2022-08-25T23:39:42.687Z] 23:39:42     INFO -     #24 0x7f7487c32f45 in nsXPCWrappedJS::CallMethod(unsigned short, nsXPTMethodInfo const*, nsXPTCMiniVariant*) /builds/worker/checkouts/gecko/js/xpconnect/src/XPCWrappedJSClass.cpp:981:17
[task 2022-08-25T23:39:42.689Z] 23:39:42     INFO -     #25 0x7f7486389632 in PrepareAndDispatch /builds/worker/checkouts/gecko/xpcom/reflect/xptcall/md/unix/xptcstubs_x86_64_linux.cpp:115:37
[task 2022-08-25T23:39:42.690Z] 23:39:42     INFO -     #26 0x7f7486388382 in SharedStub xptcstubs_x86_64_linux.cpp
[task 2022-08-25T23:39:42.691Z] 23:39:42     INFO -     #27 0x7f748634ebe2 in mozilla::RunnableTask::Run() /builds/worker/checkouts/gecko/xpcom/threads/TaskController.cpp:538:16
[task 2022-08-25T23:39:42.692Z] 23:39:42     INFO -     #28 0x7f748630f2ed in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) /builds/worker/checkouts/gecko/xpcom/threads/TaskController.cpp:851:26
[task 2022-08-25T23:39:42.694Z] 23:39:42     INFO -     #29 0x7f748630c458 in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) /builds/worker/checkouts/gecko/xpcom/threads/TaskController.cpp:683:15
[task 2022-08-25T23:39:42.695Z] 23:39:42     INFO -     #30 0x7f748630cb80 in mozilla::TaskController::ProcessPendingMTTask(bool) /builds/worker/checkouts/gecko/xpcom/threads/TaskController.cpp:461:36
[task 2022-08-25T23:39:42.696Z] 23:39:42     INFO -     #31 0x7f7486357c11 in operator() /builds/worker/checkouts/gecko/xpcom/threads/TaskController.cpp:187:37
[task 2022-08-25T23:39:42.698Z] 23:39:42     INFO -     #32 0x7f7486357c11 in mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_0>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:531:5
[task 2022-08-25T23:39:42.699Z] 23:39:42     INFO -     #33 0x7f7486330647 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1205:16
[task 2022-08-25T23:39:42.700Z] 23:39:42     INFO -     #34 0x7f748633aac4 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465:10
[task 2022-08-25T23:39:42.701Z] 23:39:42     INFO -     #35 0x7f748793afe8 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:85:21
[task 2022-08-25T23:39:42.702Z] 23:39:42     INFO -     #36 0x7f74877d7bf1 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381:10
[task 2022-08-25T23:39:42.703Z] 23:39:42     INFO -     #37 0x7f74877d7bf1 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374:3
[task 2022-08-25T23:39:42.704Z] 23:39:42     INFO -     #38 0x7f74877d7bf1 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356:3
[task 2022-08-25T23:39:42.705Z] 23:39:42     INFO -     #39 0x7f748e294c37 in nsBaseAppShell::Run() /builds/worker/checkouts/gecko/widget/nsBaseAppShell.cpp:150:27
[task 2022-08-25T23:39:42.706Z] 23:39:42     INFO -     #40 0x7f7492d9a5f7 in nsAppStartup::Run() /builds/worker/checkouts/gecko/toolkit/components/startup/nsAppStartup.cpp:295:30
[task 2022-08-25T23:39:42.707Z] 23:39:42     INFO -     #41 0x7f7492fc61fe in XREMain::XRE_mainRun() /builds/worker/checkouts/gecko/toolkit/xre/nsAppRunner.cpp:5725:22
[task 2022-08-25T23:39:42.708Z] 23:39:42     INFO -     #42 0x7f7492fc7f7e in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/checkouts/gecko/toolkit/xre/nsAppRunner.cpp:5919:8
[task 2022-08-25T23:39:42.709Z] 23:39:42     INFO -     #43 0x7f7492fc8cfb in XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/checkouts/gecko/toolkit/xre/nsAppRunner.cpp:5975:21
[task 2022-08-25T23:39:42.710Z] 23:39:42     INFO -     #44 0x55f27d957a39 in do_main(int, char**, char**) /builds/worker/checkouts/gecko/browser/app/nsBrowserApp.cpp:229:22
[task 2022-08-25T23:39:42.711Z] 23:39:42     INFO -     #45 0x55f27d956d57 in main /builds/worker/checkouts/gecko/browser/app/nsBrowserApp.cpp:433:16
[task 2022-08-25T23:39:42.712Z] 23:39:42     INFO -     #46 0x7f74ac901b96 in __libc_start_main /tmp/glibc/csu/../csu/libc-start.c:310
[task 2022-08-25T23:39:42.713Z] 23:39:42     INFO - ==1535==ABORTING
[task 2022-08-25T23:45:52.678Z] 23:45:52    ERROR - REFTEST ERROR | xpcom/string/crashtests/1113005.html (finished) | application timed out after 370 seconds with no output
[task 2022-08-25T23:45:52.678Z] 23:45:52    ERROR - REFTEST ERROR | Force-terminating active process(es).

Comment 1

[Intermittent Failures Robot]

10 failures in 3023 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5
• mozilla-central: 5

Platform and build breakdown:

• linux1804-64-asan-qr: 10
  • opt: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-09-12&endday=2022-09-18&tree=all

Comment 2

[Natalia Csoregi [:nataliaCs]]

First occurrence so far: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&tochange=eecfa46043c20f261e0f6e825609fcd658d49ea3&fromchange=b9f534734c7b19579407a73b61cb90b4acbdfe56&searchStr=linux%2C18.04%2Cx64%2Cwebrender%2Casan%2Copt%2Creftests%2Cwithout%2Cfission%2Cenabled%2Ctest-linux1804-64-asan-qr%2Fopt-crashtest-nofis%2Cc&selectedTaskRun=MW21RRVfTomJDdrUgijLvw.0

Comment 3

[Andrew McCreight [:mccr8]]

It looks like this is the shutdown terminator firing off, in ASan builds. The same basic issue as bug 1684441.

Comment 4

[Andrew McCreight [:mccr8]]

Most or all of the recent failures are happening after running xpcom/string/crashtests, which is a bit odd.

Comment 5

[Andrew McCreight [:mccr8]]

Oh, but it looks like maybe crash tests don't do chunking by directory, so who knows what is at fault here.

Comment 6

[Intermittent Failures Robot]

36 failures in 2909 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #23 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 16
• mozilla-central: 7
• try: 5
• mozilla-esr102: 2
• mozilla-beta: 3
• mozilla-release: 3

Platform and build breakdown:

• linux1804-64-asan-qr: 36
  • opt: 36

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-09-19&endday=2022-09-25&tree=all

Comment 7

[Intermittent Failures Robot]

22 failures in 2041 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #39 most frequent failure this week.

Repository breakdown:

• mozilla-central: 10
• autoland: 9
• try: 3

Platform and build breakdown:

• linux1804-64-asan-qr: 22
  • opt: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-09-26&endday=2022-10-02&tree=all

Comment 8

[Intermittent Failures Robot]

24 failures in 2449 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #44 most frequent failure this week.

Repository breakdown:

• autoland: 20
• mozilla-central: 4

Platform and build breakdown:

• linux1804-64-asan-qr: 24
  • opt: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-10-03&endday=2022-10-09&tree=all

Comment 9

[Intermittent Failures Robot]

30 failures in 3034 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #31 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 21
• mozilla-esr102: 1
• mozilla-central: 8

Platform and build breakdown:

• linux1804-64-asan-qr: 30
  • opt: 30

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-10-10&endday=2022-10-16&tree=all

Comment 10

[Intermittent Failures Robot]

32 failures in 2793 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #33 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 23
• mozilla-beta: 1
• mozilla-central: 7
• try: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 32
  • opt: 32

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-10-17&endday=2022-10-23&tree=all

Comment 11

[Intermittent Failures Robot]

22 failures in 3743 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 13
• mozilla-central: 8
• try: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 22
  • opt: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-10-24&endday=2022-10-30&tree=all

Comment 12

[Intermittent Failures Robot]

30 failures in 3626 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 19
• mozilla-central: 11

Platform and build breakdown:

• linux1804-64-asan-qr: 29
  • opt: 29
• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-10-31&endday=2022-11-06&tree=all

Comment 13

[Intermittent Failures Robot]

40 failures in 3445 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #15 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 30
• try: 2
• mozilla-central: 8

Platform and build breakdown:

• linux1804-64-asan-qr: 40
  • opt: 40

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-11-07&endday=2022-11-13&tree=all

Comment 14

[Marian-Vasile Laza]

There have been 40 total failures in the last 7 days, recent failure log.
Affected platforms are:

• linux1804-64-asan-qr

[task 2022-11-15T20:20:35.273Z] 20:20:35     INFO - RunWatchdog: Mainthread nested event loops during hang:
[task 2022-11-15T20:20:35.275Z] 20:20:35     INFO -  --- (no nested event loop active)
[task 2022-11-15T20:20:35.276Z] 20:20:35     INFO - AddressSanitizer:DEADLYSIGNAL
[task 2022-11-15T20:20:35.277Z] 20:20:35     INFO - =================================================================
[task 2022-11-15T20:20:35.278Z] 20:20:35    ERROR - ==1538==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x7f93d140b572 bp 0x7f9308cf9e10 sp 0x7f9308cf9d40 T334)
[task 2022-11-15T20:20:35.279Z] 20:20:35     INFO - ==1538==The signal is caused by a WRITE memory access.
[task 2022-11-15T20:20:35.280Z] 20:20:35     INFO - ==1538==Hint: address points to the zero page.
[task 2022-11-15T20:20:36.846Z] 20:20:36     INFO -     #0 0x7f93d140b572 in MOZ_Crash /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:261:3
[task 2022-11-15T20:20:36.848Z] 20:20:36     INFO -     #1 0x7f93d140b572 in mozilla::(anonymous namespace)::RunWatchdog(void*) /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:244:5
[task 2022-11-15T20:20:36.849Z] 20:20:36     INFO -     #2 0x7f93ecd9efe8 in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
[task 2022-11-15T20:20:36.850Z] 20:20:36     INFO -     #3 0x7f93ecac96da in start_thread /tmp/glibc/nptl/pthread_create.c:463
[task 2022-11-15T20:20:36.851Z] 20:20:36     INFO -     #4 0x7f93eb88ca3e in __clone /tmp/glibc/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
[task 2022-11-15T20:20:36.852Z] 20:20:36     INFO - AddressSanitizer can not provide additional info.
[task 2022-11-15T20:20:36.853Z] 20:20:36     INFO - SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/Assertions.h:261:3 in MOZ_Crash
[task 2022-11-15T20:20:36.854Z] 20:20:36     INFO - Thread T334 (Shutdow~minator) created by T0 here:
[task 2022-11-15T20:20:36.980Z] 20:20:36     INFO -     #0 0x5636d0f4964c in __interceptor_pthread_create /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:208:3
[task 2022-11-15T20:20:36.987Z] 20:20:36     INFO -     #1 0x7f93ecd8f0b9 in _PR_CreateThread /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:458:14
[task 2022-11-15T20:20:36.987Z] 20:20:36     INFO -     #2 0x7f93ecd8052e in PR_CreateThread /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:533:12
[task 2022-11-15T20:20:36.987Z] 20:20:36     INFO -     #3 0x7f93d140aabd in CreateSystemThread /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:113:7
[task 2022-11-15T20:20:36.988Z] 20:20:36     INFO -     #4 0x7f93d140aabd in mozilla::nsTerminator::StartWatchdog() /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:469:7
[task 2022-11-15T20:20:36.989Z] 20:20:36     INFO -     #5 0x7f93d140af01 in Start /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:381:3

Comment 15

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Given related bug 1684441 and bug 1795670 are filed in DOM: Core & HTML, I'm considering moving this bug to the same component. :jstutte as you were involved with bug 1684441 could you provide your thoughts? I also wonder if this should depend on bug 1728721?

Comment 16

[Jens Stutte [:jstutte]]

If I run a search for WillDestroyCurrentMessageLoop in bugzilla comments I can count 19 different components over 29 Bugs. This does not mean, all of those are dupes, of course, WillDestroyCurrentMessageLoop appears also in log files when everything is ok. In fact this is just a symptom of a hanging child process if seen together with a shutdown timeout.

Unfortunately the logs usually do not give us many hints on what causes the child process we are waiting on to be slow, but looking at crash-stats through bug 1279293 gives us frequently some hints on things happening in the wild.

We might want to start to make these bugs depend on bug 1724337 and here also on bug 1793525 in order to make them more actionable. Until then any component is probably equally wrong in terms of the root cause that makes the child process hang. In particular I think this bug is where some of the hangs we've seen before on bug 1684441 come out now.

Comment 17

[Intermittent Failures Robot]

38 failures in 3589 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #38 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-beta: 3
• autoland: 24
• mozilla-central: 11

Platform and build breakdown:

• linux1804-64-asan-qr: 38
  • opt: 38

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-11-14&endday=2022-11-20&tree=all

Comment 18

[Intermittent Failures Robot]

33 failures in 3888 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #41 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 22
• mozilla-central: 8
• try: 2
• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 33
  • opt: 33

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-11-21&endday=2022-11-27&tree=all

Comment 19

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Core & HTML' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 20

[Natalia Csoregi [:nataliaCs]]

Update:

There have been 39 failures in the last 7 days, all of them on Linux 18.04 x64 WebRender asan opt.

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=398603446&repo=autoland&lineNumber=23089

Comment 21

[Intermittent Failures Robot]

40 failures in 3938 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #28 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 30
• mozilla-central: 8
• mozilla-release: 1
• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 40
  • opt: 40

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-11-28&endday=2022-12-04&tree=all

Comment 22

[Intermittent Failures Robot]

32 failures in 3510 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #49 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 7
• autoland: 23
• mozilla-beta: 1
• try: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 32
  • opt: 32

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-12-05&endday=2022-12-11&tree=all

Comment 23

[Intermittent Failures Robot]

22 failures in 3558 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 13
• mozilla-central: 5
• mozilla-beta: 3
• mozilla-esr102: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 22
  • opt: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-12-12&endday=2022-12-18&tree=all

Comment 24

[Intermittent Failures Robot]

1 failures in 2999 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 25

[Intermittent Failures Robot]

1 failures in 1328 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2022-12-26&endday=2023-01-01&tree=all

Comment 26

[Intermittent Failures Robot]

2 failures in 2607 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform and build breakdown:

• linux1804-64-asan-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2023-01-02&endday=2023-01-08&tree=all

Comment 27

[Cosmin Sabou [:CosminS]]

Last failure on trunk for this bug was on 14th Dec before Bug 1793525 landed.
https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2022-12-10&endday=2023-01-09&tree=trunk&failurehash=all&bug=1791335
I would say this can be closed as an incomplete.
The failures here seem to have been moved to Bug 1805613.

Comment 28

[Intermittent Failures Robot]

1 failures in 3530 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2023-01-09&endday=2023-01-15&tree=all

Comment 29

[Jens Stutte [:jstutte]]

(In reply to Cosmin Sabou [:CosminS] from comment #27)

Last failure on trunk for this bug was on 14th Dec before Bug 1793525 landed.
https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2022-12-10&endday=2023-01-09&tree=trunk&failurehash=all&bug=1791335
I would say this can be closed as an incomplete.
The failures here seem to have been moved to Bug 1805613.

Agreed, the later failure was based on an an earlier central, it seems.

Comment 30

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=402048091&repo=try

Comment 31

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=402048091&repo=try

Comment 32

[Jens Stutte [:jstutte]]

:CosminS, any advice how we can prevent the bot from re-opening this? Eventually it will go away by itself, I assume...

Comment 33

[Cosmin Sabou [:CosminS]]

Unfortunately there isn't a way to do this. Looking at https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2022-12-17&endday=2023-01-16&tree=all&failurehash=all&bug=1791335 this hasn't been used on trunk and the beta failures will go away with the merge today.
Last time this was classified was on that try push of Andreas which as you said was on a old central rev that didn't have the latest changes.
Hopefully nobody will use it again for classification and this will not be reopened by the bot.
Used worksforme because this was "fixed" or better said moved in another bug.

Comment 34

[Intermittent Failures Robot]

1 failures in 3717 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2023-01-16&endday=2023-01-22&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 3766 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791335&startday=2023-02-06&endday=2023-02-12&tree=all