1870499 - Content shutdown hang due to infinite loop in HttpChannelChild::Release | Intermittent application crashed [@ AutoLock<MaybeMutex>::AutoLock] | single tracking bug

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=440417005&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cNj15fNMQvuF83SqVUFdCQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-12-17T10:21:54.328Z] 10:21:54     INFO - TEST-START | dom/base/test/test_youtube_flash_embed.html
[task 2023-12-17T10:21:54.483Z] 10:21:54     INFO - GECKO(1300) | [Parent 1300, Main Thread] WARNING: NS_ENSURE_TRUE(she && she->mInfo->mSharedState.Get()) failed: file /builds/worker/checkouts/gecko/docshell/shistory/SessionHistoryEntry.cpp:1159
[task 2023-12-17T10:21:54.646Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Listener is not retargetable: file /builds/worker/checkouts/gecko/netwerk/protocol/http/HttpChannelChild.cpp:3043
[task 2023-12-17T10:21:54.648Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Failed to retarget HTML data delivery to the parser thread.: file /builds/worker/checkouts/gecko/parser/html/nsHtml5StreamParser.cpp:1227
[task 2023-12-17T10:21:54.680Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Listener is not retargetable: file /builds/worker/checkouts/gecko/netwerk/protocol/http/HttpChannelChild.cpp:3043
[task 2023-12-17T10:21:54.682Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Failed to retarget HTML data delivery to the parser thread.: file /builds/worker/checkouts/gecko/parser/html/nsHtml5StreamParser.cpp:1227
[task 2023-12-17T10:21:54.699Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Listener is not retargetable: file /builds/worker/checkouts/gecko/netwerk/protocol/http/HttpChannelChild.cpp:3043
[task 2023-12-17T10:21:54.702Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Failed to retarget HTML data delivery to the parser thread.: file /builds/worker/checkouts/gecko/parser/html/nsHtml5StreamParser.cpp:1227
[task 2023-12-17T10:21:54.715Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Listener is not retargetable: file /builds/worker/checkouts/gecko/netwerk/protocol/http/HttpChannelChild.cpp:3043
[task 2023-12-17T10:21:54.718Z] 10:21:54     INFO - GECKO(1300) | [Child 1428, Main Thread] WARNING: Failed to retarget HTML data delivery to the parser thread.: file /builds/worker/checkouts/gecko/parser/html/nsHtml5StreamParser.cpp:1227
[task 2023-12-17T10:21:54.777Z] 10:21:54     INFO - GECKO(1300) | MEMORY STAT | vsize 51981MB | residentFast 230MB | heapAllocated 42MB
[task 2023-12-17T10:21:54.860Z] 10:21:54     INFO - TEST-OK | dom/base/test/test_youtube_flash_embed.html | took 532ms
[task 2023-12-17T10:21:54.916Z] 10:21:54     INFO - GECKO(1300) | [Child 1484, Main Thread] WARNING: IPC message 'PSessionStore::Msg_SessionStoreUpdate' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:545
[task 2023-12-17T10:21:54.987Z] 10:21:54     INFO - TEST-START | Shutdown
[task 2023-12-17T10:21:54.989Z] 10:21:54     INFO - Passed:  44798
[task 2023-12-17T10:21:54.990Z] 10:21:54     INFO - Failed:  0
[task 2023-12-17T10:21:54.992Z] 10:21:54     INFO - Todo:    110
[task 2023-12-17T10:21:54.993Z] 10:21:54     INFO - Mode:    e10s
[task 2023-12-17T10:21:54.995Z] 10:21:54     INFO - Slowest: 141298ms - /tests/dom/base/test/test_timeout_clamp.html
[task 2023-12-17T10:21:54.996Z] 10:21:54     INFO - SimpleTest FINISHED
[task 2023-12-17T10:21:54.998Z] 10:21:54     INFO - TEST-INFO | Ran 1 Loops
<...>
[task 2023-12-17T10:22:17.333Z] 10:22:17     INFO - zombiecheck | Checking for orphan process with PID: 1820
[task 2023-12-17T10:22:17.334Z] 10:22:17     INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump-stackwalk/minidump-stackwalk --symbols-url=https://symbols.mozilla.org/ --cyborg=/tmp/tmplapuvra1/61350e84-5a79-4211-dfe8-c121d5e78211.trace /tmp/tmptusxhahm.mozrunner/minidumps/61350e84-5a79-4211-dfe8-c121d5e78211.dmp /builds/worker/workspace/build/symbols
[task 2023-12-17T10:22:21.848Z] 10:22:21     INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/61350e84-5a79-4211-dfe8-c121d5e78211.dmp
[task 2023-12-17T10:22:21.924Z] 10:22:21     INFO - PROCESS-CRASH | application crashed [@ AutoLock<MaybeMutex>::AutoLock] | dom/base/test/mochitest.toml 
[task 2023-12-17T10:22:21.926Z] 10:22:21     INFO - Process type: unknown
[task 2023-12-17T10:22:21.926Z] 10:22:21     INFO - Process pid: 1428
[task 2023-12-17T10:22:21.927Z] 10:22:21     INFO - Crash dump filename: /tmp/tmptusxhahm.mozrunner/minidumps/61350e84-5a79-4211-dfe8-c121d5e78211.dmp
[task 2023-12-17T10:22:21.928Z] 10:22:21     INFO - Operating system: Linux
[task 2023-12-17T10:22:21.929Z] 10:22:21     INFO -                   4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018
[task 2023-12-17T10:22:21.929Z] 10:22:21     INFO - CPU: amd64
[task 2023-12-17T10:22:21.930Z] 10:22:21     INFO -      family 6 model 85 stepping 7
[task 2023-12-17T10:22:21.931Z] 10:22:21     INFO -      2 CPUs
[task 2023-12-17T10:22:21.932Z] 10:22:21     INFO - Linux Ubuntu 18.04 - bionic (Ubuntu 18.04.6 LTS)
[task 2023-12-17T10:22:21.932Z] 10:22:21     INFO - 
[task 2023-12-17T10:22:21.933Z] 10:22:21     INFO - Crash reason:  SIGABRT
[task 2023-12-17T10:22:21.934Z] 10:22:21     INFO - Crash address: 0x0000000000000000
[task 2023-12-17T10:22:21.934Z] 10:22:21     INFO - Crashing instruction: `push rbp`
[task 2023-12-17T10:22:21.935Z] 10:22:21     INFO - No memory accessed by instruction
[task 2023-12-17T10:22:21.936Z] 10:22:21     INFO - Process uptime: not available
[task 2023-12-17T10:22:21.937Z] 10:22:21     INFO - 
[task 2023-12-17T10:22:21.937Z] 10:22:21     INFO - Thread 0 Web Content (crashed)
[task 2023-12-17T10:22:21.938Z] 10:22:21     INFO -  0  firefox-bin!AutoLock<MaybeMutex>::AutoLock(MaybeMutex&) [Mutex.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 246 + 0x0]
[task 2023-12-17T10:22:21.939Z] 10:22:21     INFO -      rax = 0xaaaaaaaaaaaaaaaa    rdx = 0x0000000000000000
[task 2023-12-17T10:22:21.939Z] 10:22:21     INFO -      rcx = 0x0000000000000f00    rbx = 0x0000000000000000
[task 2023-12-17T10:22:21.940Z] 10:22:21     INFO -      rsi = 0x00007f62af000020    rdi = 0x00007ffe059b2230
[task 2023-12-17T10:22:21.941Z] 10:22:21     INFO -      rbp = 0x00007ffe059b2270    rsp = 0x00007ffe059b2208
[task 2023-12-17T10:22:21.942Z] 10:22:21     INFO -       r8 = 0x0000000000000000     r9 = 0x00000000731c9c50
[task 2023-12-17T10:22:21.942Z] 10:22:21     INFO -      r10 = 0x00007f629a1676f0    r11 = 0x00007ffe059b2950
[task 2023-12-17T10:22:21.943Z] 10:22:21     INFO -      r12 = 0x00007f62af0001b0    r13 = 0x00000000000000f0
[task 2023-12-17T10:22:21.944Z] 10:22:21     INFO -      r14 = 0x00007f62af000000    r15 = 0x0000000000000050
[task 2023-12-17T10:22:21.945Z] 10:22:21     INFO -      rip = 0x0000555f36f0bac0
[task 2023-12-17T10:22:21.945Z] 10:22:21     INFO -     Found by: given as instruction pointer in context
[task 2023-12-17T10:22:21.946Z] 10:22:21     INFO -  1  firefox-bin!arena_t::MallocSmall(unsigned long, bool) [mozjemalloc.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 3276 + 0x10]
[task 2023-12-17T10:22:21.947Z] 10:22:21     INFO -      rbx = 0x0000000000000000    rbp = 0x00007ffe059b2270
[task 2023-12-17T10:22:21.947Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2210    r12 = 0x00007f62af0001b0
[task 2023-12-17T10:22:21.948Z] 10:22:21     INFO -      r13 = 0x00000000000000f0    r14 = 0x00007f62af000000
[task 2023-12-17T10:22:21.949Z] 10:22:21     INFO -      r15 = 0x0000000000000050    rip = 0x0000555f36f122ce
[task 2023-12-17T10:22:21.950Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.950Z] 10:22:21     INFO -  2  firefox-bin!BaseAllocator::malloc(unsigned long) [mozjemalloc.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 4551 + 0x9]
[task 2023-12-17T10:22:21.951Z] 10:22:21     INFO -      rbx = 0x0000000000000048    rbp = 0x00007ffe059b2290
[task 2023-12-17T10:22:21.952Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2280    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.952Z] 10:22:21     INFO -      r13 = 0x00007ffe059b2358    r14 = 0x00007ffe059b22a8
[task 2023-12-17T10:22:21.953Z] 10:22:21     INFO -      r15 = 0x00007f629a867250    rip = 0x0000555f36f15b74
[task 2023-12-17T10:22:21.954Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.955Z] 10:22:21     INFO -  3  firefox-bin!MozJemalloc::malloc(unsigned long) [malloc_decls.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 51]
[task 2023-12-17T10:22:21.956Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.956Z] 10:22:21     INFO -  4  firefox-bin!PageMalloc(mozilla::Maybe<unsigned long> const&, unsigned long) [PHC.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1306]
[task 2023-12-17T10:22:21.957Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.958Z] 10:22:21     INFO -  5  firefox-bin!MozJemallocPHC::malloc(unsigned long) [PHC.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1310 + 0x27]
[task 2023-12-17T10:22:21.959Z] 10:22:21     INFO -      rbx = 0x0000000000000048    rbp = 0x00007ffe059b22d0
[task 2023-12-17T10:22:21.959Z] 10:22:21     INFO -      rsp = 0x00007ffe059b22a0    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.960Z] 10:22:21     INFO -      r13 = 0x00007ffe059b2358    r14 = 0x00007f6296ba7767
[task 2023-12-17T10:22:21.961Z] 10:22:21     INFO -      r15 = 0x00007f629a867250    rip = 0x0000555f36f1422c
[task 2023-12-17T10:22:21.961Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.962Z] 10:22:21     INFO -  6  firefox-bin!moz_xmalloc [mozalloc.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 52 + 0x4]
[task 2023-12-17T10:22:21.963Z] 10:22:21     INFO -      rbx = 0x0000000000000048    rbp = 0x00007ffe059b22f0
[task 2023-12-17T10:22:21.963Z] 10:22:21     INFO -      rsp = 0x00007ffe059b22e0    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.964Z] 10:22:21     INFO -      r13 = 0x00007ffe059b2358    r14 = 0x00007f6296ba7767
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r15 = 0x00007f629a867250    rip = 0x0000555f36f3291e
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -  7  libxul.so!operator new(unsigned long) [cxxalloc.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 33]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -  8  libxul.so!mozilla::NewRunnableMethod<RefPtr<mozilla::net::HttpChannelChild>&, void (mozilla::net::HttpChannelChild::*)()>(char const*, RefPtr<mozilla::net::HttpChannelChild>&, void (mozilla::net::HttpChannelChild::*)()) [nsThreadUtils.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1426 + 0x9]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rbx = 0x00007ffe059b2348    rbp = 0x00007ffe059b2330
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2300    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r13 = 0x00007ffe059b2358    r14 = 0x00007f6296ba7767
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r15 = 0x00007f629a867250    rip = 0x00007f629a867168
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -  9  libxul.so!mozilla::net::HttpChannelChild::Release() [HttpChannelChild.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 231 + 0x20]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rbx = 0x00007f628132ae00    rbp = 0x00007ffe059b2380
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2340    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x00007ffe059b2348
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r15 = 0x00007f627a509180    rip = 0x00007f629a866c88
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 10  libxul.so!mozilla::RefPtrTraits<mozilla::net::HttpChannelChild>::Release(mozilla::net::HttpChannelChild*) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 54]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 11  libxul.so!RefPtr<mozilla::net::HttpChannelChild>::ConstRemovingRefPtrTraits<mozilla::net::HttpChannelChild>::Release(mozilla::net::HttpChannelChild*) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 420]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 12  libxul.so!RefPtr<mozilla::net::HttpChannelChild>::assign_assuming_AddRef(mozilla::net::HttpChannelChild*) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 73]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 13  libxul.so!RefPtr<mozilla::net::HttpChannelChild>::operator=(std::nullptr_t) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 188]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 14  libxul.so!nsRunnableMethodReceiver<mozilla::net::HttpChannelChild, true>::Revoke() [nsThreadUtils.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 777]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 15  libxul.so!mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::net::HttpChannelChild>, void (mozilla::net::HttpChannelChild::*)(), true, (mozilla::RunnableKind)0, >::Revoke() [nsThreadUtils.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1227]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 16  libxul.so!mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::net::HttpChannelChild>, void (mozilla::net::HttpChannelChild::*)(), true, (mozilla::RunnableKind)0, >::~RunnableMethodImpl() [nsThreadUtils.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1187]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 17  libxul.so!mozilla::detail::RunnableMethodImpl<RefPtr<mozilla::net::HttpChannelChild>, void (mozilla::net::HttpChannelChild::*)(), true, (mozilla::RunnableKind)0, >::~RunnableMethodImpl() [nsThreadUtils.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1187 + 0x15]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rbx = 0x00007f626a236ec0    rbp = 0x00007ffe059b23a0
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2390    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x0000000000000000
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      r15 = 0x00007f627a509180    rip = 0x00007f629a8b814f
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO - 18  libxul.so!mozilla::Runnable::Release() [nsThreadUtils.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 66 + 0x8]
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rbx = 0x00007f626a236ec0    rbp = 0x00007ffe059b23c0
[task 2023-12-17T10:22:21.965Z] 10:22:21     INFO -      rsp = 0x00007ffe059b23b0    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x0000000000000000
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f627a509180    rip = 0x00007f629a224d88
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 19  libxul.so!mozilla::RefPtrTraits<nsIRunnable>::Release(nsIRunnable*) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 54]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 20  libxul.so!RefPtr<nsIRunnable>::ConstRemovingRefPtrTraits<nsIRunnable>::Release(nsIRunnable*) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 420]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 21  libxul.so!RefPtr<nsIRunnable>::assign_assuming_AddRef(nsIRunnable*) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 73]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 22  libxul.so!RefPtr<nsIRunnable>::operator=(std::nullptr_t) [RefPtr.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 188]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 23  libxul.so!mozilla::RunnableTask::Run() [TaskController.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 550 + 0x16]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f627a509180    rbp = 0x00007ffe059b23e0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b23d0    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x00007ffe059b2470
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f627a509180    rip = 0x00007f629a21c09f
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 24  libxul.so!mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [TaskController.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 876 + 0x8]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x0000000000000004    rbp = 0x00007ffe059b2a00
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b23f0    r12 = 0x0000000000000022
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x00007ffe059b2470
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f627a509180    rip = 0x00007f629a211d14
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 25  libxul.so!mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [TaskController.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 699 + 0xa]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aef31300    rbp = 0x00007ffe059b2a50
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2a10    r12 = 0x000000cd08341bf2
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0x00007ffe059b2a68    r14 = 0x00007f62aef435e0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f62aef31328    rip = 0x00007f629a210778
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 26  libxul.so!mozilla::TaskController::ProcessPendingMTTask(bool) [TaskController.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 485 + 0x7]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aef31300    rbp = 0x00007ffe059b2aa0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2a60    r12 = 0x000000cd08341bf2
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x0000000000000000
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f62aef31328    rip = 0x00007f629a210bf6
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 27  libxul.so!mozilla::TaskController::TaskController()::$_0::operator()() const [TaskController.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 211]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 28  libxul.so!mozilla::detail::RunnableFunction<mozilla::TaskController::TaskController()::$_0>::Run() [nsThreadUtils.h:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 548 + 0x12]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aef733c0    rbp = 0x00007ffe059b2ab0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2ab0    r12 = 0x000000cd08341bf2
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x0000000000000001
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007ffe059b2b40    rip = 0x00007f629a21fff7
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 29  libxul.so!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 1199 + 0x5]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aef733c0    rbp = 0x00007ffe059b2be0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2ac0    r12 = 0x000000cd08341bf2
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x0000000000000001
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007ffe059b2b40    rip = 0x00007f629a234603
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 30  libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 480 + 0xf]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x0000000000000000    rbp = 0x00007ffe059b2c10
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2bf0    r12 = 0x0000000000000001
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0x00007f62aefa0710    r14 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f62aef733c0    rip = 0x00007f629a23b38e
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 31  libxul.so!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [MessagePump.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 85 + 0x9]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aefa06f0    rbp = 0x00007ffe059b2c60
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2c20    r12 = 0x0000000000000001
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0x00007f62aefa0710    r14 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f62aef733c0    rip = 0x00007f629ac27b17
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 32  libxul.so!MessageLoop::RunHandler() [message_loop.cc:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 363]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 33  libxul.so!MessageLoop::Run() [message_loop.cc:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 345 + 0x4]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aef7b380    rbp = 0x00007ffe059b2c90
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2c70    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007f62aef733c0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f629aba4952
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO - 34  libxul.so!nsBaseAppShell::Run() [nsBaseAppShell.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 148 + 0xc]
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rbx = 0x00007f62aef7b380    rbp = 0x00007ffe059b2cb0
[task 2023-12-17T10:22:21.966Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2ca0    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007f62aef733c0
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f629e7cd0c9
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 35  libxul.so!nsAppShell::Run() [nsAppShell.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 470 + 0x7]
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rbx = 0x00007f62aef7b380    rbp = 0x00007ffe059b2cf0
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2cc0    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007f62aefa06f0
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f629e87e979
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 36  libxul.so!XRE_RunAppShell() [nsEmbedFunctions.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 721 + 0x5]
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rbx = 0x00007ffe059b2dd8    rbp = 0x00007ffe059b2d30
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2d00    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007f62aefa06f0
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f62a029fdfc
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 37  libxul.so!mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) [MessagePump.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 235 + 0x4]
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rbx = 0x00007ffe059b2dd8    rbp = 0x00007ffe059b2d60
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2d40    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007f62aefa06f0
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f629ac284c1
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 38  libxul.so!MessageLoop::RunHandler() [message_loop.cc:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 363]
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 39  libxul.so!MessageLoop::Run() [message_loop.cc:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 345 + 0x4]
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rbx = 0x0000000000000001    rbp = 0x00007ffe059b2d90
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2d70    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f629aba4952
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 40  libxul.so!XRE_InitChildProcess(int, char**, XREChildData const*) [nsEmbedFunctions.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 656 + 0x7]
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rbx = 0x0000000000000001    rbp = 0x00007ffe059b3080
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      rsp = 0x00007ffe059b2da0    r12 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r13 = 0x0000000000000514    r14 = 0x00007ffe059b2dd8
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -      r15 = 0x00007f62aefbbf00    rip = 0x00007f62a029f66a
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.967Z] 10:22:21     INFO - 41  firefox-bin!content_process_main(mozilla::Bootstrap*, int, char**) [plugin-container.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 57]
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: inlining
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 42  firefox-bin!main [nsBrowserApp.cpp:2d19e1f44e562ba1bb3be1f4b8f95b9884d9b07d : 375 + 0x2a]
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rbx = 0x0000000000000016    rbp = 0x00007ffe059b40f0
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rsp = 0x00007ffe059b3090    r12 = 0x00007ffe059b41d8
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      r13 = 0xaaaaaaaaaaaaaaaa    r14 = 0x00007f62aef036e0
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      r15 = 0x00007ffe059b41d8    rip = 0x0000555f36f06d59
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 43  libc.so.6 + 0x21b96
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rbx = 0x0000000000000000    rbp = 0x0000555f36fd8080
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rsp = 0x00007ffe059b4100    r12 = 0x0000555f36f06780
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      r13 = 0x00007ffe059b41d0    r14 = 0x0000000000000000
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      r15 = 0x0000000000000000    rip = 0x00007f62af423b97
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: call frame info
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 44  ld-linux-x86-64.so.2 + 0x10782
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rsp = 0x00007ffe059b4180    rip = 0x00007f62b0565783
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: stack scanning
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 45  libmozsandbox.so!<.plt ELF section in libmozsandbox.so> + 0x15e7
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rsp = 0x00007ffe059b4188    rip = 0x00007f62b0777418
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: stack scanning
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 46  libpthread.so.0 + 0x219baf
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rsp = 0x00007ffe059b4190    rip = 0x00007f62b054fbb0
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: stack scanning
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 47  firefox-bin!_start + 0x28
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -      rsp = 0x00007ffe059b41c0    rip = 0x0000555f36f067a9
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO -     Found by: stack scanning
[task 2023-12-17T10:22:21.968Z] 10:22:21     INFO - 

Comment 1

[Iulian Moraru]

Hi Gregory! Can you please take another look at this? Even with the changes reverted for test_youtube_flash_embed.html which landed in Bug 1869556, we are still seeing new crashes after TEST-START | dom/base/test/test_youtube_flash_embed.html.
Thank you!

Comment 2

[Gregory Pappas [:gregp]]

Even with the changes reverted for test_youtube_flash_embed.html which landed in Bug 1869556, we are still seeing new crashes after TEST-START | dom/base/test/test_youtube_flash_embed.html

That's unfortunate. I won't have time to properly debug this for another few weeks, so feel free to disable the test. We won't completely lose coverage for this feature because there's also test_bug769117.html, so it should be ok for now.

Comment 3

[Iulian Moraru]

Attachment: Bug 1870499 - disable test_youtube_flash_embed.html on linux debug for causing multiple new crashes after this test start. r=#intermittent-reviewers,gregp,aryx

Comment 4

[Iulian Moraru]

Landed the patch to disable test_youtube_flash_embed.html on linux debug. Please remember to re-enable the test when investigating the issues that it caused.
Thank you!

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6f3fa2f686c5
disable test_youtube_flash_embed.html on linux debug for causing multiple new crashes after this test start. r=gregp,intermittent-reviewers,jmaher

Comment 6

[Intermittent Failures Robot]

1 failures in 4224 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2023-12-11&endday=2023-12-17&tree=all

Comment 7

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/6f3fa2f686c5

Comment 8

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

There's a cluster of bugs here that all look like the same thing: a content process shutdown hang because of an infinite loop where a runnable dispatches another instance of itself to the main thread, which then runs and dispatches another, etc. We can see that a lot of the reports are in code related to the event loop itself (e.g., bug 1870112), indicating that that's where a significant amount of time is being spent.

The stack this bug was filed for, however, is the interesting one: the signal was caught while taking a lock, inside malloc, trying to allocate a new runnable for HttpChannelChild, which is called from HttpChannelChild's own Release method, which in turn is called from a runnable being destroyed.

Specifically, the loop seems to be here: when the last reference to an HttpChannelChild is dropped, if certain conditions are true, the object will retroactively un-drop itself and dispatch an owning runnable to call DoNotifyListener, and I'm guessing that method is meant to change state so that it won't be called again, but in this case it fails to do that, so when it finishes and drops its reference the whole process repeats.

Comment 9

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Also… is this really related to test_youtube_flash_embed.html, or is that just the last test (in alphabetical order) in that test run? Maybe there has been an HttpChannelChild in a loop of failing to destroy itself since some earlier test?

Comment 14

[Andrew McCreight (out of office until 8/21) [:mccr8]]

(In reply to Jed Davis [:jld] ⟨⏰|UTC-8⟩ ⟦he/him⟧ from comment #9)

Also… is this really related to test_youtube_flash_embed.html, or is that just the last test (in alphabetical order) in that test run? Maybe there has been an HttpChannelChild in a loop of failing to destroy itself since some earlier test?

The theory was that it was related because it started spiking up after that test changed, but given that the test was reverted and the issues were still happening, maybe that was a red herring? On the other hand, the test has now been disabled entirely, so are the failures still happening?

Comment 19

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 20

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 22

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 25

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 26

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 27

[Intermittent Failures Robot]

1 failures in 3349 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2023-12-18&endday=2023-12-24&tree=all

Comment 29

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 30

[Intermittent Failures Robot]

3 failures in 2274 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 3
  • debug: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-01-01&endday=2024-01-07&tree=all

Comment 31

[Intermittent Failures Robot]

3 failures in 3217 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-qr: 3
  • debug: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-01-08&endday=2024-01-14&tree=all

Comment 32

[Intermittent Failures Robot]

4 failures in 3599 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 4

Platform and build breakdown:

• linux1804-64-qr: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-01-15&endday=2024-01-21&tree=all

Comment 33

[Intermittent Failures Robot]

4 failures in 4034 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 2
• autoland: 1

Platform and build breakdown:

• linux1804-64-qr: 3
  • debug: 3
• linux2204-64-wayland: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-01-22&endday=2024-01-28&tree=all

Comment 34

[Intermittent Failures Robot]

1 failures in 3952 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-01-29&endday=2024-02-04&tree=all

Comment 35

[Intermittent Failures Robot]

4 failures in 4009 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 2
  • debug: 2
• linux1804-64-ccov-qr: 1
  • opt: 1
• linux2204-64-wayland: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-02-05&endday=2024-02-11&tree=all

Comment 36

[Iulian Moraru]

Fyi, the orange factor here doesn't reflect the reality because there are many crashes that happen and they have new bugs filed for them, so, all the failures go to them and not to this bug.
Hi Jed! Can you please take another look at this? or maybe redirect this to someone who can?
Thank you!

Comment 37

[Intermittent Failures Robot]

4 failures in 3125 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-02-12&endday=2024-02-18&tree=all

Comment 38

[Intermittent Failures Robot]

3 failures in 3382 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 3
  • debug: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-02-19&endday=2024-02-25&tree=all

Comment 39

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

(In reply to Iulian Moraru from comment #36)

Fyi, the orange factor here doesn't reflect the reality because there are many crashes that happen and they have new bugs filed for them, so, all the failures go to them and not to this bug.
Hi Jed! Can you please take another look at this? or maybe redirect this to someone who can?
Thank you!

I don't think I know enough about Necko to actually fix this, but I might be able to turn all of these into assertion failure in one place, which should help manage them if nothing else.

Comment 40

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

Attachment: Bug 1870499 - Crash instead of looping infinitely in `HttpChannelChild::Release`.

When the last reference to HttpChannelChild is dropped, under some
circumstances it can un-drop itself to make sure that cleanup happens
correctly, by posting a runnable to the main thread to invoke a callback.
Unfortunately, under some circumstances this doesn't finalize the object,
causing an infinite loop of runnable dispatch, which causes a shutdown hang.

In particular, on NS_FREE_PERMANENT_DATA builds (where we always wait
for child processes to shut down fully — including debug, ASan, and
ccov, but not release builds), the timeout in ProcessWatcher obtains
a crash report, which in this case is often in some random code related
to event loops or runnables. It may be possible for this loop to happen
at times other than shutdown, but I suspect that the underlying cause is
something shutdown releated.

Instead, this patch checks for the same HttpChannelChild object
repeatedly looping through that case in Release, and crashes in one
convenient place (but only on NS_FREE_PERMANENT_DATA builds). This
doesn't fix the cause of this problem, but at least it will make the
crashes easier to assign to the correct bug.

Comment 41

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - HttpChannelChild::Release should pass itself using std::move r=#necko

Comment 42

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - Break potential runnable loop in HttpChannelChild::Release r=#necko

Comment 43

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - Make sure ContinueDoNotifyListener is called immediately during release r=#necko

The causes of the runnable loop are as follows:

1. In Release we have a special case to make sure OnStart&OnStop are always
   called. We do this by dispatching a runnable that calls DoNotifyListener.
2. DoNotifyListener is called, then the reference to the channel is again
   dropped. But at this point only OnStartRequest was called, and
   ContinueDoNotifyListener was added to mEventQ. That means another
   runnable would be dispatched.

While the loop was broken in the previous patch, we still want to ensure
OnStopRequest is called before the channel is released, so this patch
adds an argument to DoNotifyListener to select whether ContinueDoNotifyListener
should be called synchronously or dispatched to mEventQ.

Comment 45

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/63e88afd6891
HttpChannelChild::Release should pass itself using std::move r=necko-reviewers,kershaw
https://hg.mozilla.org/integration/autoland/rev/47d03af2398d
Break potential runnable loop in HttpChannelChild::Release r=necko-reviewers,kershaw
https://hg.mozilla.org/integration/autoland/rev/edfd64d3c6a0
Make sure ContinueDoNotifyListener is called immediately during release r=necko-reviewers,kershaw

Comment 46

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/63e88afd6891
https://hg.mozilla.org/mozilla-central/rev/47d03af2398d
https://hg.mozilla.org/mozilla-central/rev/edfd64d3c6a0

Comment 47

[Intermittent Failures Robot]

1 failures in 4052 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870499&startday=2024-02-26&endday=2024-03-03&tree=all

Comment 48

[Valentin Gosu [:valentin] (he/him)]

No more failures since this landed.

Comment 49

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:valentin, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox124 to wontfix.

For more information, please visit BugBot documentation.

Comment 50

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - HttpChannelChild::Release should pass itself using std::move r=#necko

Original Revision: https://phabricator.services.mozilla.com/D202796

Comment 51

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - Break potential runnable loop in HttpChannelChild::Release r=#necko

Original Revision: https://phabricator.services.mozilla.com/D202797

Comment 52

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - Make sure ContinueDoNotifyListener is called immediately during release r=#necko

The causes of the runnable loop are as follows:

1. In Release we have a special case to make sure OnStart&OnStop are always
   called. We do this by dispatching a runnable that calls DoNotifyListener.
2. DoNotifyListener is called, then the reference to the channel is again
   dropped. But at this point only OnStartRequest was called, and
   ContinueDoNotifyListener was added to mEventQ. That means another
   runnable would be dispatched.

While the loop was broken in the previous patch, we still want to ensure
OnStopRequest is called before the channel is released, so this patch
adds an argument to DoNotifyListener to select whether ContinueDoNotifyListener
should be called synchronously or dispatched to mEventQ.

Original Revision: https://phabricator.services.mozilla.com/D202798

Comment 53

[Phabricator Automation]

Uplift Approval Request

• Is Android affected?: yes
• Fix verified in Nightly: yes
• String changes made/needed: None
• Steps to reproduce for manual QE testing: -
• User impact if declined: Potential shutdown hangs.
• Code covered by automated testing: yes
• Risk associated with taking this patch: Low.
• Needs manual QE test: no
• Explanation of risk level: The patches make sure to not keep dispatching events to release the channel when doing so fails. Failures on mozilla-central have stopped, since we landed this, with no unexpected regressions.

Comment 54

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - HttpChannelChild::Release should pass itself using std::move r=#necko

Original Revision: https://phabricator.services.mozilla.com/D202796

Comment 55

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - Break potential runnable loop in HttpChannelChild::Release r=#necko

Original Revision: https://phabricator.services.mozilla.com/D202797

Comment 56

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1870499 - Make sure ContinueDoNotifyListener is called immediately during release r=#necko

The causes of the runnable loop are as follows:

1. In Release we have a special case to make sure OnStart&OnStop are always
   called. We do this by dispatching a runnable that calls DoNotifyListener.
2. DoNotifyListener is called, then the reference to the channel is again
   dropped. But at this point only OnStartRequest was called, and
   ContinueDoNotifyListener was added to mEventQ. That means another
   runnable would be dispatched.

While the loop was broken in the previous patch, we still want to ensure
OnStopRequest is called before the channel is released, so this patch
adds an argument to DoNotifyListener to select whether ContinueDoNotifyListener
should be called synchronously or dispatched to mEventQ.

Original Revision: https://phabricator.services.mozilla.com/D202798

Comment 57

[Phabricator Automation]

Uplift Approval Request

• Risk associated with taking this patch: Low
• Code covered by automated testing: yes
• Explanation of risk level: The patches make sure to not keep dispatching events to release the channel when doing so fails. Failures on mozilla-central have stopped, since we landed this, with no unexpected regressions.
• Needs manual QE test: no
• String changes made/needed: none
• Is Android affected?: yes
• Fix verified in Nightly: yes
• User impact if declined: Potential content process shutdown hangs.
• Steps to reproduce for manual QE testing: -

Comment 58

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/2b16cbdb3cd1
https://hg.mozilla.org/releases/mozilla-beta/rev/5a47f23677a3
https://hg.mozilla.org/releases/mozilla-beta/rev/86caf172561c

Comment 59

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/59f87ac1fb42
https://hg.mozilla.org/releases/mozilla-esr115/rev/aa4cd4d62f70
https://hg.mozilla.org/releases/mozilla-esr115/rev/04ba905a0086

Comment 61

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 65

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 66

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 67

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.