Crash in [@ shutdownhang | mozilla::SpinEventLoopUntil<T> | mozilla::dom::quota::QuotaManager::Observer::Observe]

ASSIGNED
Assigned to

Status

()

defect
P1
critical
ASSIGNED
2 months ago
10 days ago

People

(Reporter: janv, Assigned: janv)

Tracking

({crash, topcrash})

Trunk
Unspecified
Windows 10
Points:
---

Firefox Tracking Flags

(firefox67+ affected, firefox68 affected)

Details

(crash signature)

Assignee

Description

2 months ago

This bug is for crash report bp-44477099-72f5-4a39-8359-a8c4b0190406.

Top 10 frames of crashing thread:

0 ntdll.dll NtWaitForAlertByThreadId 
1 ntdll.dll RtlSleepConditionVariableSRW 
2 kernelbase.dll SleepConditionVariableSRW 
3 mozglue.dll mozilla::detail::ConditionVariableImpl::wait mozglue/misc/ConditionVariable_windows.cpp:50
4 xul.dll struct already_AddRefed<nsIRunnable> mozilla::ThreadEventQueue<mozilla::PrioritizedEventQueue<mozilla::EventQueue> >::GetEvent xpcom/threads/ThreadEventQueue.cpp:149
5 xul.dll nsThread::ProcessNextEvent xpcom/threads/nsThread.cpp:1108
6 xul.dll NS_ProcessNextEvent xpcom/threads/nsThreadUtils.cpp:482
7 xul.dll static bool mozilla::SpinEventLoopUntil<mozilla::ProcessFailureBehavior::ReportToCaller, `lambda at z:/task_1553508105/build/src/dom/quota/ActorsParent.cpp:2552:5'> xpcom/threads/nsThreadUtils.h:348
8 xul.dll nsresult mozilla::dom::quota::QuotaManager::Observer::Observe dom/quota/ActorsParent.cpp:2552
9 xul.dll nsObserverList::NotifyObservers xpcom/ds/nsObserverList.cpp:66

Assignee

Updated

2 months ago
Crash Signature: [@ shutdownhang | mozilla::SpinEventLoopUntil<T> | mozilla::dom::quota::QuotaManager::Observer::Observe] → [@ shutdownhang | mozilla::SpinEventLoopUntil<T> | mozilla::dom::quota::QuotaManager::Observer::Observe] [@ shutdownhang | __psynch_cvwait | <name omitted> | mozilla::dom::quota::QuotaManager::Observer::Observe]
Assignee

Updated

2 months ago
Crash Signature: [@ shutdownhang | mozilla::SpinEventLoopUntil<T> | mozilla::dom::quota::QuotaManager::Observer::Observe] [@ shutdownhang | __psynch_cvwait | <name omitted> | mozilla::dom::quota::QuotaManager::Observer::Observe] → [@ shutdownhang | mozilla::SpinEventLoopUntil<T> | mozilla::dom::quota::QuotaManager::Observer::Observe] [@ shutdownhang | __psynch_cvwait | <name omitted> | mozilla::dom::quota::QuotaManager::Observer::Observe] [@ shutdownhang | ntdll.dll | kernelbase.…
Assignee

Updated

a month ago
Assignee: nobody → jvarga
Status: NEW → ASSIGNED
Assignee

Comment 1

a month ago

This is a new bug for the crash that has been tracked in bug 1487194.
There were several changes in signatures, so the old bug now has too many signatures which makes it a bit hard to understand.
This new bug is mostly useful for FF 68 and even FF 67.
The old one is good for FF 66.

Keywords: topcrash
Priority: -- → P1

Jan, looking at the crash graph here makes me think something got fixed in nightly that hasn't (yet?) made it to beta?

Flags: needinfo?(jvarga)
Assignee

Comment 3

a month ago

Andrew, this graph is a little bit misleading. It's a bit more complicated, but basically starting with FF 68, these shutdown hangs are tracked in bug 1541776 and bug 1541928.
Anyway, bug 1542541 may have helped with reducing shutdown hangs and we should see after today's new beta.
Note, that beta still uses the old crash signatures (as in this bug).
Sorry for the confusion.

Flags: needinfo?(jvarga)

Thanks for clarifying - it is indeed a complicated situation but I'm confident we're on the right track to see major improvements here soon!

The volume of crashes didn't significantly go down after the patch in bug 1542541 which landed in beta 12, jan is there anything safe we could do to redice these crashes in the 67 timeframe (release and dot releases?) Thanks

Flags: needinfo?(jvarga)
Assignee

Comment 6

17 days ago

Yeah, I should have more time for that now. I'm going to add more debugging info to crash annotations as the first thing.

Flags: needinfo?(jvarga)

(In reply to Jan Varga [:janv] from comment #6)

Yeah, I should have more time for that now. I'm going to add more debugging info to crash annotations as the first thing.

Any news on this front? Is there something we could do to mitigate the crashes in a RC2 this week?

Flags: needinfo?(jvarga)
Assignee

Comment 8

10 days ago

I'm afraid that this can't be easily fixed/mitigated. It's a long standing issue, we still don't know exactly why is this happening. I'm going to add more info to crash annotations as I mentioned earlier. I was just too busy with last (hopefully) major LSNG bug 1534222.
When we have more info in crash annotations, we need to wait several days to collect and process data from crash-stats.
It's possible that we will have to add even more info to crash annotations. Once we know what the problem is, it's likely that the fix will be risky.
Btw, this bug tracks the same issue as bug 1487194. The only difference is that crash signatures changed since FF 66. So this is not a new issue, we already ship with it.
See also bug 1487194 comment 43.

Flags: needinfo?(jvarga)

(In reply to Jan Varga [:janv] from comment #8)

I'm afraid that this can't be easily fixed/mitigated. It's a long standing issue, we still don't know exactly why is this happening. I'm going to add more info to crash annotations as I mentioned earlier. I was just too busy with last (hopefully) major LSNG bug 1534222.
When we have more info in crash annotations, we need to wait several days to collect and process data from crash-stats.
It's possible that we will have to add even more info to crash annotations. Once we know what the problem is, it's likely that the fix will be risky.
Btw, this bug tracks the same issue as bug 1487194. The only difference is that crash signatures changed since FF 66. So this is not a new issue, we already ship with it.
See also bug 1487194 comment 43.

Not a new regression so we are not going to block 67 on it, thanks!

Thanks for the details, Jan. Bug 1542478 is where we're going to add more info to IDB shutdown hang crash annotations.

You need to log in before you can comment on or make changes to this bug.