1732274 - ClearRequestBase::DeleteFiles: Add some more record shutdown step (for example removal failed case)

Status: RESOLVED FIXED

(Core :: Storage: Quota Manager, task)

Description

[Haiyang Xu]

Crash report: https://crash-stats.mozilla.org/report/index/b3e169e8-4690-489e-a5be-96d470210921

MOZ_CRASH Reason: MOZ_CRASH(Quota manager shutdown timed out)

Top 10 frames of crashing thread:

0 xul.dll static mozilla::dom::quota::QuotaManager::Shutdown::<unnamed-tag>::operator dom/quota/ActorsParent.cpp:3739
1 xul.dll nsTimerImpl::Fire xpcom/threads/nsTimerImpl.cpp:618
2 xul.dll nsTimerEvent::Run xpcom/threads/TimerThread.cpp:248
3 xul.dll nsThread::ProcessNextEvent xpcom/threads/nsThread.cpp:1142
4 xul.dll NS_ProcessNextEvent xpcom/threads/nsThreadUtils.cpp:466
5 xul.dll mozilla::dom::quota::QuotaManager::Shutdown dom/quota/ActorsParent.cpp:3808
6 xul.dll static mozilla::dom::quota::QuotaManager::ShutdownInstance dom/quota/ActorsParent.cpp:3300
7 xul.dll mozilla::dom::quota::RecvShutdownQuotaManager dom/quota/ActorsParent.cpp:2685
8 xul.dll mozilla::ipc::BackgroundParentImpl::RecvShutdownQuotaManager ipc/glue/BackgroundParentImpl.cpp:1091
9 xul.dll mozilla::ipc::PBackgroundParent::OnMessageReceived ipc/ipdl/PBackgroundParent.cpp:4759

In ClearRequestBase::DeleteFiles https://searchfox.org/mozilla-central/source/dom/quota/ActorsParent.cpp#9293
We are not recording if we failed to remove directories. Change the NS_WARNING to record shutdown step.

Comment 1

[Haiyang Xu]

Attachment: Bug 1732274 - Record a message when directory removal fails in quota manager shutdown; r=#dom-storage-reviewers

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e15aa8a5a0c6
Record a message when directory removal fails in quota manager shutdown; r=dom-storage-reviewers,jstutte

Comment 3

[Haiyang Xu]

Attachment: Bug 1732274 - Use QM_WARNONLY_TRY when directory removal fails during clearing origins; r=#dom-storage-reviewers

Comment 4

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/e15aa8a5a0c6

Comment 5

[Jan Varga [:janv]]

Just for completeness, this is a stack trace of the QuotaManager IO thread:

0 ntdll.dll NtQueryDirectoryFileEx
1 KERNELBASE.dll FindNextFileW
2 xul.dll nsDirEnumerator::HasMoreElements(bool*)
3 xul.dll nsDirEnumerator::GetNextFile(nsIFile**)
4 xul.dll mozilla::dom::quota::anonymous namespace'::ClearRequestBase::DeleteFiles(mozilla::dom::quota::QuotaManager&, mozilla::dom::quota::PersistenceType) 5 xul.dll mozilla::dom::quota::anonymous namespace'::ClearRequestBase::DoDirectoryWork(mozilla::dom::quota::QuotaManager&)
6 xul.dll mozilla::dom::quota::`anonymous namespace'::OriginOperationBase::Run()
7 xul.dll nsThread::ProcessNextEvent(bool, bool*)
8 xul.dll mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*)
9 xul.dll MessageLoop::RunHandler()
10 xul.dll MessageLoop::Run()
11 xul.dll static nsThread::ThreadFunc(void*)
12 nss3.dll _PR_NativeRunThread(void*)
13 nss3.dll pr_root(void*)
14 ucrtbase.dll thread_start<unsigned int (__cdecl*)(void*), 1>
15 kernel32.dll BaseThreadInitThunk
16 ntdll.dll RtlUserThreadStart

So, the thread was trying to get next dir entry for the clear operation which may indicate we are doing a lot of scanning during shudown.

I think we can do an optimization which should speed up clearing of origins during shutdown. I'll file a bug for it.

Comment 6

[Jan Varga [:janv]]

I filed bug 1733107.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b015bd3df596
Use QM_WARNONLY_TRY when directory removal fails during clearing origins; r=dom-storage-reviewers,janv

Comment 8

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/b015bd3df596