821992 - OOM crash in mozilla::safebrowsing::HashStore::Open

Status: RESOLVED WONTFIX

(Toolkit :: Safe Browsing, defect, P5)

Description

[Scoobidiver (away)]

It's #148 top browser crasher w/o hangs in 17.0.1 and #82 in 18.0b4.
It first showed up in 17.0a2/20120907. The regression range might be (discontinuous across builds):
http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=c2cc63c864e4&tochange=41eab5a0e537

There are two kinds of stack traces:
Frame 	Module 	Signature 	Source
0 	mozalloc.dll 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:23
1 	mozalloc.dll 	mozalloc_handle_oom 	memory/mozalloc/mozalloc_oom.cpp:27
2 	mozalloc.dll 	moz_xmalloc 	memory/mozalloc/mozalloc.cpp:59
3 	xul.dll 	mozilla::safebrowsing::HashStore::Open 	toolkit/components/url-classifier/HashStore.cpp:252
4 	xul.dll 	mozilla::safebrowsing::Classifier::TableRequest 	toolkit/components/url-classifier/Classifier.cpp:248
5 	xul.dll 	nsUrlClassifierDBServiceWorker::GetTables 	toolkit/components/url-classifier/nsUrlClassifierDBService.cpp:409
6 	xul.dll 	UrlClassifierDBServiceWorkerProxy::GetTablesRunnable::Run 	toolkit/components/url-classifier/nsUrlClassifierProxies.cpp:47
7 	xul.dll 	nsThread::ProcessNextEvent 	xpcom/threads/nsThread.cpp:620
8 	xul.dll 	nsThread::ThreadFunc 	xpcom/threads/nsThread.cpp:258
9 	nspr4.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c:395

Frame 	Module 	Signature 	Source
0 	mozalloc.dll 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:23
1 	mozalloc.dll 	mozalloc_handle_oom 	memory/mozalloc/mozalloc_oom.cpp:27
2 	mozalloc.dll 	moz_xmalloc 	memory/mozalloc/mozalloc.cpp:59
3 	xul.dll 	mozilla::safebrowsing::HashStore::Open 	toolkit/components/url-classifier/HashStore.cpp:252
4 	xul.dll 	mozilla::safebrowsing::Classifier::RegenActiveTables 	toolkit/components/url-classifier/Classifier.cpp:457
5 	xul.dll 	mozilla::safebrowsing::Classifier::Open 	toolkit/components/url-classifier/Classifier.cpp:212
6 	xul.dll 	nsUrlClassifierDBServiceWorker::OpenDb 	toolkit/components/url-classifier/nsUrlClassifierDBService.cpp:784
7 	xul.dll 	nsUrlClassifierDBServiceWorker::BeginUpdate 	toolkit/components/url-classifier/nsUrlClassifierDBService.cpp:452
8 	xul.dll 	UrlClassifierDBServiceWorkerProxy::BeginUpdateRunnable::Run 	toolkit/components/url-classifier/nsUrlClassifierProxies.cpp:73
9 	xul.dll 	nsThread::ProcessNextEvent 	xpcom/threads/nsThread.cpp:620
10 	xul.dll 	nsThread::ThreadFunc 	xpcom/threads/nsThread.cpp:258
11 	nspr4.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c:395
12 	nspr4.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c:90

More reports at:
https://crash-stats.mozilla.com/report/list?signature=mozalloc_abort%28char+const*+const%29+|+mozalloc_handle_oom%28unsigned+int%29+|+moz_xmalloc+|+mozilla%3A%3Asafebrowsing%3A%3AHashStore%3A%3AOpen%28%29
https://crash-stats.mozilla.com/report/list?signature=mozalloc_abort+|+malloc+|+mozalloc_handle_oom+|+moz_xmalloc+|+mozilla%3A%3Asafebrowsing%3A%3AHashStore%3A%3AOpen

Comment 1

[Gian-Carlo Pascutto [:gcp]]

http://mxr.mozilla.org/mozilla-release/source/toolkit/components/url-classifier/HashStore.cpp#251

We're OOMing trying to allocate a file buffer, that's usually <2M.

Sampling the reports, most of these people seem to either have something (not even Firefox) gobbling up all of their memory, and most importantly Windows pagefile, or are running with, well, dumb, Windows pagefile settings that cause us to OOM even when there's still some physical memory available.
I think that's correlated with most of these reports being from Windows XP.

From my side this is a WONTFIX. We can't do anything sane if there's less than 2M RAM available.

Some of these reports are very strange:
https://crash-stats.mozilla.com/report/index/a99fa7e2-3f1a-486f-8235-39c162130120
OOM of a 2M allocation with >1G physical RAM free and pagefile free

https://crash-stats.mozilla.com/report/index/924fa07b-cca9-4211-8ad6-59f7b2130121
OOM of a 2K(!!) allocation with 400M physical RAM and 5M pagefile free

The other thing we can do here is to run a correlation analysis and see if some plugin or other software installed is making this worse. Especially the OOM's with plenty of RAM and pagefile free are perhaps worth further investigation.

Comment 2

[Scoobidiver (away)]

(In reply to Gian-Carlo Pascutto (:gcp) from comment #1)
> From my side this is a WONTFIX. We can't do anything sane if there's less
> than 2M RAM available.
In addition, while there are crashes in Betas of 17.0 and 18.0, there are none in 19.0 Beta so it has been likely fixed since 19.0. It might OOM elsewhere though.

Comment 3

[Gian-Carlo Pascutto [:gcp]]

Well, I'm sure this code didn't change, so it can't have magically fixed itself, unless something else now OOMs first. I think there's now a whole bunch of crashes grouped under the nsTArray::SizeTooBig signature.

Comment 4

[François Marier [:francois]]

Marking as WONTFIX as per comment 1.