1439393 - Consistent OOM | small crash after upgrade to 59b1 with RSS feeds

Status: VERIFIED FIXED

(MailNews Core :: Feed Reader, defect)

Description

[Fernando Hartmann]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0
Build ID: 20180215111455

Steps to reproduce:

Open TB and wait for about 1 minute, and TB crashes with OOM ie: 1e3d563e-a624-4e29-91df-def420180219

Looking to memory usage, it show a fast growth of heap-unclassified


Actual results:

TB crashes in about a minute after started


Expected results:

Use TB normally as before version upgrade

Comment 1

[Wayne Mery (:wsmwk)]

Does crash go away if you disable the lookout addon?


Your crash history, all recent, all with 59.0b1 ...

bp-80e0d57a-41ff-47c6-bf6e-464f20180219    2018-02-19 19:00:08     OOM | small  
bp-f23b4841-bcde-4926-bb42-3f7cd0180219    2018-02-19 18:04:32     OOM | small  
bp-bfe9f6cb-14ed-4375-a105-750e90180219    2018-02-19 17:17:03     mozilla::mailnews::MsgDBReporter::GetPath  
bp-faaff517-d915-4561-9113-a582e0180219    2018-02-19 16:59:09     PtrToNodeMatchEntry  
bp-f734544e-aff1-4087-9968-a5f290180219    2018-02-19 16:56:48     mozilla::mailnews::MsgDBReporter::CollectReports  
bp-1e3d563e-a624-4e29-91df-def420180219    2018-02-19 16:51:18     OOM | small  
bp-6d6b212a-75ba-48fb-9afd-417a10180219    2018-02-19 16:48:09     PtrToNodeMatchEntry  
bp-3bbe7cf8-9550-45ef-8313-176920180219    2018-02-19 16:45:39     OOM | small  
bp-527a85c8-090f-40d6-a6ba-d77890180219    2018-02-19 16:39:02     OOM | small  
bp-f1d6bb0a-d413-4c78-ba8b-600110180219    2018-02-19 16:03:11     OOM | small  
bp-672bb095-c514-4358-b8c3-d49bd0180219    2018-02-19 14:55:05     @0x0 | mozilla::mailnews::MsgDBReporter::CollectReports  
bp-ce4dff63-939d-4060-8786-7cdf70180219    2018-02-19 14:40:18     PtrToNodeMatchEntry  
bp-741b1ad7-d948-415f-9e28-23f000180219    2018-02-19 14:36:52     PtrToNodeMatchEntry  
bp-4f5dd1fb-edb9-45b6-85d2-95e0b0180219    2018-02-19 13:59:27     OOM | small  
bp-0152a450-4d59-489d-a09c-50ad70180219    2018-02-19 12:57:28     OOM | large | mozalloc_abort | mozalloc_handle_oom | moz_xrealloc | nsTArray_base<T>::EnsureCapacity<T> | nsTArray_Impl<T>::AppendElement<T> | mozilla::dom::FragmentOrElement::CanSkip  
bp-d0f4ee03-5871-43b0-ae0d-6c8ae0180219    2018-02-19 12:53:13     PtrToNodeMatchEntry  
bp-83c139c6-e124-4dce-8ce2-2ea560180219    2018-02-19 12:50:51     OOM | small

Comment 2

[Fernando Hartmann]

(In reply to Wayne Mery (:wsmwk) from comment #1)
> Does crash go away if you disable the lookout addon?

No, I disabled lookout addon and the crash happened again ID: 34eaad1c-6f41-4232-95e6-15f8e0180219

Comment 3

[Wayne Mery (:wsmwk)]

bp-34eaad1c-6f41-4232-95e6-15f8e0180219 is another OOM | small

PtrToNodeMatchEntry is Bug 1353702 - which seems to be back on the map (graph [1]) with 59.0b1 and 60.0a1.  bp-cfba0ece-03e3-4c6e-a30d-910ec0180215 talks about rss feeds.  (I am working with another user experiencing crashes who only uses RSS)

[1] https://crash-stats.mozilla.com/signature/?product=Thunderbird&release_channel=beta&release_channel=nightly&_sort=-version&_sort=email&_sort=-date&signature=PtrToNodeMatchEntry&date=%3E%3D2017-08-19T16%3A07%3A41.000Z&date=%3C2018-02-19T15%3A07%3A41.000Z#graphs

Comment 4

[Wayne Mery (:wsmwk)]

Does crash go away if you install previous beta?
 https://releases.mozilla.org/pub/thunderbird/releases/58.0b3/win32/en-US/Thunderbird%20Setup%2058.0b3.exe

Comment 5

[Fernando Hartmann]

(In reply to Wayne Mery (:wsmwk) from comment #4)
> Does crash go away if you install previous beta?
>  https://releases.mozilla.org/pub/thunderbird/releases/58.0b3/win32/en-US/
> Thunderbird%20Setup%2058.0b3.exe

Yes, the crash got away when I installed the old (58b3) version over the 59b1.

Comment 6

[Fernando Hartmann]

For the records, I just upgraded to 59b2 and the problem persist, crashes:
* ID: c6fc94e8-66bb-4bec-9de2-1c08c0180302
* ID: 56b3d79a-8cdb-4faa-b426-221550180302
* ID: 44d78a17-0c44-4ca1-9c3e-2f7580180302

Comment 7

[Fernando Hartmann]

The same problem with TB 60b1
* ID: 058c6b31-f5a5-4860-a8cb-71a530180327
* ID: 043e66ad-272d-421e-8d53-7a51c0180327
* ID: aa562b73-6705-4cda-8b54-487500180327

If should I stop testing new beta version and publishing here, please let me know.

Comment 8

[Wayne Mery (:wsmwk)]

Feel free to stay on the beta if it is not a hardship.  

Unfortunately still no idea what the cause is.  All the crashes you listed are OOM | small.  If that should change, it might get interesting.

Comment 9

[Wayne Mery (:wsmwk)]

Do you have RSS feeds defined?

Comment 10

[Fernando Hartmann]

(In reply to Wayne Mery (:wsmwk) from comment #9)
> Do you have RSS feeds defined?

Yes, actually I've 50+ RSS feeds divided in 3 accounts.

Comment 11

[Wayne Mery (:wsmwk)]

Have you tried these earlier betas?
https://archive.mozilla.org/pub/thunderbird/releases/58.0b1/win32/en-US/Thunderbird%20Setup%2058.0b1.exe
https://archive.mozilla.org/pub/thunderbird/releases/57.0b1/win32/en-US/Thunderbird%20Setup%2057.0b1.exe

Comment 12

[Wayne Mery (:wsmwk)]

bp-1e3d563e-a624-4e29-91df-def420180219 has XML parser on stack which jcranmer pointed out is on stack of his crash bp-594a5e05-0864-43c9-b3a2-d21580180328 and he has RSS feeds.  ref also bug 1353704 comment 11 regarding RSS

0 	mozglue.dll	mozalloc_abort(char const* const)	memory/mozalloc/mozalloc_abort.cpp:33
1 	mozglue.dll	mozalloc_handle_oom(unsigned int)	memory/mozalloc/mozalloc_oom.cpp:54
2 	mozglue.dll	moz_xmalloc	memory/mozalloc/mozalloc.cpp:72
3 	xul.dll	NS_NewXMLElement(mozilla::dom::Element**, already_AddRefed<mozilla::dom::NodeInfo>&&)	dom/xml/nsXMLElement.cpp:19
4 	xul.dll	NS_NewElement(mozilla::dom::Element**, already_AddRefed<mozilla::dom::NodeInfo>&&, mozilla::dom::FromParser, nsTSubstring<char16_t> const*)	dom/base/nsNameSpaceManager.cpp:220
5 	xul.dll	nsXMLContentSink::CreateElement(char16_t const**, unsigned int, mozilla::dom::NodeInfo*, unsigned int, nsIContent**, bool*, mozilla::dom::FromParser)	dom/xml/nsXMLContentSink.cpp:460
6 	xul.dll	nsXMLContentSink::HandleStartElement(char16_t const*, char16_t const**, unsigned int, unsigned int, bool)	dom/xml/nsXMLContentSink.cpp:992
7 	xul.dll	nsXMLContentSink::HandleStartElement(char16_t const*, char16_t const**, unsigned int, unsigned int)	dom/xml/nsXMLContentSink.cpp:950
8 	xul.dll	nsExpatDriver::HandleStartElement(char16_t const*, char16_t const**)	parser/htmlparser/nsExpatDriver.cpp:324
9 	xul.dll	Driver_HandleStartElement	parser/htmlparser/nsExpatDriver.cpp:64
10 	xul.dll	doContent	parser/expat/lib/xmlparse.c:2442
11 	xul.dll	contentProcessor	parser/expat/lib/xmlparse.c:2098
12 	xul.dll	MOZ_XML_ParseBuffer	parser/expat/lib/xmlparse.c:1658
13 	xul.dll	MOZ_XML_Parse	parser/expat/lib/xmlparse.c:1629
14 	xul.dll	nsExpatDriver::ParseBuffer(char16_t const*, unsigned int, bool, unsigned int*)	parser/htmlparser/nsExpatDriver.cpp:887
15 	xul.dll	nsExpatDriver::ConsumeToken(nsScanner&, bool&)	parser/htmlparser/nsExpatDriver.cpp:985
16 	xul.dll	nsParser::Tokenize(bool)	parser/htmlparser/nsParser.cpp:1539
17 	xul.dll	nsParser::ResumeParse(bool, bool, bool)	parser/htmlparser/nsParser.cpp:1055
18 	xul.dll	nsParser::OnDataAvailable(nsIRequest*, nsISupports*, nsIInputStream*, unsigned __int64, unsigned int)	parser/htmlparser/nsParser.cpp:1437
19 	xul.dll	mozilla::dom::XMLHttpRequestMainThread::StreamReaderFunc(nsIInputStream*, void*, char const*, unsigned int, unsigned int, unsigned int*)	dom/xhr/XMLHttpRequestMainThread.cpp:1759
20 	xul.dll	nsPipeInputStream::ReadSegments(nsresult (*)(nsIInputStream*, void*, char const*, unsigned int, unsigned int, unsigned int*), void*, unsigned int, unsigned int*)	xpcom/io/nsPipe3.cpp:1448
21 	xul.dll	mozilla::dom::XMLHttpRequestMainThread::OnDataAvailable(nsIRequest*, nsISupports*, nsIInputStream*, unsigned __int64, unsigned int)	dom/xhr/XMLHttpRequestMainThread.cpp:1926
22 	xul.dll	mozilla::net::nsStreamListenerWrapper::OnDataAvailable(nsIRequest*, nsISupports*, nsIInputStream*, unsigned __int64, unsigned int)	netwerk/base/nsStreamListenerWrapper.h:31
23 	xul.dll	nsBaseChannel::OnDataAvailable(nsIRequest*, nsISupports*, nsIInputStream*, unsigned __int64, unsigned int)	netwerk/base/nsBaseChannel.cpp:904
24 	xul.dll	nsInputStreamPump::OnStateTransfer()	netwerk/base/nsInputStreamPump.cpp:591
25 	xul.dll	nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*)	netwerk/base/nsInputStreamPump.cpp:424
26 	xul.dll	nsInputStreamReadyEvent::Run()	xpcom/io/nsStreamUtils.cpp:97
27 	xul.dll	nsThread::ProcessNextEvent(bool, bool*)	xpcom/threads/nsThread.cpp:1040
28 	ntdll.dll	TimeToDaysAndFraction	
29 	kernelbase.dll	SystemTimeToFileTime	
30 	xul.dll	NS_ProcessNextEvent(nsIThread*, bool)	xpcom/threads/nsThreadUtils.cpp:517
31 	xul.dll	mozilla::dom::XMLHttpRequestMainThread::SendInternal(mozilla::dom::BodyExtractorBase const*)	dom/xhr/XMLHttpRequestMainThread.cpp:3110

Comment 13

[Fernando Hartmann]

(In reply to Wayne Mery (:wsmwk) from comment #11)
> Have you tried these earlier betas?
> https://archive.mozilla.org/pub/thunderbird/releases/58.0b1/win32/en-US/
> Thunderbird%20Setup%2058.0b1.exe
> https://archive.mozilla.org/pub/thunderbird/releases/57.0b1/win32/en-US/
> Thunderbird%20Setup%2057.0b1.exe

Yes, I tried them, and they worked without any OOM crash

Comment 14

[Joshua Cranmer [:jcranmer]]

Some more testing locally:

* Disabling all feeds seems to make it more stable.
* Most of the memory is stuck in heap-unclassified. 1,442.60MB of 1,719.06MB in one report I managed to extract. (Note that "normal" memory usage here should run about 150MB or so in Windows' memory column).
* There's some logging information for RSS if you up the log level to debug in about:config (Feeds.logging.console, set it to debug). One of my feeds apparently had 10,000 items to parse. The RSS feed in question is: https://bugzilla.mozilla.org/buglist.cgi?chfieldfrom=-1&chfieldto=Now&component=Layout&component=Style%20System%20%28CSS%29&component=XForms&emailassigned_to1=1&emailassigned_to2=1&emailqa_contact2=1&emailreporter2=1&field-1-0-0=component&field-1-1-0=product&known_name=Core_update&product=Core&query_format=advanced&type-1-0-0=anyexact&type-1-1-0=anyexact&value-1-0-0=Layout%2CStyle%20System%20%28CSS%29%2CXForms&value-1-1-0=Core&title=Bug%20List:%20Style%20updates&ctype=atom

It is a ~22MB file, I kind of see why the DOM construction might chew up a load of memory.

After deleting the offending feed from the list of feeds to update, the near-immediate crashes don't seem to be an issue. I still have to see if there is a slower, more persistent leak though.

Comment 15

[Wayne Mery (:wsmwk)]

rctgamer3's mozregression results - https://hg.mozilla.org/comm-central/pushloghtml?fromchange=c387f1f2e93e85eafa3e1536ceee229111b9ca69&tochange=1654a1adc838380b2262f3981986d61cf9ee4a80

So regression is either bug 1420473 or something in roughly https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2017-11-29+11%3A05%3A00&enddate=2017-11-29+23%3A59%3A00

Comment 16

[Jorg K (CEST = GMT+2)]

Will be fixed by partial backout in bug 1420473.

Comment 17

[Jorg K (CEST = GMT+2)]

https://hg.mozilla.org/comm-central/rev/9f58c4a6ec4a
Bug 1420473 - Partially backed out changeset 1654a1adc838 (FeedUtils.jsm) to fix crash (bug 1439393). a=jorgk

Comment 18

[Jorg K (CEST = GMT+2)]

Beta (TB 60 beta 2):
https://hg.mozilla.org/releases/comm-beta/rev/4d755f1cf38cbd870ffb3e2625a26c1d2ed51208

Comment 19

[Wayne Mery (:wsmwk)]

Jorg, thanks for the uplift. I have additional users confirming this patch helps

Comment 20

[Wayne Mery (:wsmwk)]

Fernando, you should find this crash is gone in version 60 beta.

We an see in the graph that the crash rate did go down ~April
https://crash-stats.mozilla.com/signature/?product=Thunderbird&_sort=email&_sort=-date&version=60.0b3&version=60.0b2&version=60.0b1&version=60.0b4&version=60.0b5&version=59.0b2&version=59.0b14&signature=OOM%20%7C%20small&date=%3E%3D2017-11-06T08%3A50%3A23.000Z&date=%3C2018-05-06T09%3A50%3A23.000Z#graphs