1291294 - Crash in OOM | large | mozalloc_abort | mozalloc_handle_oom | moz_xrealloc | NS_CycleCollectorSuspect3

Status: RESOLVED WONTFIX

(Core :: XPCOM, defect)

Description

[alex_mayorga]

This bug was filed from the Socorro interface and is 
report bp-b73518c5-b313-4a27-9224-c8c272160802.
=============================================================

¡Hola!

Trying to help over at SuMo https://support.mozilla.org/en-US/questions/1123864?page=3#answer-902668

User has a Firefox that is crash happy due to OOM issues caused by facebook.com

349 crashes in the past week at https://crash-stats.mozilla.com/signature/?product=Firefox&signature=OOM%20%7C%20large%20%7C%20mozalloc_abort%20%7C%20mozalloc_handle_oom%20%7C%20moz_xrealloc%20%7C%20NS_CycleCollectorSuspect3

¡Gracias!
Alex

Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	mozglue.dll 	mozalloc_abort(char const* const) 	memory/mozalloc/mozalloc_abort.cpp:33
1 	mozglue.dll 	mozalloc_handle_oom(unsigned int) 	memory/mozalloc/mozalloc_oom.cpp:46
2 	mozglue.dll 	moz_xrealloc 	memory/mozalloc/mozalloc.cpp:107
3 	xul.dll 	NS_CycleCollectorSuspect3 	xpcom/base/nsCycleCollector.cpp:3999
4 	xul.dll 	nsTArray_base<nsTArrayInfallibleAllocator, nsTArray_CopyWithMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator>(unsigned int, unsigned int) 	xpcom/glue/nsTArray-inl.h:183
5 	xul.dll 	nsContentList::PopulateSelf(unsigned int) 	dom/base/nsContentList.cpp:935
6 	xul.dll 	nsContentList::BringSelfUpToDate(bool) 	dom/base/nsContentList.cpp:1000
7 	xul.dll 	nsContentList::GetLength(unsigned int*) 	dom/base/nsContentList.cpp:618
8 	xul.dll 	mozilla::dom::HTMLCollectionBinding::get_length 	obj-firefox/dom/bindings/HTMLCollectionBinding.cpp:25
9 	xul.dll 	nsTimerImpl::InitWithNameableFuncCallback(void (*)(nsITimer*, void*), void*, unsigned int, unsigned int, void (*)(nsITimer*, void*, char*, unsigned int)) 	xpcom/threads/nsTimerImpl.cpp:319
10 	xul.dll 	js::Shape::search<0>(js::ExclusiveContext*, js::Shape*, jsid, js::ShapeTable::Entry**) 	js/src/vm/Shape-inl.h:75
11 	xul.dll 	mozilla::dom::GetPropertyOnPrototype(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<jsid>, bool*, JS::MutableHandle<JS::Value>) 	dom/bindings/BindingUtils.cpp:1819
12 		@0xae641415

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Hey mccr8, do you know if there's a meta bug to hang these OOMs off of? I couldn't find one after a quick pass.

Comment 2

[Andrew McCreight [:mccr8]]

It says the allocation is 500kb, which is quite large, but the top of the stack doesn't make any sense to me.

(In reply to Mike Conley (:mconley) - (Needinfo me!) from comment #1)
> Hey mccr8, do you know if there's a meta bug to hang these OOMs off of?

Not that I know of. njn just filed some OOM related bug, but I'm not sure if he wants to put individual OOMs there or not.

Comment 3

[Nicholas Nethercote [inactive]]

> > Hey mccr8, do you know if there's a meta bug to hang these OOMs off of?
> 
> Not that I know of. njn just filed some OOM related bug, but I'm not sure if
> he wants to put individual OOMs there or not.

Not really.

Comment 4

[John Hesling [:John99] (NeedInfo me)]

Just a brief not on the background of the Sumo thread from which this was filed.
(It is rather a long, thread but it stays on topic. One of my recent posts in that thread to Bob is https://support.mozilla.org/en-US/questions/1123864?page=5#answer-902985 )

The user Bob is asking on Sumo specifically about crashes when using Facebook.

Bob is apparently able to reliably and often within minutes reproduce OOM type crashes with various signatures  on Facebook on three separate machines with differing OS versions. Bob is also apparently noting 7GB RAM usage when using Fx64bit on Facebook. I am at present trying to clarify STR.

Presumably it would be more appropriate to file a separate bug about the Facebook OOM issue, but if you need more info from someone able to reproduce this bug1291294 I am sure Bob is willing and eager to help.

Or if any of you are already aware of the facebook OOM issues, and an additional bugs is not needed to be filed please comment so that I do not ask Bob for unnecessary information. 
Any help or advice welcome, Thanks.

Comment 5

[Nicholas Nethercote [inactive]]

John: if comment 0 and comment 4 involve the same user, then it's fine to have everything in this bug. If they are different users, then separate bugs is probably a good idea.

I see you pointed Bob at the documentation for about:memory, which is great. That's definitely the right starting point for investigating this. No progress will be made without the information from about:memory.

Comment 6

[John Hesling [:John99] (NeedInfo me)]

Thanks Nicholas it is the same user

Comment 7

[John Hesling [:John99] (NeedInfo me)]

I have looked again at Bob's Sumo thread & asked Bob for about:memory reports.

This is what Bob describes as STR
Posted today in the same Sumo thread. No facebook account myself so unable to verify STR)
> {Post https://support.mozilla.org/en-US/questions/1123864?page=5#answer-903272 }
>Has ANYONE tried this? It really is simple. 
>Open task manager so you can watch Firefox memory, 
>login to Facebook, got to newsfeed 
>(likely already there, and tap page down until Firefox 32 blows up. 
>64 will not crash - at least until well after using a ridiculous 7 GB of RAM. 
>But that in itself indicates a memory issue.
 
Note previous posts from Bob mentioned about 200 screens of newsfeed being looked at before crashing, however I note  Bob's bp-b73518c5-b313-4a27-9224-c8c272160802 only had an uptime of only about 20 minutes so it seems a relatively quick STR.
Bob also comments in the same Sumo post
> With 49 I cannot disable Plugin Container. If I could, I could try more things. 
>But that eats the memory on Facebook page even though Flash is disabled.
 
I have asked for about:memory reports from the Sumo thread.

Comment 8

[Andrew McCreight [:mccr8]]

Honestly, if they are OOMing just on Facebook, odds are they have some addon that is leaking, or they have some graphics driver issue. Lots of people leave Facebook open all the time without problems, so the question is what is unique for this user. about:memory should tell us whether it is addons or graphics drivers.

Comment 9

[John Hesling [:John99] (NeedInfo me)]

Attachment: abstract 2x about:memory reports as text

The reports are from Bob the person being helped in Comment 0 
The full memory reports are available.

Comment 10

[John Hesling [:John99] (NeedInfo me)]

I have the 64bit Firefox report but it contains sensitive information. I have two 32 bit Firefox full reports but Bob would prefer them not to be posted publicly in case they do contain any confidential information, but is happy for me to forward them to anyone who may be able to help with this.

Anyone able to offer advice or help for Bob's Facebook crashes please ?

Comment 11

[alex_mayorga]

¡Hola Nicholas!

Could you please take a look at https://bugzilla.mozilla.org/attachment.cgi?id=8778707 and confirm if you're interested in seeing https://bugzilla.mozilla.org/show_bug.cgi?id=1291294#c10 ?

¡Gracias!
Alex

Comment 12

[Nicholas Nethercote [inactive]]

The attachment contains two memory reports. The first one looks pretty normal, and is a 32-bit Firefox, as indicated by this:

> 4,095.94 MB (100.0%) -- address-space


The second one is 64-bit Firefox, as indicated by this:

> 8,388,607.94 MB (100.0%) -- address-space

Facebook is taking up an unusually large amount of memory, mostly due to JS stuff:

> 2,043.59 MB (100.0%) -- explicit
> ├──1,062.09 MB (51.97%) -- window-objects
> │  ├──1,051.41 MB (51.45%) -- top(https://www.facebook.com/, id=67)
> │  │  ├────879.56 MB (43.04%) -- active
> │  │  │    ├──866.85 MB (42.42%) -- window(https://www.facebook.com/)
> │  │  │    │  ├──693.53 MB (33.94%) -- js-compartment(https://www.facebook.com/)
> │  │  │    │  │  ├──684.78 MB (33.51%) -- classes
> │  │  │    │  │  │  ├──224.08 MB (10.97%) ++ shapes
> │  │  │    │  │  │  ├──205.60 MB (10.06%) ++ class(Object)/objects
> │  │  │    │  │  │  ├───90.22 MB (04.41%) ++ class(Array)/objects
> │  │  │    │  │  │  ├───73.83 MB (03.61%) ++ class(Function)/objects
> │  │  │    │  │  │  ├───60.34 MB (02.95%) ++ class(Call)/objects
> │  │  │    │  │  │  └───30.70 MB (01.50%) ++ (18 tiny)

Could well be some kind of leak. Whether it's a Facebook leak, an add-on leak, or a Firefox leak is hard to say.

Then among the top-level measurements, "private" and "resident" are surprisingly high (~5 GiB), primarily because "system-heap-allocated" is 3 GiB! How strange. That's memory that's not being allocated by mozjemalloc, which is typically a *much* smaller number. Unfortunately, we have not insight into it. glandium, that's one you might be interested in.

>    1,303.01 MB ── heap-allocated
>    1,401.00 MB ── heap-mapped
>    5,444.78 MB ── private
>    5,059.05 MB ── resident
>    5,056.44 MB ── resident-unique
>        0.00 MB ── shmem-allocated
>        0.00 MB ── shmem-mapped
>    3,144.09 MB ── system-heap-allocated
>    7,063.14 MB ── vsize

Overall, the second memory report is definitely unusual and suspicious, but there's not much to go on in terms of making further progress. Getting the full .json.gz memory report files is unlikely to provide anything more, because these text files include all the interesting stuff in this case.

Comment 13

[alex_mayorga]

¡Hola Mike!

n?'ing you per https://bugzilla.mozilla.org/show_bug.cgi?id=1291294#c12

¡Gracias!
Alex

Comment 14

[John Hesling [:John99] (NeedInfo me)]

I have another about:memory report from Bob on the Firefox 32bit system.
Bob - That is the person mentioned in Comment 0  

This was in Firefox safe mode and resulted in a crash but with a different signature. (Possibly a more generic out of memory signature bp-bp-f709a9be-8459-433b-bfe3-d02302160810  - bug1257387  )

I have the about memory report the following are some of the details, which may be of most relevance. 

 679.81 MB (100.0%) -- explicit
├──234.47 MB (34.49%) -- window-objects
│  ├──224.55 MB (33.03%) -- top(<anonymized-40>, id=40)
│  │  ├──180.57 MB (26.56%) -- active
│  │  │  ├──176.00 MB (25.89%) -- window(<anonymized-64>)
│  │  │  │  ├──140.78 MB (20.71%) -- js-compartment(<anonymized-8>)

3,078.41 MB ── private
2,761.39 MB ── resident
2,756.62 MB ── resident-unique
2,257.99 MB ── system-heap-allocated
3,776.71 MB ── vsize
  109.79 MB ── vsize-max-contiguous

End of Main Process (pid 7628) 

Would using Windows Safe mode help as a troubleshooting step? 
Bob is reluctant to try that unless he knows it is a worthwhile troubleshooting step. 
Bob points out he has this issue on Facebook on three separate PCs with different Windows Versions but has no problem when using an alternative browser. Bob also points out Firefox 64 bit uses a lot of memory but does not crash.
I have permission to forward the full reports, but apparently those are not required.

Comment 15

[Nicholas Nethercote [inactive]]

> Would using Windows Safe mode help as a troubleshooting step? 

Firefox has a safe mode -- are you referring to that? If so, that would be helpful, because it disables add-ons, which will indicate if the problem is caused by an add-on or not. https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode has instructions on how to use safe mode.

Comment 16

[Nicholas Nethercote [inactive]]

Oh, comment 14 mentioned Firefox safe mode. Sorry.

Comment 17

[John Hesling [:John99] (NeedInfo me)]

Yes Bob should be using Firefox's safe mode with all plugins disabled when trying to troubleshoot this issue. The associated crash report confirms safemode 1 & "No extensions were installed."

Comment 18

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'OOM | large | mozalloc_abort | mozalloc_handle_oom | moz_xrealloc | NS_CycleCollectorSuspect3':
 - nightly (version 51): 0 crashes from 2016-08-01.
 - aurora  (version 50): 0 crashes from 2016-08-01.
 - beta    (version 49): 1 crash from 2016-08-02.
 - release (version 48): 126 crashes from 2016-07-25.
 - esr     (version 45): 346 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly       0       0       0
 - aurora        0       0       0
 - beta          0       1       0
 - release      35      36      28
 - esr          29      29      30

Affected platform: Windows

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly
 - aurora
 - beta
 - release #578      #488
 - esr     #209

Comment 19

[alex_mayorga]

Attachment: memory-report.json.gz

¡Hola Nicholas!

So I was trying to reproduce Bob's crashes unsuccessfully.

Yet this Nightly somehow got to a 16,370.93 MB ── vsize and has since stayed there even after 12 or so hours since I closed the facebook.com tab.

Is this normal?

¡Gracias!
Alex

Comment 20

[Nicholas Nethercote [inactive]]

> Yet this Nightly somehow got to a 16,370.93 MB ── vsize and has since stayed
> there even after 12 or so hours since I closed the facebook.com tab.
> 
> Is this normal?

No, it's not normal. I don't recall seeing such a large value, esp. one that's so much higher than any of the other measurements.

Having said that, vsize isn't such an important measure on 64-bit where there are terabytes of virtual memory available. The resident numbers are more important, because they indicate physical memory usage, and those measurements are on the high side but not crazy.

So overall I'm not really sure what to make of it, sorry :(

Comment 21

[John Hesling [:John99] (NeedInfo me)]

It appears this issue is resolved for Bob - person mentioned as having the crashes in comment0 - by use of Fx50 as now reported in the original Sumo thread.

Comment 22

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'OOM | large | mozalloc_abort | mozalloc_handle_oom | moz_xrealloc | NS_CycleCollectorSuspect3':
 - nightly (version 52): 0 crashes from 2016-09-19.
 - aurora  (version 51): 2 crashes from 2016-09-19.
 - beta    (version 50): 16 crashes from 2016-09-20.
 - release (version 49): 1 crash from 2016-09-05.
 - esr     (version 45): 547 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly       0       0
 - aurora        2       0
 - beta         12       4
 - release       1       0
 - esr          56      69

Affected platform: Windows

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly
 - aurora  #554
 - beta    #1885     #485
 - release #20301
 - esr     #197

Comment 23

[alex_mayorga]

¡Hola Nicholas!

So the user (Bob) was expecting the crash to be fixed on 50, it seems the crash is still present but has morphed into a OOM | small, please see https://crash-stats.mozilla.com/report/index/301cefb7-d2a9-4d4b-a205-50be72161116

Shall we file a new bug based on that new report?

¡Gracias!
Alex

Comment 24

[Nicholas Nethercote [inactive]]

"OOM | small" crashes are hard to pin down because lots of different out-of-memory scenarios can trigger them. So I don't know if you can say with confidence that a crash with another signature has morphed into an "OOM | small" crash.

Comment 25

[alex_mayorga]

¡Hola Nicholas!

Please forgive my OOM ignorance, per the reporter he's doing the same STR and now gets this new crash on 50.

Shall we file a new bug based on https://crash-stats.mozilla.com/report/index/301cefb7-d2a9-4d4b-a205-50be72161116 ?

¡Gracias!
Alex

Comment 26

[Nicholas Nethercote [inactive]]

You can file a bug if you like. Unfortunately it'll just be one more "OOM | small" bug report, of which we already have many. When an OOM of this kind occurs it's hard to pin it down to any particular cause, unfortunately, so the bug report is unlikely to result in useful action.

The crash report indicates that the user has plenty of RAM (9 GiB) so you could suggest that he try a "Windows 64-bit" version of Firefox, available at https://www.mozilla.org/en-US/firefox/all/. There's a good chance this will make the problem go away.

Comment 27

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.