670765 - firefox 5.0.x to 7.0.x: memory management issues

Status: RESOLVED DUPLICATE of bug 669034

(Firefox :: General, defect)

Description

[Rafał Mużyło]

Mozilla/5.0 (X11; Linux i686; rv:2.0.1) Gecko/20110519 Firefox/4.0.1 (this one works)
Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/20100101 Firefox/5.0 (this one does not)

I happen to use a huge session (200+ tabs shared between 2 windows).
In 4.0.1 this works more or less OK (barring occasional slowdowns, likely sqlite related), in 5.0 same session crashes seemingly randomly with out of memory messages printed to the console.
I can't reliably trigger the crash, in fact I usually manage to load the whole session and do quite a bit of browsing before it happens. However it *seems* the crashes happen while one of the sqlite databases is being updated.

To make things a bit more interesting, I need to disable java plugin first (sun-jdk 1.6.0.26), as it dies earlier with:
Error occured during initialization of VM
Could not reserve enough space for object heap
firefox-bin: ../../../../src/plugin/share/plugin2/common/LocalFramePusher.h:25: LocalFramePusher::LocalFramePusher(int): Assertion env!=__null failed
(though I'm unsure whether that actually affects the "out of memory" crash)

Comment 1

[George Carstoiu]

Mozilla/5.0 (X11; Linux i686; rv:8.0a1) Gecko/20110711 Firefox/8.0a1

Can you please copy and paste the links from about:crashes the next time a crash occurs?

Also, please try a clean profile:
http://support.mozilla.com/en-US/kb/Managing-profiles

Thanks!

Comment 2

[Rafał Mużyło]

If it wasn't clear from my initial report - the catch here is that I'm running close to memory saturation (only about 200MB of normal memory free + significant swap space).
Barring the occasional slowdowns upon access to the sqlite databases, 4.0.1 handled it quite gracefully, 5.0, on the other hand, just crashes in similar conditions.

Comment 3

[Rafał Mużyło]

So, I've just built 6.0.
This time, instead of "out of memory" error, I get plain segfault.

0xb79f8176 in JSCompartment::wrap(JSContext*, js::Value*) ()
   from /usr/lib/firefox/libxul.so
(gdb) bt
#0  0xb79f8176 in JSCompartment::wrap(JSContext*, js::Value*) ()
   from /usr/lib/firefox/libxul.so
#1  0xb79f7e50 in JSCompartment::wrap(JSContext*, JSObject**) ()
   from /usr/lib/firefox/libxul.so
#2  0xb79d4b8b in JS_WrapObject () from /usr/lib/firefox/libxul.so
#3  0xb7363e7c in ?? () from /usr/lib/firefox/libxul.so
#4  0xb73674c3 in ?? () from /usr/lib/firefox/libxul.so
#5  0xb737c7d2 in ?? () from /usr/lib/firefox/libxul.so
#6  0xb737688c in ?? () from /usr/lib/firefox/libxul.so
#7  0xb7875d12 in ?? () from /usr/lib/firefox/libxul.so
#8  0xb6ee555f in ?? () from /usr/lib/firefox/libxul.so
#9  0xb6ee5bf6 in ?? () from /usr/lib/firefox/libxul.so
#10 0xb6f01f30 in ?? () from /usr/lib/firefox/libxul.so
#11 0xb6f02e19 in ?? () from /usr/lib/firefox/libxul.so
#12 0xb6c70bfb in ?? () from /usr/lib/firefox/libxul.so
#13 0xb6c75b09 in ?? () from /usr/lib/firefox/libxul.so
#14 0xb6c75ba6 in ?? () from /usr/lib/firefox/libxul.so
#15 0xb70303a4 in ?? () from /usr/lib/firefox/libxul.so
#16 0xb7031d4b in ?? () from /usr/lib/firefox/libxul.so
#17 0xb702d2c1 in ?? () from /usr/lib/firefox/libxul.so
#18 0xb764d422 in ?? () from /usr/lib/firefox/libxul.so
#19 0xb7659ba2 in ?? () from /usr/lib/firefox/libxul.so
#20 0xb7659cdc in ?? () from /usr/lib/firefox/libxul.so
#21 0xb765a7bc in ?? () from /usr/lib/firefox/libxul.so
#22 0xb5a01064 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#23 0xb5da99e0 in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#24 0xb5dbc2be in ?? () from /usr/lib/libgobject-2.0.so.0
#25 0xb5dc513a in g_signal_emit_valist () from /usr/lib/libgobject-2.0.so.0
#26 0xb5dc5522 in g_signal_emit () from /usr/lib/libgobject-2.0.so.0
#27 0xb5b2e966 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#28 0xb5b45a5f in gtk_window_propagate_key_event ()
   from /usr/lib/libgtk-x11-2.0.so.0
#29 0xb5b4870c in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#30 0xb5a01064 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#31 0xb5da8337 in ?? () from /usr/lib/libgobject-2.0.so.0
#32 0xb5da99e0 in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#33 0xb5dbc0b5 in ?? () from /usr/lib/libgobject-2.0.so.0
#34 0xb5dc513a in g_signal_emit_valist () from /usr/lib/libgobject-2.0.so.0
#35 0xb5dc5522 in g_signal_emit () from /usr/lib/libgobject-2.0.so.0
#36 0xb5b2e966 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#37 0xb59ff2c9 in gtk_propagate_event () from /usr/lib/libgtk-x11-2.0.so.0
#38 0xb59ff5e7 in gtk_main_do_event () from /usr/lib/libgtk-x11-2.0.so.0
#39 0xb586aa9a in ?? () from /usr/lib/libgdk-x11-2.0.so.0
#40 0xb5cd5a31 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#41 0xb5cd6200 in ?? () from /usr/lib/libglib-2.0.so.0
#42 0xb5cd64b4 in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0
#43 0xb765aaa5 in ?? () from /usr/lib/firefox/libxul.so
#44 0xb767659d in ?? () from /usr/lib/firefox/libxul.so
#45 0xb7861fca in ?? () from /usr/lib/firefox/libxul.so
#46 0xb781eb63 in ?? () from /usr/lib/firefox/libxul.so
#47 0xb7737823 in ?? () from /usr/lib/firefox/libxul.so
#48 0xb789309b in ?? () from /usr/lib/firefox/libxul.so
#49 0xb7675ecd in ?? () from /usr/lib/firefox/libxul.so
#50 0xb74c3b4f in ?? () from /usr/lib/firefox/libxul.so
#51 0xb6a76095 in XRE_main () from /usr/lib/firefox/libxul.so
#52 0x08049588 in ?? ()
#53 0xb669de16 in __libc_start_main () from /lib/libc.so.6
#54 0x080493a1 in ?? ()

OK, I *know* this backtrace is without debug symbols.

Unfortunately, after applying backported patch for bug 605290, I'm back to oom crash:

#0  0xb7fe1424 in __kernel_vsyscall ()
#1  0xb66b1e61 in raise () from /lib/libc.so.6
#2  0xb66b36ac in abort () from /lib/libc.so.6
#3  0xb5eb5115 in mozalloc_abort(char const*) ()
   from /usr/lib/firefox/libmozalloc.so
#4  0xb5eb5140 in mozalloc_handle_oom() () from /usr/lib/firefox/libmozalloc.so
#5  0xb5eb4dd0 in moz_xrealloc () from /usr/lib/firefox/libmozalloc.so
#6  0xb6aa793d in ?? () from /usr/lib/firefox/libxul.so
#7  0xb7370652 in ?? () from /usr/lib/firefox/libxul.so
#8  0xb737f47d in ?? () from /usr/lib/firefox/libxul.so
#9  0xb737f53a in ?? () from /usr/lib/firefox/libxul.so
#10 0xb737f582 in ?? () from /usr/lib/firefox/libxul.so
#11 0xb737e408 in ?? () from /usr/lib/firefox/libxul.so
#12 0xb737f8cd in ?? () from /usr/lib/firefox/libxul.so
#13 0xb7388aaf in ?? () from /usr/lib/firefox/libxul.so
#14 0xb7a2b384 in ?? () from /usr/lib/firefox/libxul.so
#15 0xb7a2c5c2 in JSCompartment::finalizeObjectArenaLists(JSContext*) ()
   from /usr/lib/firefox/libxul.so
#16 0xb7a2d14c in ?? () from /usr/lib/firefox/libxul.so
#17 0xb7a2d56b in ?? () from /usr/lib/firefox/libxul.so
#18 0xb7a2d6b1 in ?? () from /usr/lib/firefox/libxul.so
#19 0xb7a2df8a in ?? () from /usr/lib/firefox/libxul.so
#20 0xb7aaf1f3 in ?? () from /usr/lib/firefox/libxul.so
#21 0xb7bee3fc in ?? () from /usr/lib/firefox/libxul.so
#22 0xb7a372ad in ?? () from /usr/lib/firefox/libxul.so
#23 0xb7a375ff in ?? () from /usr/lib/firefox/libxul.so
#24 0xb7a380f1 in ?? () from /usr/lib/firefox/libxul.so
#25 0xb79daf75 in JS_CallFunctionValue () from /usr/lib/firefox/libxul.so
#26 0xb737c915 in ?? () from /usr/lib/firefox/libxul.so
#27 0xb73768ec in ?? () from /usr/lib/firefox/libxul.so
#28 0xb7875d72 in ?? () from /usr/lib/firefox/libxul.so
#29 0xb6e08a13 in ?? () from /usr/lib/firefox/libxul.so
#30 0xb6d23cee in ?? () from /usr/lib/firefox/libxul.so
#31 0xb6d2747d in ?? () from /usr/lib/firefox/libxul.so
#32 0xb6e880c0 in ?? () from /usr/lib/firefox/libxul.so
#33 0xb6e88166 in ?? () from /usr/lib/firefox/libxul.so
#34 0xb7147a86 in ?? () from /usr/lib/firefox/libxul.so
#35 0xb7143c22 in ?? () from /usr/lib/firefox/libxul.so
#36 0xb71487b8 in ?? () from /usr/lib/firefox/libxul.so
#37 0xb714bff4 in ?? () from /usr/lib/firefox/libxul.so
#38 0xb786212b in ?? () from /usr/lib/firefox/libxul.so
#39 0xb781ebc3 in ?? () from /usr/lib/firefox/libxul.so
#40 0xb7737883 in ?? () from /usr/lib/firefox/libxul.so
#41 0xb78930fb in ?? () from /usr/lib/firefox/libxul.so
#42 0xb7675f2d in ?? () from /usr/lib/firefox/libxul.so
#43 0xb74c3baf in ?? () from /usr/lib/firefox/libxul.so
#44 0xb6a760f5 in XRE_main () from /usr/lib/firefox/libxul.so
#45 0x08049588 in ?? ()
#46 0xb669de16 in __libc_start_main () from /lib/libc.so.6
#47 0x080493a1 in ?? ()

I'm getting a feeling, that somebody put a performance fix somewhere in xulrunner, that results in no performance at all on a memory constrained system.

Comment 4

[Rafał Mużyło]

I suspect that patch from bug 667915 would help, if not for a minor catch:
while most of it seems to backport relatively clean, the parts for js/src/vm are simply a horror (mostly due to http://hg.mozilla.org/projects/jaegermonkey/rev/bb9e5496b0ac that happened between 6.0 and that bug).

On a semi-related note: I find cgit/gitweb interfaces clearly superior to the one used on hg.mozilla.org for mercurial.

Comment 5

[Rafał Mużyło]

Well, I think I'm at a point I should just simply seriously consider using a different browser.
Yesterday 7.0 was released...
I built it with patches from commit 0b2ab968dd32a32d78e4105cf26254250389094b and commit 28be8df0deb7de5209b976009ae8519eac374272, so I think I should have been in the clear in regard of memory usage... but it still reports going OOM and crashes because of it.

The most annoying part of it is that (as I already said) sometimes the whole session does load and I can even browse for a couple minutes, but then it randomly fails.

Just a guess, but I'm beginning to suspect that it's in a way sqlite related, as in the crash seems to happen when one of the sqlite databases is being updated. That is an sqlite file is getting mmaped or something and firefox believes it's out of memory, even though if a bit of swapping was thrown in between, it would be just fine.

Comment 6

[Rafał Mużyło]

So, as yet another attempt to push this forward, I've built firefox with debug.

The result was both surprising and not quite:

* Call to xpconnect wrapped JSObject produced this error:  *
[Exception... "[JavaScript Error: "out of memory" {file: "resource:///components/nsSessionStore.js" line: 3898}]"  nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)"  location: "JS frame :: resource:///components/nsSessionStore.js :: sss_toJSONString :: line 3898"  data: yes]


The size of sessionstore file is 8502450 - perhaps that's just a tad too much for the way it's usually handled ?

Comment 7

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to Rafał Mużyło from comment #6)
> The size of sessionstore file is 8502450 - perhaps that's just a tad too
> much for the way it's usually handled ?

That's 8mb, which while *very* large, is not the biggest I've seen. Perhaps on a memory constrained system though it's causing issues (in combination with everything else). Best thing I can suggest is cleaning up a little bit - close tabs you don't need, wipe recently closed tabs & windows list (in about:config, set max_tabs_undo & max_windows_undo to 0, then reset them to default). That will help a lot. If you have any tabs that are about:sessionrestore, close them (that's often the cause for overly large sessions)

And then lastly, enabling "don't load tabs until selected" would help with memory consumption at startup.

Now beyond that, it could be sqlite related. The safe browsing service usually updates it's database a few minutes into the session. That should be getting better soon as we move away from an entirely db-backed approach.

Comment 8

[Rafał Mużyło]

OK, after a *very* long irc chat, the conclusion seems to be this is, in a way, a dupe of bug 669034.
If browser.sessionstore.max_concurrent_tabs is set to 0, the crash doesn't happen.

I was facing the slowdowns on sessionstore saving even in 4.0.

:whine: But really, why does this needs to be that difficult to fix (at least if hardware upgrade isn't really an option atm) ?