945702 - Excessive RAM consumption on session restore

Status: NEW

(Firefox :: Session Restore, defect)

Description

[Xuân Baldauf]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0 (Beta/Release)
Build ID: 2013102400

Steps to reproduce:

1. Have 6000 tabs in about 20 windows, a machine with at least 4GB RAM, up to 16GB RAM.
2. (Optional) Have the BarTab Lite X extension https://addons.mozilla.org/de/firefox/addon/bartab-lite-x/ to be able to unload tabs.
3. (Optional) Have the Session Manager extension http://sessionmanager.mozdev.org/ to be able to store and load sessions at any time.
4. Observe the RAM usage of Firefox. It will be around 3GB to 4GB.
5. Close Firefox.
6. Open Firefox. Thus trigger a session restore.


Actual results:

1. The session will be attempted to be restored. The windows may be spawned.
2. The RAM usage will rise to about 3GB to 4GB.
3. Suddenly, the RAM usage will rise much more sharply, exceeding 13GB.
4. The system will go into swap-related thrashing, and thus will be unusable, probably until you reboot.


Expected results:

1. The session will be attempted to be restored. The windows may be spawned.
2. The RAM usage will rise to about 3GB to 4GB.
3. The RAM usage will stay at 3GB to 4GB
4. The system will not go into swap-related thrashing, and thus stay usable.

Comment 1

[Xuân Baldauf]

Analysis of this bug shows that firefox stays in the CacheChildren() methods during the session restore phase where excessive RAM consumption happens. It seems that filling these caches do not seem to be necessary for normal Firefox operation. It seems that filling these caches seems to trigger loading all these tabs into RAM, which may explain the excessive RAM consumption.

Disabling filling these caches fixes the bug, and allows to actually make Firefox usable. It also seems that Firefox' startup is faster with filling these caches disabled even for sessions which do not contain as many tabs.


This patch disables filling these caches:


diff -r d20d499b219f accessible/src/generic/Accessible.cpp
--- a/accessible/src/generic/Accessible.cpp     Tue Nov 12 18:27:48 2013 -0500
+++ b/accessible/src/generic/Accessible.cpp     Tue Dec 03 15:14:57 2013 +0100
@@ -3097,13 +3097,15 @@
 void
 Accessible::CacheChildren()
 {
-  DocAccessible* doc = Document();
-  NS_ENSURE_TRUE_VOID(doc);
-
-  TreeWalker walker(this, mContent);
-
-  Accessible* child = nullptr;
-  while ((child = walker.NextChild()) && AppendChild(child));
+  if (false) { // Fix bug https://bugzilla.mozilla.org/show_bug.cgi?id=945702
+    DocAccessible* doc = Document();
+    NS_ENSURE_TRUE_VOID(doc);
+  
+    TreeWalker walker(this, mContent);
+  
+    Accessible* child = nullptr;
+    while ((child = walker.NextChild()) && AppendChild(child));
+  }
 }
 
 void
diff -r d20d499b219f accessible/src/generic/DocAccessible.cpp
--- a/accessible/src/generic/DocAccessible.cpp  Tue Nov 12 18:27:48 2013 -0500
+++ b/accessible/src/generic/DocAccessible.cpp  Tue Dec 03 15:14:57 2013 +0100
@@ -1445,16 +1445,18 @@
 void
 DocAccessible::CacheChildren()
 {
-  // Search for accessible children starting from the document element since
-  // some web pages tend to insert elements under it rather than document body.
-  dom::Element* rootElm = mDocumentNode->GetRootElement();
-  if (!rootElm)
-    return;
-
-  TreeWalker walker(this, rootElm);
-
-  Accessible* child = nullptr;
-  while ((child = walker.NextChild()) && AppendChild(child));
+  if (false) { // Fix bug https://bugzilla.mozilla.org/show_bug.cgi?id=945702
+    // Search for accessible children starting from the document element since
+    // some web pages tend to insert elements under it rather than document body.
+    dom::Element* rootElm = mDocumentNode->GetRootElement();
+    if (!rootElm)
+      return;
+  
+    TreeWalker walker(this, rootElm);
+  
+    Accessible* child = nullptr;
+    while ((child = walker.NextChild()) && AppendChild(child));
+  }
 }
 
 ////////////////////////////////////////////////////////////////////////////////

Comment 2

[Cornel Ionce [:noni] [Hubs QA]]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/25.0 

Confirming this behaviour on latest release (Build ID: 20131205075310), the system becomes unusable until a reboot using the STR from comment 0.

Tested on a machine with 8GB RAM, the consumption filling the entire memory.

Comment 3

[Tim Taubert [:ttaubert] (inactive)]

6000 tabs, holy cow. Interesting that a11y is assumed to be the problem here - I unfortunately have no clue about the purpose of Accessible::CacheChildren().

Alexander, can you maybe shed some light on this? Is this something we do for every user out there or is that enabled by some a11y pref? Is it possible that there is some leak in there?

Xuân, how exactly did you figure out what's causing the excessive RAM usage?

Comment 4

[alexander :surkov (:asurkov)]

(In reply to Tim Taubert [:ttaubert] from comment #3)
> 6000 tabs, holy cow. Interesting that a11y is assumed to be the problem here
> - I unfortunately have no clue about the purpose of
> Accessible::CacheChildren().
>
> Alexander, can you maybe shed some light on this? Is this something we do
> for every user out there or is that enabled by some a11y pref? Is it
> possible that there is some leak in there?

6000 tabs probably means there are no leaks. I'm curious how DOM/layout can be kept so small: a11y needs to make same trick.

additionally, we should resurrect our telemetry branch (in particular bug 570785) and continue the work on memory reducing (like having separate trees on each platform).

Comment 5

[alexander :surkov (:asurkov)]

Session restore shouldn't trigger all tabs loading into memory right away (neither DOM, layout nor a11y). But if a11y is enabled then something triggers it. Figuring out what is it we should be able to a fix the problem. CC'ing Boris for ideas.

Comment 6

[Tim Taubert [:ttaubert] (inactive)]

(In reply to alexander :surkov from comment #5)
> Session restore shouldn't trigger all tabs loading into memory right away

It doesn't by default. It loads everything however with browser.sessionstore.restore_on_demand=false. With that set to true all we do is create about:blank tabs, 6000 of them.

Comment 7

[Boris Zbarsky [:bzbarsky]]

Breakpoint on a LoadURI for an http:// URI and see what the C++ and JS stacks are?

Comment 8

[alexander :surkov (:asurkov)]

so is the problem that Firefox loads 6000 about:blank and it takes 13GB or it loads 6000 web sites?

Comment 9

[Xuân Baldauf]

(In reply to Tim Taubert [:ttaubert] from comment #3)
> Xuân, how exactly did you figure out what's causing the excessive RAM usage?

I used GDB, looked for the characteristic RAM increase, interrupted execution and produced a stack dump. I repeated this process multiple times to sample where the problem happened.

Comment 10

[The 8472]

> Interesting that a11y is assumed to be the problem here 

I've described similar problems in bug 1187587, setting accessibility.force_disabled = 1 may help here if accessibility features are not needed.