1446838 - Memory leak across page navigations when foreground tab is always busy

Status: NEW

(Core :: DOM: Core & HTML, defect, P3)

Description

[Daniel Wagner-Hall]

Attachment: Example webdriver script in Rust which reproduces the issue. Its only dependency is webdriver_client = "0.2.0"

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.162 Safari/537.36

Steps to reproduce:

I've been running a WebDriver script which navigates to a long list of URLs quickly.

I have also reproduced this by manually visiting the same list of URLs quickly manually, so this is not a WebDriver specific bug.

Interestingly, if I run the same script, but open a new blank tab which is focussed (i.e. a tab in the background is constantly churning visiting new pages), I never see the memory use go above 500MB.

However, if Firefox itself is headless (e.g. has the MOZ_HEADLESS environment variable set), this uncontrollable growth is still reproducible.

If I add 1 second pauses after every page load, or 10 second pauses after every batch of 50 page loads, I still see uncontrollable growth.


Actual results:

Memory usage of "FirefoxCP Web Content" process kept growing very quickly. For a list of 1800 URLs with fairly simple content, I've seen the process grow to over 2.5GB. Visiting any one of those pages, the process takes less than 500M of memory.


Expected results:

Memory should not grow uncontrollably.

My guess is that what's happening is that there's a heuristic to delay GC if a tab is active to give a better interactivity. If this is the case, I would recommend altering the heuristic to not trigger if more than a certain amount of RAM is being spent on objects from unloaded pages. I would tentatively propose that either 500MB, or "5% of system RAM" would both be reasonable limits.

Comment 1

[Emil Pasca [:emilpasca]]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0

I have tested this issue on Mac 10.13 with the latest Firefox release (59.0.1) and the latest Nightly (61.0a1-20180321040527) and managed to reproduce it.

After manually opening > 1260 links from the provided list (see attachment 8960047 [details]), each in its own tab, I have observed that in the "Activity Monitor" the Memory usage is about 2.2GB for each of "FirefoxCP Web Content" process.

I'm going to assign the "Core:DOM" component for this issue which seems to be the most appropriate one.

Comment 2

[Henri Sivonen (:hsivonen)]

mccr8, does this look like CC or GC to you?

Comment 3

[Andrew McCreight [:mccr8]]

(In reply to Henri Sivonen (:hsivonen) from comment #2)
> mccr8, does this look like CC or GC to you?

This is probably the GC scheduling heuristics which live in dom/base/nsJSEnvironment.cpp. The GC has its own heuristics around compacting which might possibly be coming into play here, but I would guess that's not the issue.

Generally speaking, we don't try to handle the case of rapidly navigating to hundreds of pages, because no user is going to experience it. Dealing with it really requires a special mode where we disregard all concerns except freeing memory. This does cause some issues in our own test automation. There's a privileged function on nsDOMWindowUtils called runNextCollectorTimer() which tries to make us do more collector work than we'd do normally.

It is a little surprising that there's no improvement with a 1 second delay. I'd have to try to reproduce it to figure out if we're not starting enough GCs/CCs or if we don't have a large enough budget when they are running.

Comment 4

[Olli Pettay [:smaug][[e-mail]]]

Background tabs run a lot slower in general comparing to foreground. The DOM for them is created from html using idle timer, animationFrameCallbacks run very rarely, timers run rarely etc. And we create a lot less any graphics related objects. So, less memory is used.