Closed Bug 1476032 Opened 6 years ago Closed 5 years ago

2GB structured-clone-holder memory usage with 9 extensions enabled

Categories

(WebExtensions :: Storage, defect, P1)

61 Branch
defect

Tracking

(firefox66 fixed)

RESOLVED FIXED
mozilla66
Tracking Status
firefox66 --- fixed

People

(Reporter: mozbug, Assigned: kmag)

References

Details

(Whiteboard: [MemShrink:P1])

Attachments

(7 files)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0
Build ID: 20180704194937

Steps to reproduce:

Since upgrading to Firefox Quantum I have frequently encountered the case where Firefox uses up several GBs of RAM, until my system runs out and then freezes up. At that point it typically takes about 10-30 mins for my system to recover, generally by killing the worker process, while I am unable to use my machine at all. This happens to me atleast a couple of times a week, and is still happening on FF 61.

Running on Ubuntu GNOME 16.04 without swap space.


Actual results:

It seems to me that each time I load a page, Firefox uses up more memory. A few seconds (5-30) after loading the page, Firefox cleans up the old memory, and memory usage goes back to normal.

The problem occurs when you flick through pages quickly, for example looking at a load of items on a shopping site where you click through to a product, then go back to the previous page, click to the next product etc.
  In this circumstance, if you are flicking through too quickly, then that idle delay is never reached, meaning each page load increases the RAM usage without freeing anything up until the system freezes.

I also encountered this at https://www.ssllabs.com/ssltest/index.html where the page refreshes every few seconds to display progress. Watching the memory usage, it appeared that half the time the refresh was happening just fast enough to stop the memory getting cleared (but not enough to be 100% reproducible).


Expected results:

Firefox should be smarter about it's memory management, perhaps ensuring that page loads of the same site (or even the same page) can share memory such that loading the same page again and again doesn't increase memory usage, and/or ensuring that memory is reclaimed a little more aggressively when it is starting to spiral out of control during frequent page loads.
Hi Sam,

I tested this on Ubuntu 16.04 with latest Nightly, but could not manage to reproduce.  Memory usage looks correct to me.

Could you please provide the memory report? Thanks

Here are the steps to generate a memory report:

1. Wait until memory usage is high enough
2. Open a new tab and type "about:memory" into the address bar and hit enter
3. Click on the "Measure and save..." button
4. Attach the file to this bug.

Detailed information is available here: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/about:memory


My test environment:
Version 	63.0a1
Build ID 	20180718220144
Update Channel 	nightly
User Agent 	Mozilla/5.0 (X11; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
Flags: needinfo?(mozbug)
Attached file Large memory usage
Flags: needinfo?(mozbug)
Attached file After cleanup
OK, so I managed to reproduce it by just clicking some links and going back several times. I switched to the memory tab and saved the report, I then saved another memory report 30 seconds later after Firefox had cleaned up the memory.

The clearly suspicious thing here is in the Web Content (5095) process, which is using 2GB of memory in the first log, mostly on explicit/dom/structured-clone-holder. In the second log after cleaning up, it has dropped to 150MB of memory.
erahm, comment or suggestion on this report? Thanks
Flags: needinfo?(erahm)
Yes, the structured clone holder number is pretty high in the content process.

> 2,197.38 MB (100.0%) -- explicit
> ├──1,962.25 MB (89.30%) -- dom
> │  ├──1,962.20 MB (89.30%) ── structured-clone-holder [2415]

Can you test in safe mode [1]? My guess is this is extension related, if that helps can you try selectively disabling extensions to figure out which one is causing the issue?

[1] https://support.mozilla.org/kb/troubleshoot-firefox-issues-using-safe-mode
Flags: needinfo?(erahm) → needinfo?(mozbug)
Summary: memory usage climbs indefinitely with frequent page loads → 2GB structured-clone-holder memory usage with 9 extensions enabled
Whiteboard: [MemShrink]
Hm. Well, the numbers after CC look pretty reasonable, so I don't think there's a leak. I'm surprised that it ever gets anywhere near as high as 2GB across all content processes, but my guess is that some extension is just doing *tons* of messaging.

It would be nice to find out which one.

I'm not sure there's much we can do about it, other than forcing the CC to run more often when the numbers get that high, though.
Fortunately, I'm not using many extensions, so it was not difficult to find.

Seems the culprit is this extension: https://addons.mozilla.org/en-GB/firefox/addon/uautopagerize/

Annoyingly, there's no download link on the page and I can't figure out where the code is installed locally, so I'm unable to have a peek at the source code and see what it's doing.
Flags: needinfo?(mozbug)
(In reply to Sam Bull from comment #8)
> Fortunately, I'm not using many extensions, so it was not difficult to find.
> 
> Seems the culprit is this extension:
> https://addons.mozilla.org/en-GB/firefox/addon/uautopagerize/

Um. Yeah, so basically this add-on is crazy.

It stores about a MB of JSON data in its local storage, and loads that data for every page it's loaded into. That's what creates the StructuredCloneHolders, since we use them to transfer storage data to child processes.

It then iterates over a top-level list and checks the URL against each of 60-some regular expressions, and if it matches one, tries matching the URL against dozens of other URL patterns beneath it... All of which is crazy, though probably not related to the memory issues. It's just awful for performance.

I'm not sure how big that MB of JSON data winds up being when it's structured clone serialized, or how we wind up with enough of them to add up to 2GB, though.

In any case, those StructuredCloneHolder objects should go away with the indexedDB migration in bug 1406181. However, the content script will still keep an entire deserialized copy alive for every page it's loaded into, regardless of whether it's active on that page.
Component: Untriaged → Storage
Product: Firefox → WebExtensions
P1 for now, we're seeing quite a few dup's.
Whiteboard: [MemShrink] → [MemShrink:P1]
Can somebody please check profile recording attachment to #bug 1488051 (it is marked as duplicate).
Attached file large memory usage
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0
Ubuntu 18.04.1 LTS
Attached file after cleanup
Assignee: nobody → kmaglione+bmo
Priority: -- → P1

I'm going to try to work around this by neutering single-use StructuredCloneHolder objects, and dropping their storage, when we deserialize them. The objects themselves will still stick around until the GC cleans them up, but they should use much less memory.

Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
https://hg.mozilla.org/integration/mozilla-inbound/rev/005a8fce14e71875a1e784d7adf3ec73fe5e694b
Bug 1476032: Neuter StructuredCloneHolder objects after deserializing. r=aswan
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
See Also: → 1532886

I tested this on Ubuntu 16.04 x64 with Latest FF Nightly and I was not able to reproduce the issue. I will attach the memory usage for my test and https://addons.mozilla.org/en-GB/firefox/addon/uautopagerize/ addon installed.

Please let me know if the fix solved the problem for you.

Flags: needinfo?(mozbug)

This fix possibly causes a regression: bug 1552112

Flags: needinfo?(mozbug)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: