Very high memory usage after watching many HTML5 movies on Firefox, even after closing them all

NEW
Unassigned

Status

()

P2
major
a year ago
11 days ago

People

(Reporter: Virtual, Unassigned)

Tracking

(5 keywords)

56 Branch
x86_64
Windows 7
memory-footprint, memory-leak, nightly-community, regression, regressionwindow-wanted
Points:
---

Firefox Tracking Flags

(firefox-esr52 unaffected, firefox-esr60 wontfix, firefox54 unaffected, firefox55 unaffected, firefox56 wontfix, firefox57- wontfix, firefox58 wontfix, firefox59 wontfix, firefox60 wontfix, firefox61 wontfix, firefox62 fix-optional, firefox63 affected, firefox64 affected)

Details

(Whiteboard: [MemShrink:P2])

User Story

Regression range

Good:
https://ftp.mozilla.org/pub/firefox/nightly/2017/07/2017-07-21-03-02-04-mozilla-central/

Bad:
https://ftp.mozilla.org/pub/firefox/nightly/2017/07/2017-07-22-07-26-31-mozilla-central/

Regression range pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0faada5c2f308f101ab7c54a87b3dce80b97d0e3&tochange=7ce557b85b611536b69539a7c18d4834ffc92eea

Could be caused by one of these bugs:
Bug 1366694 - Enable Windows level 3 content process sandbox by default on Nightly.
Bug 1377280 - Enable keyboard APZ on nightly
Bug 1351148 - Add an event queue to nsThread for input events and annotate input IPC messages to use it

==============================================================================================

STR

1. Create clean new fresh Firefox profile without any addons (extensions, plugins, themes, etc.)
2. Install uBlock Orgin
3. In uBlock Orgin Dashboard, in 3rd-party filters tab, enable all filters in category:
- uBlock
- Ads filters (exept EayList without element hiding rules)
- Privacy
- Malware domains
- Social
- Multipuropse
- in Regions, languages category enable only:
-- EU: Prebake - Filter Obtrusive Cookie Notices
-- POL: polskie filtry do Adblocka i uBlocka
-- POL: polskie filtry do uBlocka uzupelnienie
4. Apply changes, Purge all caches and Update now
5. Restart Firefox
6. Open about:memory and create memory report
7. Open YouTube Trending ( https://www.youtube.com/feed/trending )
8. Open about 20-30 tabs in background (could be more depending on your avaiable memory, just do notexeed 90% of your memory, to prevent Firefox duping memory cache to disc cache, which slow down everything later)
9. Open about:memory and close all other tabs
10. Create memory report
11. Wait about 1min
12. Create memory report again

23% of all 4GB RAM used with Firefox on start - first memory report
~90% of all 4GB RAM used with Firefox on browsing session
~80% of all 4GB RAM used with Firefox on end browsing session - second & third memory report
44% of all 4GB RAM used with Firefox on end browsing session with waiting 1 min - fourth memory report

If I will browse for more then 30min, this 44% of all memory used with Firefox on end browsing session will be more and more, this doesn't happen with older Firefox versions, where all useless memory is dumped into oblivion, not bloating browser and whole memory.

Attachments

(5 attachments, 10 obsolete attachments)

67.26 KB, application/gzip
Details
160.41 KB, application/gzip
Details
96.39 KB, application/gzip
Details
82.59 KB, application/gzip
Details
91.22 KB, application/gzip
Details
STR:
1. Watch more than 10-20 HTML5 movies on some streaming website pages, it could be one by one or open 2-4 more in background to start buffering them while watching one
2. After "cinema time" ends, close all tabs while having only blank one opened
and see that memory usage is still extremely high and not going low while time passes
Created attachment 8892362 [details]
anonymized-memory-report (GC+CC+Minimaze memory usage).json.gz
Created attachment 8892363 [details]
anonymized-memory-report (GC+CC+Minimaze memory usage).json.gz
Created attachment 8892364 [details]
anonymized-memory-report (GC+CC).json.gz
Attachment #8892363 - Attachment is obsolete: true
Even doing:
- GC
- GC + CC
- GC + CC + Minimaze memory usage
don't change anything (or that much like it should).
Attachment #8892364 - Attachment description: anonymized-memory-report (CG+CC).json.gz → anonymized-memory-report (GC+CC).json.gz
Attachment #8892364 - Attachment filename: anonymized-memory-report (CG+CC).json.gz → anonymized-memory-report (GC+CC).json.gz
With clean new fresh profile without any addons (extensions, plugins, themes, etc.) it's not reproducible.
After playing with extensions, looks like Firefox with uBlock Orgin extension have various memory leaks.

Better STR:
1. Open some 10 movies on YouTube
2. Wait patiently for every tab to stop loading
3. Close all tabs
4. Repeat STR #1-3 steps about 5-10 times, depend on available memory
5. Close all tabs, while leaving only blank one
6. Wait some time and see that memory aren't purged

FYI - I'm using Mozilla Firefox Nightly 56.0a1 (2017-07-30) with Block Origin 1.13.9b3 (webext) [I had same issue on stable Block Origin 1.13.8]



@ Raymond Hill [:gorhill] - Any ideas what could be the cause here?
status-firefox54: unaffected → ?
status-firefox-esr52: unaffected → ?
Flags: needinfo?(rhill)
Summary: Very high memory usage after watching many HTML5 movies, even after closing them all → Very high memory usage after watching many HTML5 movies on Firefox with uBlock Orgin, even after closing them all

Comment 8

a year ago
Could you provide me URLs to the 10 videos in step 1?

I just want to go through exactly what you went through to reproduce the issue -- leaving out any guess work as much as possible.
Flags: needinfo?(rhill)

Comment 9

a year ago
Looking at your "anonymized-memory-report (GC+CC+Minimaze memory usage).json.gz", I am puzzled as to why you believe uBlock Origin specifically is causing an issue.

You also have other add-ons installed, which you do not mention in your comments. For example, DownloadThemAll.

Also, I can't make sense of "still extremely high and not going low while time passes". Here is what I see in your memory snapshot data:

Main process: 271.03 MB, or 208 MB when I subtract heap-unclassified and heap-overhead.
Content process: 133.93 MB, or 39.43 MB when I subtract heap-unclassified and heap-overhead.

There is another process in there which suggest that you were maybe using out of process extensions? (a key information in any such bug report):

Out of process: 75.67 MB, or 55.75 MB when I subtract heap-unclassified and heap-overhead.

Summary, not only do I not see "extremely high and not going low" memory usage, there are other add-ons you are using and which you left out. So far, I do not see anything like "extremely high" memory usage (hundreds of MB would qualify), also considering you were using OOP-extensions feature.

I will go through my own measurement once you provide me with the exact 10 URLs used in step 1, but so far I fail to see anything like "extremely high" memory usage from your about:memory report.

Comment 10

a year ago
TL;DR: I try to reproduce the steps and see if there was anything wrong with memory usage. Result: there is nothing wrong.

My repro steps, with current Nightly:

1. Launch Nightly with a newly created profile (keep all default settings)
2. Install uBO 1.13.9b3 from https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/versions/beta
3. Open uBO dashboard, go to "3rd-party filters" pane, then for an update of filter lists
4. Once the filter lists have been all updated, quit Nightly
5. Restart Nightly with the profile created in 1
6. Navigate to "about:memory" from the only existing tab
7. Click "GC", "CC", and "Minimize memory usage" with enough time between each click for the operation to complete.
8. Repeat 7. two more times.
9. Click "Measure and save...", save to "memory-report-before.json.gz"
10. Quit Nightly.
11. Launch Nightly with the profile created in 1
12. Navigate to "https://youtube.com/"
13. Right-click then "Open Link in New Tab" for the 10 video displayed at the top of the page
14. Bring forth all of the newly create 10 tabs such that video playback start for each of these tabs
15. Close all the 10 newly opened video tabs
16. Open a new tab
17. Close the "https://youtube.com/" tab created in 12
18. Repeat 12 with the newly created tab in 16
19. Repeat 13 to 18 4 more times
20. Now only a "New Tab" tab should be left opened
21. Repeat steps 6 to 8
22. Click "Measure and save...", save to "memory-report-after.json.gz"
22. Click "Load and diff..." using both "before" and "after" memory report created in 9 and 22.

Result on my side:

Main Process (pid NNN)
Explicit Allocations
36.30 MB (100.0%) -- explicit
├──17.89 MB (49.28%) ++ heap-overhead
├───8.54 MB (23.53%) ── heap-unclassified
├───6.18 MB (17.03%) ++ js-non-window
├──-5.34 MB (-14.72%) ++ startup-cache
├───4.73 MB (13.03%) ++ images
├──-0.24 MB (-0.66%) ++ workers/workers(chrome)
├───1.55 MB (04.26%) ++ script-preloader
├───0.95 MB (02.61%) ++ add-ons
├───0.75 MB (02.07%) ++ gfx
├───0.62 MB (01.72%) ++ (9 tiny)
├───0.28 MB (00.78%) ++ window-objects
└───0.38 MB (01.05%) ++ network
Other Measurements
  19.02 MB (100.0%) ++ decommitted
  38 (100.0%) ++ event-counts
  32.27 MB (100.0%) ++ heap-committed
  4.73 MB (100.0%) ++ images
  7.02 MB (100.0%) ++ js-main-runtime
  48 (100.0%) ++ js-main-runtime-compartments
  5.02 MB (100.0%) ++ js-main-runtime-gc-heap-committed
  28 (100.0%) ++ message-manager
  113 (100.0%) ++ observer-service
  153 (100.0%) ++ observer-service-suspect
  14 (100.0%) ++ preference-service
  0.39 MB (100.0%) ++ window-objects
   14.38 MB ── heap-allocated
   58.00 MB ── heap-mapped
    0.18 MB ── imagelib-surface-cache-estimated-locked
    0.18 MB ── imagelib-surface-cache-estimated-total
  1,260,632 ── page-faults-soft
   48.49 MB ── resident
   33.91 MB ── resident-peak
   -2.19 MB ── resident-unique
    0.00 MB ── shmem-allocated
    0.02 MB ── shmem-mapped
  160.76 MB ── vsize
End of Main Process (pid NNN)
  

Web Content (pid NNN)
Explicit Allocations
34.56 MB (100.0%) -- explicit
├───9.94 MB (28.75%) ++ js-non-window
├───8.44 MB (24.43%) ── heap-unclassified
├───5.49 MB (15.87%) ++ window-objects/top(about:newtab, id=NNN)
├───4.05 MB (11.73%) ++ heap-overhead
├───1.63 MB (04.70%) ++ script-preloader/heap
├───0.96 MB (02.78%) ++ images
├───0.88 MB (02.55%) ++ (10 tiny)
├───0.85 MB (02.46%) ── xpti-working-set [+]
├───0.65 MB (01.88%) ── profiler/profiler-state
├───0.48 MB (01.38%) ++ layout
├───0.46 MB (01.32%) ++ dom
├───0.38 MB (01.10%) ── preferences [+]
└───0.37 MB (01.06%) ++ gfx
Other Measurements
  1.60 MB (100.0%) ++ decommitted
  114 (100.0%) ++ event-counts
  1 (100.0%) ++ file-blob-urls
  27.10 MB (100.0%) ++ heap-committed
  0.96 MB (100.0%) ++ images
  14.41 MB (100.0%) ++ js-main-runtime
  55 (100.0%) ++ js-main-runtime-compartments
  6.40 MB (100.0%) ++ js-main-runtime-gc-heap-committed
  17 (100.0%) ++ message-manager
  291 (100.0%) ++ observer-service
  171 (100.0%) ++ preference-service
  1.01 MB (100.0%) ++ window-objects
      0.00 MB ── gfx-surface-image [+]
     23.04 MB ── heap-allocated [+]
      1.00 MB ── heap-chunksize [+]
     40.00 MB ── heap-mapped [+]
            1 ── host-object-urls [+]
      0.00 MB ── imagelib-surface-cache-estimated-locked [+]
      0.00 MB ── imagelib-surface-cache-estimated-total [+]
      1.81 MB ── js-main-runtime-temporary-peak [+]
       33,718 ── page-faults-soft [+]
    114.48 MB ── resident [+]
    285.66 MB ── resident-peak [+]
     48.44 MB ── resident-unique [+]
  1,780.83 MB ── vsize [+]
End of Web Content (pid NNN)


Observations:

The diff shows that one WebContent process is lingering into memory, which was not present in the "before" measurement. My understanding is that this is expected, Nightly keeps WebContent processes around for a while for reuse, in the spirit of efficiency. Firefox devs are better place than me to answer this.

The Main Process shows an extra ~36 MB for "Explicit Allocations". Out of the ~36 MB, ~26 MB are "heap-overhead" and "heap-unclassified", leaving ~10 MB as the diff between "after" and "before".

At this point, I can say the statement "memory usage is still extremely high and not going low while time passes" is not reproducible on my side. The extra 10 MB in "Main Process" can't be solely attributed to uBlock Origin either, see:

36.30 MB (100.0%) -- explicit
├──17.89 MB (49.28%) ++ heap-overhead
├───8.54 MB (23.53%) ── heap-unclassified
├───6.18 MB (17.03%) -- js-non-window
│   ├──5.74 MB (15.80%) -- zones/zone(0xNNN)
│   │  ├──2.39 MB (06.58%) ++ (229 tiny)
│   │  ├──0.43 MB (01.19%) ++ strings
│   │  ├──0.74 MB (02.04%) ++ compartment(moz-nullprincipal:{NNNNNNNN-NNNN-NNNN-NNNN-NNNNNNNNNNNN}, XPConnect Compilation Compartment)
│   │  ├──0.64 MB (01.77%) ++ compartment([System Principal], resource://gre/modules/MessageChannel.jsm)
│   │  ├──0.58 MB (01.59%) ++ shapes
│   │  ├──0.51 MB (01.42%) ++ compartment([System Principal], resource://gre/modules/TelemetrySend.jsm)
│   │  └──0.44 MB (01.23%) ++ compartment([System Principal], jar:file:///.../omni.ja!/components/nsBlocklistService.js)
│   ├──0.07 MB (00.19%) ++ runtime
│   └──0.38 MB (01.03%) ── gc-heap/chunk-admin
├──-5.34 MB (-14.72%) ++ startup-cache
├───4.73 MB (13.03%) -- images
│   ├──4.68 MB (12.90%) -- chrome
│   │  ├──4.63 MB (12.74%) -- vector/used
│   │  │  ├──1.14 MB (03.13%) ── image(1078x54, chrome://pocket-shared/skin/library-pocket-animation.svg)/source
│   │  │  ├──0.81 MB (02.22%) ── image(1078x54, chrome://browser/skin/library-bookmark-animation.svg)/source
│   │  │  ├──0.80 MB (02.21%) ++ image(468x20, chrome://browser/skin/reload-to-stop.svg)
│   │  │  ├──0.80 MB (02.21%) ++ image(468x20, chrome://browser/skin/stop-to-reload.svg)
│   │  │  ├──0.63 MB (01.73%) ── image(660x33, chrome://browser/skin/bookmark-animation.svg)/source
│   │  │  └──0.45 MB (01.24%) ++ (14 tiny)
│   │  └──0.06 MB (00.16%) ++ raster/used
│   └──0.05 MB (00.13%) ++ content
├──-0.24 MB (-0.66%) ++ workers/workers(chrome)
├───1.55 MB (04.26%) ++ script-preloader
├───0.95 MB (02.61%) ++ add-ons
├───0.75 MB (02.07%) ++ gfx
├───0.62 MB (01.72%) ++ (9 tiny)
├───0.28 MB (00.78%) ++ window-objects
└───0.38 MB (01.05%) ++ network

I will attach the memory reports after evaluating whether they contains privacy sensitive stuff (I expect not given this is a new profile).

Comment 11

a year ago
Created attachment 8892460 [details]
Comment 10 "before" memory report

Comment 12

a year ago
Created attachment 8892461 [details]
Comment 10 "after" memory report

Comment 13

a year ago
Forgot one step after step 20 above: leave the browser idle for 5 minutes.
Closing as per the following comment.

(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #7)
> With clean new fresh profile without any addons (extensions, plugins,
> themes, etc.) it's not reproducible.
> After playing with extensions, looks like Firefox with uBlock Orgin
> extension have various memory leaks.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → INVALID
@ Raymond Hill [:gorhill] - I will post in few days detailed STR,
but it's reproducible with latest Nightly + latest uBlock Orgin on this page https://www.youtube.com/feed/trending while doing STR from comment #7
and like you already caught my memory reports were done on my profile with all extensions enabled, I will redo test with new profile with only uBlock Orgin, to get memory report cleaner and easier for reading.
Thanks for looking into it.

@  Kohei Yoshino [:kohei] - It's not invalid, as Firefox should forbid memory leaks, even caused by extensions, especially WebExtensions, but for now I don't know if it's extension issue or Firefox. I will diagnose it deeper in few days.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
I thought Bugzilla usually doesn't handle a bug in a specific extension, but there's at least a better component for such cases...
Component: Untriaged → Add-ons
Product: Firefox → Tech Evangelism
Version: 56 Branch → unspecified
I want to add that it's not reproducible in Firefox stable,
so in the end it could be Firefox issue, not extension one.
status-firefox54: ? → unaffected
status-firefox-esr52: ? → unaffected

Comment 18

a year ago
This problem began with the assembly of August 22, on the assembly of August 21, there are no problems with memory collection.

Comment 19

a year ago
> I will post in few days detailed STR,

Aside not being able to test on Windows (can only test on Linux), what in my repro steps in comment 10 is preventing me from reproducing the issue? How will your repro steps differ from what I went through in comment 10? With that information I would go ahead and try to reproduce again.
Andy, something the add-ons team is concerned about?
status-firefox56: affected → fix-optional
Flags: needinfo?(amckay)
(In reply to Wolf from comment #18)
> This problem began with the assembly of August 22, on the assembly of August
> 21, there are no problems with memory collection.

So that means it started one year ago?
or you simply misspelled month where it started?



(In reply to R. Hill from comment #19)
> > I will post in few days detailed STR,
> 
> Aside not being able to test on Windows (can only test on Linux), what in my
> repro steps in comment 10 is preventing me from reproducing the issue? How
> will your repro steps differ from what I went through in comment 10? With
> that information I would go ahead and try to reproduce again.

Not that much, except I'm on Windows system. I will try to post detailed STR, but I'm kinda busy lately, so it won't be very fast. If someone also could reproduce issue, please post it here.
Flags: needinfo?(hugajojoca)

Comment 22

a year ago
Not until we've got more details and I don't think we've got enough to work with. Krupa, maybe worth testing?
Flags: needinfo?(amckay)

Comment 23

a year ago
(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #21)
> (In reply to Wolf from comment #18)
> > This problem began with the assembly of August 22, on the assembly of August
> > 21, there are no problems with memory collection.
> 
> So that means it started one year ago?
> or you simply misspelled month where it started?

Oops, I'm sorry, I made a mistake for a month ... the problem with liberation memory began with the assembly on July 22, 2017.
(In reply to Wolf from comment #23)
> (In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see
> your comment/reply/question/etc.) from comment #21)
> > (In reply to Wolf from comment #18)
> > > This problem began with the assembly of August 22, on the assembly of August
> > > 21, there are no problems with memory collection.
> > 
> > So that means it started one year ago?
> > or you simply misspelled month where it started?
> 
> Oops, I'm sorry, I made a mistake for a month ... the problem with
> liberation memory began with the assembly on July 22, 2017.

Thank you very much!
Could you maybe pinpoint which patch caused this issue with mozregression ( https://github.com/mozilla/mozregression/releases )? It's very easy to use, see "Quick start" documentation ( http://mozilla.github.io/mozregression/quickstart.html ), only GUI part will do. It will be extremely helpful.

Comment 25

a year ago
Tested this issue on Windows 7 , following steps from comment #10 and I can't see any unusual memory usage, but I will attach my memory gathered data, maybe someone else can understand better the memory logs

Comment 26

a year ago
Created attachment 8896272 [details]
Before-After generated comaprison on Windows7, latest Nightly

Comment 27

a year ago
Created attachment 8896274 [details]
"before" memory report Windows7, latest Nightly

Comment 28

a year ago
Created attachment 8896275 [details]
"After" memory report Windows7, latest Nightly

Comment 29

a year ago
(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #24)

> 
> Thank you very much!
> Could you maybe pinpoint which patch caused this issue with mozregression (
> https://github.com/mozilla/mozregression/releases )? It's very easy to use,
> see "Quick start" documentation (
> http://mozilla.github.io/mozregression/quickstart.html ), only GUI part will
> do. It will be extremely helpful.
I can not, there is no time for this. But I repeat that on the build of 07/21/2017, there were no problems with cleaning the memory after closing the YouTube tab or some other page with html5 players, but the problem with unloading the memory began with the build on 07/22/2017 and this problem Is still relevant.

(Translation: google translate)
Flags: needinfo?(hugajojoca)
(In reply to Wolf from comment #29)
> (In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see
> your comment/reply/question/etc.) from comment #24)
> 
> > 
> > Thank you very much!
> > Could you maybe pinpoint which patch caused this issue with mozregression (
> > https://github.com/mozilla/mozregression/releases )? It's very easy to use,
> > see "Quick start" documentation (
> > http://mozilla.github.io/mozregression/quickstart.html ), only GUI part will
> > do. It will be extremely helpful.
> I can not, there is no time for this. But I repeat that on the build of
> 07/21/2017, there were no problems with cleaning the memory after closing
> the YouTube tab or some other page with html5 players, but the problem with
> unloading the memory began with the build on 07/22/2017 and this problem Is
> still relevant.
> 
> (Translation: google translate)

OK. Thank you very much! I will try to do the rest.
It's also extremely useful information.

So for future regression range search

Good:
https://ftp.mozilla.org/pub/firefox/nightly/2017/07/2017-07-21-03-02-04-mozilla-central/

Bad:
https://ftp.mozilla.org/pub/firefox/nightly/2017/07/2017-07-22-07-26-31-mozilla-central/

Regression range pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0faada5c2f308f101ab7c54a87b3dce80b97d0e3&tochange=7ce557b85b611536b69539a7c18d4834ffc92eea

Maybe caused by one of these bugs:
Bug 1366694 - Enable Windows level 3 content process sandbox by default on Nightly.
Bug 1377280 - Enable keyboard APZ on nightly
Bug 1351148 - Add an event queue to nsThread for input events and annotate input IPC messages to use it

So I'm changing produce to "Firefox", as it's Firefox issue per regression, not extension one.
Status: REOPENED → NEW
status-firefox55: ? → unaffected
status-firefox56: fix-optional → affected
status-firefox57: --- → affected
Component: Add-ons → Untriaged
Flags: needinfo?(Virtual)
Product: Tech Evangelism → Firefox
Version: unspecified → 56 Branch
Do we know if this is uBlock causing this and if so, what rev? question: 

1) Can we repo without uBlock installed?
2) Is the rev of uBlock the newer webextension based rev?
Switching this to Firefox :: General until further updates.
Component: Untriaged → General
(In reply to Jim Mathies [:jimm] from comment #31)
> Do we know if this is uBlock causing this and if so, what rev? question: 
> 
> 1) Can we repo without uBlock installed?
> 2) Is the rev of uBlock the newer webextension based rev?

It's not reproducible without uBlock Orgin extension,
but it's also not reproducible with uBlock Orgin extension on older version of the Firefox, see regression range in Comment #30.
I tested this on uBlock Orgin 1.13.8 (stable, legacy) and uBlock Orgin 1.13.9b3 (beta, Webext-hybrid).

So looks like something in Firefox changed, that Firefox with some extensions doesn't anymore release used memory.
Flags: needinfo?(Virtual)
Whiteboard: [MemShrink]
Anthony, is this something your team can help to test? It may be an a/v problem and not a specific problem with uBlock. Thanks.
Flags: needinfo?(ajones)
Given the regression range, I'd try setting security.sandbox.content.level=2 to see if that makes a difference.
Looking on it one more time, this bug could be similar to bug #1371255 or even duplicate.
Based on regression range in Comment #30, and looking which patches touched WebExtensions code, it could be that this regression is caused by patch from bug #1381687.

@ Kris Maglione [:kmag] - Any opinions about this or it's missed call?
Flags: needinfo?(Virtual) → needinfo?(kmaglione+bmo)
See Also: → bug 1371255
Do you have about:memory dumps from an affected revision? I don't see anything wrong in any of the attached reports.
I also don't see any issues when I run the steps from comment 7 in a recent nightly. The only difference after those steps is a bit of additional bin-unused in a web content process, and some extra heap-textures in the main process.
Flags: needinfo?(kmaglione+bmo)
(In reply to Kris Maglione [:kmag] (long backlog; ping on IRC if you're blocked) from comment #38)
> I also don't see any issues when I run the steps from comment 7 in a recent
> nightly. The only difference after those steps is a bit of additional
> bin-unused in a web content process, and some extra heap-textures in the
> main process.

Okay sounds like this is fixed. Please reopen if you still see the issue.
Status: NEW → RESOLVED
Last Resolved: a year agoa year ago
Resolution: --- → WORKSFORME
Reopening, as I'm still able to reproduce the issue.
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Can you provide a relevant memory report, then? I can't do anything more without one.
They're located in comment 2, comment 3, comment 4 & comment 5,
I will redo STR 'soon' on clean profile with only uBlock Orgin.
Flags: needinfo?(Virtual)
(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #42)
> They're located in comment 2, comment 3, comment 4 & comment 5,
> I will redo STR 'soon' on clean profile with only uBlock Orgin.

Hm. Well, the profile in comment 5 has about 100MB of heap overhead, which is a bit worrying. But it's also got at least one SDK add-on, the CPOW prefetcher, and a bunch of devtools stuff loaded into it.
1. Create clean new fresh Firefox profile without any addons (extensions, plugins, themes, etc.)
2. Install uBlock Orgin
3. In uBlock Orgin Dashboard, in 3rd-party filters tab, enable all filters in category:
- uBlock
- Ads filters (exept EayList without element hiding rules)
- Privacy
- Malware domains
- Social
- Multipuropse
- in Regions, languages category enable only:
-- EU: Prebake - Filter Obtrusive Cookie Notices
-- POL: polskie filtry do Adblocka i uBlocka
-- POL: polskie filtry do uBlocka uzupelnienie
4. Apply changes, Purge all caches and Update now
5. Restart Firefox
6. Open about:memory and create memory report
7. Open YouTube Trending ( https://www.youtube.com/feed/trending )
8. Open about 20-30 tabs in background (could be more depending on your avaiable memory, just do notexeed 90% of your memory, to prevent Firefox duping memory cache to disc cache, which slow down everything later)
9. Open about:memory and close all other tabs
10. Create memory report
11. Wait about 1min
12. Create memory report again

23% of all 4GB RAM used with Firefox on start - first memory report
~90% of all 4GB RAM used with Firefox on browsing session
~80% of all 4GB RAM used with Firefox on end browsing session - second & third memory report
44% of all 4GB RAM used with Firefox on end browsing session with waiting 1 min - fourth memory report

If I will browse for more then 30min, this 44% of all memory used with Firefox on end browsing session will be more and more, this doesn't happen with older Firefox versions, where all useless memory is dumped into oblivion, not bloating browser and whole memory.
Flags: needinfo?(Virtual)
Created attachment 8905470 [details]
4 - memory-report.json.gz
Attachment #8892360 - Attachment is obsolete: true
Attachment #8892361 - Attachment is obsolete: true
Attachment #8892362 - Attachment is obsolete: true
Attachment #8892364 - Attachment is obsolete: true
Attachment #8892460 - Attachment is obsolete: true
Attachment #8892461 - Attachment is obsolete: true
Attachment #8896272 - Attachment is obsolete: true
Attachment #8896274 - Attachment is obsolete: true
Attachment #8896275 - Attachment is obsolete: true
> 23% of all 4GB RAM used with Firefox on start - first memory report
> ~90% of all 4GB RAM used with Firefox on browsing session
> ~80% of all 4GB RAM used with Firefox on end browsing session - second &
> third memory report
> 44% of all 4GB RAM used with Firefox on end browsing session with waiting 1
> min - fourth memory report

How are you measuring this? I don't see anything like 2GB of RAM being used in the last memory report.

In any case, I still don't see anything especially concerning here. The main and web content processes do wind up with a lot of heap overhead in the end, but there's not much I can do about that.

I would be interested to see how the reports differ when no extensions are enabled, though.
Eric, can you take a look at the memory reports in comments 44-48 and let me know if anything about them worries you? I don't particularly like the heap-overhead numbers, but I don't know if they're anything to worry about. And I'm not used to looking at some of the non-explicit numbers like committed, decommitted, address-space/* in detail.
Flags: needinfo?(erahm)
(In reply to Kris Maglione [:kmag] (long backlog; ping on IRC if you're blocked) from comment #49)
> > 23% of all 4GB RAM used with Firefox on start - first memory report
> > ~90% of all 4GB RAM used with Firefox on browsing session
> > ~80% of all 4GB RAM used with Firefox on end browsing session - second &
> > third memory report
> > 44% of all 4GB RAM used with Firefox on end browsing session with waiting 1
> > min - fourth memory report
> 
> How are you measuring this? I don't see anything like 2GB of RAM being used
> in the last memory report.

It's just all physical memory used that time by system and software, not just only Firefox, as I'm too lazy to count all these Firefox processes as one. ;)



(In reply to Kris Maglione [:kmag] (long backlog; ping on IRC if you're blocked) from comment #49)
> I would be interested to see how the reports differ when no extensions are
> enabled, though.

I will try to do that on next week, same as attaching this with older Firefox.
Component: General → Extension Compatibility
QA Contact: Virtual
(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #51)
> I will try to do that on next week, same as attaching this with older
> Firefox.

Did you by any chance get around to it yet?
status-firefox56: affected → wontfix
Flags: needinfo?(Virtual)
Can you test with the latest web extension version of uBlock?
Flags: needinfo?(erahm)
(In reply to Panos Astithas [:past] (56 Regression Engineering Owner) (please ni?) from comment #52)
> (In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see
> your comment/reply/question/etc.) from comment #51)
> > I will try to do that on next week, same as attaching this with older
> > Firefox.
> 
> Did you by any chance get around to it yet?

Unfortunately, no, I don't have that much time right now to do that, as reliable steps to reproduce are too much time consuming.



(In reply to Eric Rahm [:erahm] (please no mozreview requests) from comment #53)
> Can you test with the latest web extension version of uBlock?

It's still reproducible with uBlock Origin 1.14.11rc7 (WebExtension) on Mozilla Firefox Nightly 58.0a1 (2017-09-28) (64-bit), same as it's with uBlock Origin 1.13.8 (legacy extension).
status-firefox58: --- → affected
Flags: needinfo?(Virtual)

Comment 55

a year ago
I can reproduce this With Firefox 57beta and uBlock Origin 1.14.10 , memory high usage happens with YouTube and twitch even after close all tabs while having only one opened.
I'll see if I can repro on Windows.
Flags: needinfo?(erahm)
Flags: needinfo?(ajones)
Priority: -- → P2

Comment 57

11 months ago
What is the reason the steps in comment 10 still have not been undertaken by whoever is able to reproduce the issue?

If whoever suffers the issue wants things to move forward, there has to be some efforts to narrow it down. As said, I have tried to reproduce, to no avail. This narrowing is unavoidable, that would at least lift a weight on my shoulders by at least having some better idea about whether uBO is causing this issue -- I currently have no reason to believe so, but if it is the case, nothing will move forward without actual efforts to narrow it. I have provided pretty detailed steps to help toward that goal, I don't know what else I can do on my side.

Comment 58

10 months ago
Someone posted a case of Youtube causing high-memory usage on reddit.[1] I think the case deserves to be looked into, the user posted a memory report.[2]

Notably, I see tens of "top(none)/detached" Youtube pages in the main proces (for some reasons multiprocess appears disabled), I would like to know whether these are normal or not. The report shows that no extension were installed (just the built-in ones).

[1] https://www.reddit.com/r/firefox/comments/7cn7w3/memory_leak_does_ff_really_need_7_gb_of_ram_when/dprd3dp/

[2] https://www.dropbox.com/s/6nw5xo08vspnvth/memory-report.json.gz?dl=0

Updated

10 months ago
Flags: needinfo?(erahm)
Whiteboard: [MemShrink] → [MemShrink:P2]
(In reply to Virtual_ManPL [:Virtual] - (please needinfo? me - so I will see your comment/reply/question/etc.) from comment #0)
> STR:
> 1. Watch more than 10-20 HTML5 movies on some streaming website pages, it
> could be one by one or open 2-4 more in background to start buffering them
> while watching one
> 2. After "cinema time" ends, close all tabs while having only blank one
> opened
> and see that memory usage is still extremely high and not going low while
> time passes
I've also seen this after watching several Youtube vids.  I don't use uBlock or AdBlocker.
Looking at about:memory  I find that all or most all of the vids watched were now shown as 'Ghost Windows' holding memory.
Summary: Very high memory usage after watching many HTML5 movies on Firefox with uBlock Orgin, even after closing them all → Very high memory usage after watching many HTML5 movies on Firefox, even after closing them all
[Tracking Requested - why for this release]: Regression
tracking-firefox57: --- → ?
tracking-firefox58: --- → ?
tracking-firefox59: --- → ?
Component: Extension Compatibility → General
Tracking requests on an old bug with no new/actionable information aren't all that useful.
status-firefox57: affected → wontfix
tracking-firefox57: ? → -
tracking-firefox58: ? → ---
tracking-firefox59: ? → ---
Component: General → Audio/Video: Playback
Product: Firefox → Core
status-firefox58: affected → wontfix
status-firefox59: affected → wontfix
status-firefox60: affected → fix-optional

Comment 62

6 months ago
I'm not sure if this is the right bug to post in because memory doesn't really rise that much for me (only 300MB system and 200MB GPU) but playing 7+ 22min episodes on Netflix makes the page unresponsive and 12+ makes the whole browser unresponsive.
Using AMD FX 8320, HD5750, 16GB RAM. uBlock Dev.

Two perf profiles after skipping videos, seeking and skipping intros for ~20mins (no-where near as laggy as watching constantly for 2hrs+):  https://perfht.ml/2DlPhrz

after restarting browser: https://perfht.ml/2HqYMIi

First profile is getting 8second BHR-detected hang / event processing delays. Performance degradation people are seeing might be because of this rather than a memory leak.
(In reply to Jules A from comment #62)
> Two perf profiles after skipping videos, seeking and skipping intros for
> ~20mins (no-where near as laggy as watching constantly for 2hrs+): 
> https://perfht.ml/2DlPhrz

It could very well be related. This profile shows two 4-5s hard janks from Netflix's Cadmium player code. ~1.2s of each of those is spend in the GC, and a lot of the rest is spent allocating and manipulating JS strings. If there's a memory leak, that could be part of the reason we spend so much time in GC.

But the Netflix code is a bit suspect, anyway...

The reduce function that winds up triggering most of the GCs is:

  d.reduce(function(d,n,T){return T<m?d+String.fromCharCode(n):d;},"")

Which looks like it's meant to turn a slice of a Uint8Array into a string.

It doesn't help that the first place it's used is:

    function thing(d, m) {
        m || (m = d.length);
        return d.reduce(function(d, n, T) {
            return T < m ? d + String.fromCharCode(n) : d;
        }, "");
    }

    for (var m = {}, n = 0; 256 > n; ++n) {
        var r = thing([n]);
        m[r] = n;
    }

Which is just a really slow way of writing:

  for (let i = 0; i < 256; i++)
    m[String.fromCharCode(i)] = i;

Somebody other than me should probably look into this to try to figure out what exactly is going on here, and how it can be fixed.

Comment 64

6 months ago
(In reply to Kris Maglione [:kmag] (long backlog; ping on IRC if you're blocked) from comment #63)
> It could very well be related. This profile shows two 4-5s hard janks from
> Netflix's Cadmium player code. ~1.2s of each of those is spend in the GC,
> and a lot of the rest is spent allocating and manipulating JS strings. If
> there's a memory leak, that could be part of the reason we spend so much
> time in GC.

I went and watched Netflix for a couple of hours, when I went to check about:memory and went back to the tab it was just left as a spinner. This is what it looks like: https://perfht.ml/2HrUuQW . After 10mins or so it finally re-rendered but the page is unresponsive. 

Memory doesn't seem to be the problem but I could be wrong. I attached a memory report in-case.

Comment 65

6 months ago
Created attachment 8958307 [details]
memory-report.json.gz (Jules.A)
status-firefox60: fix-optional → wontfix
status-firefox61: affected → wontfix
status-firefox62: affected → fix-optional
status-firefox-esr60: --- → wontfix
You need to log in before you can comment on or make changes to this bug.