Firefox doesn't deallocate unclassified heap until it reaches 1.8GB for a given CLJS program

UNCONFIRMED
Assigned to

Status

()

Core
Canvas: WebGL
P2
normal
UNCONFIRMED
a year ago
a year ago

People

(Reporter: Rovanion Luckey, Assigned: jgilbert)

Tracking

50 Branch
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [MemShrink:P1] gfx-noted)

Attachments

(3 attachments)

(Reporter)

Description

a year ago
Created attachment 8813213 [details]
Memory report after GC

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161114145007

Steps to reproduce:

Clone 

git clone https://github.com/WeatherMagic/weather-front/
cd weather-front
git checkout 4f6a4f3
lein figwheel 

Once figwheel has started it should open your default browser to http://localhost:3449/

Now you'll simply have to let the application run.


Actual results:

You'll see that Firefox allocates increasingly large amounts of memory. On my 32-bit Ubuntu 16.04 system Firefox allocates increasing amounts of memory until it reaches 1.8GB of heap-unclassified soon after which it's properly GC'd.


Expected results:

The memory should be GC'd much sooner. In Chromium it's collected about every three seconds.
(Reporter)

Comment 1

a year ago
Created attachment 8813214 [details]
Memory report just before GC
(Reporter)

Comment 2

a year ago
A working copy of the program can be found here: http://rovanion.se/weather-front/resources/public/
(Reporter)

Comment 3

a year ago
A perhaps related issue is https://bugzilla.mozilla.org/show_bug.cgi?id=1305389

A web server with this code running can be found here: http://rovanion.se/webgl-clojurescript-tutorial/resources/public/
So just to be clear, the STR is:

#1 - Load http://rovanion.se/webgl-clojurescript-tutorial/resources/public/
#2 - Watch memory and CPU usage spike

On a 53 debug/dmd build I can't even get a memory report to complete for the content process.
Whiteboard: [MemShrink]

Updated

a year ago
Component: Untriaged → Canvas: WebGL
Product: Firefox → Core
(Reporter)

Comment 5

a year ago
Sorry that i put info on the both bug reports in each other. For this bug the STR should be 

#1 - Load http://rovanion.se/weather-front/resources/public/
#2 - Watch memory and CPU usage spike
Created attachment 8814257 [details]
dmd output

Looks like a ton of stuff in shaders.
(Reporter)

Comment 7

a year ago
Yes, I've since making this bug noticed the cause of this: We were compiling the shader once per frame.

I've produced a copy of the program which uses a magnitude less CPU, for obvious reasons, but which still leaks memory but at a slower pace of about 20MB/s.

http://rovanion.se/weather-front-2/resources/public/
(Reporter)

Comment 8

a year ago
Worth noting is that I'm using the Intel drivers on an i7 2640M.

With this second version of the program though the memory leak appears to be outside Firefox' address space since neither the virtual, resident or shared memory of Firefox in htop goes up; but then neither does any other precesses memory. But never the less the RAM gets filled up and so does the swap.
(Reporter)

Comment 9

a year ago
But it's clear that it's the application that's causing the allocation of the memory since it all deallocates once tab is closed.

Same difference can be seen on mobile where firefox crashes running [2] after roughly ten seconds while the built in Android browser runs it without issues, indefinitely.

Issue [0] is a destilled example of the extreme shader compilation like [1] which on my machine causes firefox to crash. Since the code in [1] has both the problem of that in [0] and [2], do you think we should let this bug be solely about [2]? But then again that invalidates the memory and dmd outputs so perhaps I should just start a new one for [2]?

[0] https://bugzilla.mozilla.org/show_bug.cgi?id=1305389
[1] http://rovanion.se/weather-front/resources/public/
[2] http://rovanion.se/weather-front-2/resources/public/
Flags: needinfo?(jgilbert)
(Assignee)

Comment 10

a year ago
We're keeping the ANGLE shader validator around because it's easy, not because it's required. This causes shaders to have an outsized memory impact, though it's usually fine for normal workloads.

I'm interested by the outside-our-memory-space leak you're seeing.
Does this reproduce if you set the pref layers.acceleration.disabled=false and restart the browser?
Flags: needinfo?(jgilbert) → needinfo?(rovanion.luckey)
Whiteboard: [MemShrink] → [MemShrink:P1]
(Reporter)

Comment 12

a year ago
layers.acceleration.disabled=false was already the default value in my Firefox.
Jeff, any next steps here?
Flags: needinfo?(rovanion.luckey) → needinfo?(jgilbert)
(Assignee)

Comment 14

a year ago
I'll look into why this may be.
I suspect we aren't communicating how large our GC object is, so the GC doesn't realize how much memory pressure there is.

I do think this problem will go away if we aggressively discard/reset the ANGLE shader translator instance.
Assignee: nobody → jgilbert
Flags: needinfo?(jgilbert)
Priority: -- → P2
Whiteboard: [MemShrink:P1] → [MemShrink:P1] gfx-noted
You need to log in before you can comment on or make changes to this bug.