Last Comment Bug 678859 - Runaway memory usage with DOM Animation (Windows only?)
: Runaway memory usage with DOM Animation (Windows only?)
Status: RESOLVED FIXED
[MemShrink:P2][inbound]
:
Product: Core
Classification: Components
Component: Graphics (show other bugs)
: Trunk
: x86_64 Windows Vista
: -- normal (vote)
: mozilla10
Assigned To: Robert O'Callahan (:roc) (email my personal email if necessary)
:
: Milan Sreckovic [:milan]
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-08-14 13:46 PDT by Michael Haufe
Modified: 2011-11-02 15:31 PDT (History)
17 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
about:memory on Win7 (9.51 KB, text/plain)
2011-08-30 21:36 PDT, Hugh Nougher [:Hughman]
no flags Details
Jprof of running this on a fast Linux box under trunk (1.55 MB, text/html)
2011-09-06 13:38 PDT, Randell Jesup [:jesup]
no flags Details
About:memory (linux) (3.60 KB, text/plain)
2011-09-06 13:39 PDT, Randell Jesup [:jesup]
no flags Details
fix (2.33 KB, patch)
2011-10-25 17:55 PDT, Robert O'Callahan (:roc) (email my personal email if necessary)
tnikkel: review+
Details | Diff | Splinter Review
fix v2 (2.74 KB, patch)
2011-10-25 20:23 PDT, Robert O'Callahan (:roc) (email my personal email if necessary)
tnikkel: review+
Details | Diff | Splinter Review

Description Michael Haufe 2011-08-14 13:46:30 PDT
User Agent: Mozilla/5.0 (Windows NT 6.0; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

Visit the following URL: http://cubiq.org/dropbox/wolf-ani/


Actual results:

Notice that the browser's memory consumption exceeds 1-1.5GB in seconds.


Expected results:

Compare this footprint with Chrome and IE9
Comment 1 Matthias Versen [:Matti] 2011-08-15 09:32:30 PDT
I can confirm that the memory consumption is at ~850mb+ with FF5 and the animation is very very slow. My Seamonkey trunk crashes frequently on this testcase and it's not faster as FF5

layout is a pure guess from me
Comment 2 Boris Zbarsky [:bz] (still a bit busy) 2011-08-23 21:55:10 PDT
Using a current build, I see completely stable memory usage at 260MB (plus minus 2MB) on Mac.

Is the memory growth here Windows-only?  If so, graphics is a lot more likely than layout.

I can definitely see the very slow running bit, but that's a separate issue.
Comment 3 Boris Zbarsky [:bz] (still a bit busy) 2011-08-23 22:00:10 PDT
At least on Mac, 90+% of the time is spent painting....
Comment 4 Bas Schouten (:bas.schouten) 2011-08-23 22:06:46 PDT
I'll triage the GFX part.
Comment 5 :Ehsan Akhgari 2011-08-30 13:41:54 PDT
(In reply to Bas Schouten (:bas) from comment #4)
> I'll triage the GFX part.

Bas, did you manage to triage this?  We wanted to prioritize this bug in this week's MemShrink meeting, but we didn't have enough information to do so.
Comment 6 Bas Schouten (:bas.schouten) 2011-08-30 15:24:43 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #5)
> (In reply to Bas Schouten (:bas) from comment #4)
> > I'll triage the GFX part.
> 
> Bas, did you manage to triage this?  We wanted to prioritize this bug in
> this week's MemShrink meeting, but we didn't have enough information to do
> so.

I did not. I raised a bug to get a renewal on my manager profiler's license, which, when I tried to run it, turned out to be expired.
Comment 7 Bas Schouten (:bas.schouten) 2011-08-30 15:25:00 PDT
(In reply to Bas Schouten (:bas) from comment #6)
> (In reply to Ehsan Akhgari [:ehsan] from comment #5)
> > (In reply to Bas Schouten (:bas) from comment #4)
> > > I'll triage the GFX part.
> > 
> > Bas, did you manage to triage this?  We wanted to prioritize this bug in
> > this week's MemShrink meeting, but we didn't have enough information to do
> > so.
> 
> I did not. I raised a bug to get a renewal on my manager profiler's license,
> which, when I tried to run it, turned out to be expired.

Erm. Memory profiler, that is.
Comment 8 Hugh Nougher [:Hughman] 2011-08-30 21:36:36 PDT
Created attachment 557086 [details]
about:memory on Win7

I don't know if this will be any help, since it does not say anything about where the memory is used at all, but I managed to get an about:memory of this page in latest nightly.

I will also note that I was getting 5 to 30 seconds between frame updates and the memory would be back to a normal rate if the page is in a non-active tab (as in I needed a new window to view about:memory in). All the "Program Not Responding" messages got very annoying.

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a1) Gecko/20110827 Firefox/9.0a1
Comment 9 Nicholas Nethercote [:njn] 2011-09-06 13:24:54 PDT
Kevin and Randall will try to reproduce this while running their mem profiling tools.
Comment 10 Randell Jesup [:jesup] 2011-09-06 13:38:07 PDT
Created attachment 558586 [details]
Jprof of running this on a fast Linux box under trunk

Note: No leak obvious.  ~750MB VSS, ~100MB RSS in a fresh new profile.  Stable.

Uses 100% CPU almost all painting (see profile)
Comment 11 Randell Jesup [:jesup] 2011-09-06 13:39:17 PDT
Created attachment 558587 [details]
About:memory (linux)

Note: didn't do anything but hit "back" to go back to the bug.  VSS dropped.
Comment 12 Hugh Nougher [:Hughman] 2011-09-08 14:37:28 PDT
I just noticed another thing. Through Process Explorer and its GPU Graph tab it says that firefox uses about 1GB of shared GPU memory for this web page.

I think this accounts for most of the memory not accounted for in about:memory since I believe shared memory lives in RAM.
Comment 13 :Ehsan Akhgari 2011-09-27 13:23:46 PDT
Bas: did you get a chance to see if you can reproduce this bug on Windows?
Comment 14 Bas Schouten (:bas.schouten) 2011-09-27 13:31:15 PDT
(In reply to Ehsan Akhgari [:ehsan] from comment #13)
> Bas: did you get a chance to see if you can reproduce this bug on Windows?

I've raised the priority of the IT request a short while ago, I just pinged on it again.
Comment 15 Johnny Stenback (:jst, jst@mozilla.com) 2011-10-11 13:38:19 PDT
Any luck on the IT request? If not, let me know and maybe I can help nudge that along.
Comment 16 Bas Schouten (:bas.schouten) 2011-10-24 08:14:10 PDT
So far, it appears I can't find all these allocations in the native heap using the memory profiler. But I can see a very large amount of thebes layers doing allocations. A bunch of that possibly on the GPU.
Comment 17 Bas Schouten (:bas.schouten) 2011-10-24 08:30:24 PDT
So this page causes a huge amount of 5x5 ThebesLayers (2000+) to be created. These layers are all component alpha because of the text on there, causing 4000+ textures to be created to support these. Since 4000 * 25 * 4 should only be 400K of memory my best guess is that overhead and minimum allocation sizes in the GPU are responsible, but I'll need to investigate that a little further, 250K+ per texture seems a little harsh, although not impossible. In any case we should probably improve usage of the layer system here.
Comment 18 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-24 18:24:56 PDT
Should we simply have a lower-bound on size before we make layers active? Presumably very small layers are cheap to composite on the CPU so we can use lower-overhead structures.
Comment 19 Bas Schouten (:bas.schouten) 2011-10-25 10:30:14 PDT
(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #18)
> Should we simply have a lower-bound on size before we make layers active?
> Presumably very small layers are cheap to composite on the CPU so we can use
> lower-overhead structures.

Yes, that's what I was thinking, some limit like 50x50 or 25x25 seems sensible, at least for now as a simple solution.
Comment 20 Nicholas Nethercote [:njn] 2011-10-25 12:09:43 PDT
Getting data about this in about:memory would be good, but that sounds hard.  So implementing the workaround in comment 19 seems good enough for this bug.
Comment 21 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-25 12:29:51 PDT
(In reply to Bas Schouten (:bas) from comment #17)
> I'll need to investigate that a little
> further, 250K+ per texture seems a little harsh, although not impossible. In
> any case we should probably improve usage of the layer system here.

That seems worth investingating. 250K per 5x5 layer does seem like a big problem.
Comment 22 Bas Schouten (:bas.schouten) 2011-10-25 14:56:33 PDT
I did some stand alone textures.

Creating 2000 5x5 textures uses practically no CPU memory, and a low amount of GPU memory. I created up to 30 000 and was still all fine.

Creating 2000 Direct2D DXGI render targets for these 2000 textures consumed roughly 250M of private working set memory. So that's about 250K of Direct2D overhead per surface, high, but by itself not the cause of this issue.

I'll need to investigate further where our memory is going to get to 1.4 GB. My memory profiler does not appear to show that kind of consumption on any of the native memory heaps.
Comment 23 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-25 17:55:33 PDT
Created attachment 569571 [details] [diff] [review]
fix

This seems to fix the memory issues. The demo is still super-slow.
Comment 24 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-25 17:56:23 PDT
The slowness is quite likely to be some O(N^2) stuff in FrameLayerBuilder.
Comment 25 Timothy Nikkel (:tnikkel) 2011-10-25 17:59:52 PDT
Comment on attachment 569571 [details] [diff] [review]
fix

Mention that the size is in pixels.

That 'if' is getting complicated.
Comment 26 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-25 18:44:49 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/9bbb30ec51f5
Comment 27 Timothy Nikkel (:tnikkel) 2011-10-25 18:48:12 PDT
I meant the new constant should be mentioned to be in pixels. :) Either way is fine.
Comment 28 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-25 19:58:32 PDT
Backed out:
https://hg.mozilla.org/integration/mozilla-inbound/rev/98013fe19dcb

Tests started unexpectedly passing, which reveals the problem that we use <canvas>es to force active layers in reftests, and this patch can defeat that if the canvas is too small. In fact I think we shouldn't be making canvases inactive with this patch.
Comment 29 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-25 20:23:08 PDT
Created attachment 569597 [details] [diff] [review]
fix v2

Move size check into nsDisplayItem::GetLayerState so it can be customized per-display-item-type.
Comment 30 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-26 13:12:49 PDT
http://hg.mozilla.org/integration/mozilla-inbound/rev/70e60379729b
Comment 31 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-10-26 16:10:12 PDT
Followup test fix:
http://hg.mozilla.org/integration/mozilla-inbound/rev/39fa89ebe5e2
Comment 33 Ben Turner (not reading bugmail, use the needinfo flag!) 2011-11-02 08:21:21 PDT
(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #24)
> The slowness is quite likely to be some O(N^2) stuff in FrameLayerBuilder.

Just out of curiosity did something get filed on this?
Comment 34 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-11-02 15:31:34 PDT
No.

Note You need to log in before you can comment on or make changes to this bug.