678859 - Runaway memory usage with DOM Animation (Windows only?)

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Michael Haufe]

User Agent: Mozilla/5.0 (Windows NT 6.0; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

Visit the following URL: http://cubiq.org/dropbox/wolf-ani/


Actual results:

Notice that the browser's memory consumption exceeds 1-1.5GB in seconds.


Expected results:

Compare this footprint with Chrome and IE9

Comment 1

[Matthias Versen [:Matti]]

I can confirm that the memory consumption is at ~850mb+ with FF5 and the animation is very very slow. My Seamonkey trunk crashes frequently on this testcase and it's not faster as FF5

layout is a pure guess from me

Comment 2

[Boris Zbarsky [:bzbarsky]]

Using a current build, I see completely stable memory usage at 260MB (plus minus 2MB) on Mac.

Is the memory growth here Windows-only?  If so, graphics is a lot more likely than layout.

I can definitely see the very slow running bit, but that's a separate issue.

Comment 3

[Boris Zbarsky [:bzbarsky]]

At least on Mac, 90+% of the time is spent painting....

Comment 4

[Bas Schouten (:bas.schouten)]

I'll triage the GFX part.

Comment 5

[(no longer active)]

(In reply to Bas Schouten (:bas) from comment #4)
> I'll triage the GFX part.

Bas, did you manage to triage this?  We wanted to prioritize this bug in this week's MemShrink meeting, but we didn't have enough information to do so.

Comment 6

[Bas Schouten (:bas.schouten)]

(In reply to Ehsan Akhgari [:ehsan] from comment #5)
> (In reply to Bas Schouten (:bas) from comment #4)
> > I'll triage the GFX part.
> 
> Bas, did you manage to triage this?  We wanted to prioritize this bug in
> this week's MemShrink meeting, but we didn't have enough information to do
> so.

I did not. I raised a bug to get a renewal on my manager profiler's license, which, when I tried to run it, turned out to be expired.

Comment 7

[Bas Schouten (:bas.schouten)]

(In reply to Bas Schouten (:bas) from comment #6)
> (In reply to Ehsan Akhgari [:ehsan] from comment #5)
> > (In reply to Bas Schouten (:bas) from comment #4)
> > > I'll triage the GFX part.
> > 
> > Bas, did you manage to triage this?  We wanted to prioritize this bug in
> > this week's MemShrink meeting, but we didn't have enough information to do
> > so.
> 
> I did not. I raised a bug to get a renewal on my manager profiler's license,
> which, when I tried to run it, turned out to be expired.

Erm. Memory profiler, that is.

Comment 8

[Hugh Nougher [:Hughman]]

Attachment: about:memory on Win7

I don't know if this will be any help, since it does not say anything about where the memory is used at all, but I managed to get an about:memory of this page in latest nightly.

I will also note that I was getting 5 to 30 seconds between frame updates and the memory would be back to a normal rate if the page is in a non-active tab (as in I needed a new window to view about:memory in). All the "Program Not Responding" messages got very annoying.

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a1) Gecko/20110827 Firefox/9.0a1

Comment 9

[Nicholas Nethercote [inactive]]

Kevin and Randall will try to reproduce this while running their mem profiling tools.

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Jprof of running this on a fast Linux box under trunk

Note: No leak obvious.  ~750MB VSS, ~100MB RSS in a fresh new profile.  Stable.

Uses 100% CPU almost all painting (see profile)

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

Attachment: About:memory (linux)

Note: didn't do anything but hit "back" to go back to the bug.  VSS dropped.

Comment 12

[Hugh Nougher [:Hughman]]

I just noticed another thing. Through Process Explorer and its GPU Graph tab it says that firefox uses about 1GB of shared GPU memory for this web page.

I think this accounts for most of the memory not accounted for in about:memory since I believe shared memory lives in RAM.

Comment 13

[(no longer active)]

Bas: did you get a chance to see if you can reproduce this bug on Windows?

Comment 14

[Bas Schouten (:bas.schouten)]

(In reply to Ehsan Akhgari [:ehsan] from comment #13)
> Bas: did you get a chance to see if you can reproduce this bug on Windows?

I've raised the priority of the IT request a short while ago, I just pinged on it again.

Comment 15

[Johnny Stenback (:jst)]

Any luck on the IT request? If not, let me know and maybe I can help nudge that along.

Comment 16

[Bas Schouten (:bas.schouten)]

So far, it appears I can't find all these allocations in the native heap using the memory profiler. But I can see a very large amount of thebes layers doing allocations. A bunch of that possibly on the GPU.

Comment 17

[Bas Schouten (:bas.schouten)]

So this page causes a huge amount of 5x5 ThebesLayers (2000+) to be created. These layers are all component alpha because of the text on there, causing 4000+ textures to be created to support these. Since 4000 * 25 * 4 should only be 400K of memory my best guess is that overhead and minimum allocation sizes in the GPU are responsible, but I'll need to investigate that a little further, 250K+ per texture seems a little harsh, although not impossible. In any case we should probably improve usage of the layer system here.

Comment 18

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Should we simply have a lower-bound on size before we make layers active? Presumably very small layers are cheap to composite on the CPU so we can use lower-overhead structures.

Comment 19

[Bas Schouten (:bas.schouten)]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #18)
> Should we simply have a lower-bound on size before we make layers active?
> Presumably very small layers are cheap to composite on the CPU so we can use
> lower-overhead structures.

Yes, that's what I was thinking, some limit like 50x50 or 25x25 seems sensible, at least for now as a simple solution.

Comment 20

[Nicholas Nethercote [inactive]]

Getting data about this in about:memory would be good, but that sounds hard.  So implementing the workaround in comment 19 seems good enough for this bug.

Comment 21

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to Bas Schouten (:bas) from comment #17)
> I'll need to investigate that a little
> further, 250K+ per texture seems a little harsh, although not impossible. In
> any case we should probably improve usage of the layer system here.

That seems worth investingating. 250K per 5x5 layer does seem like a big problem.

Comment 22

[Bas Schouten (:bas.schouten)]

I did some stand alone textures.

Creating 2000 5x5 textures uses practically no CPU memory, and a low amount of GPU memory. I created up to 30 000 and was still all fine.

Creating 2000 Direct2D DXGI render targets for these 2000 textures consumed roughly 250M of private working set memory. So that's about 250K of Direct2D overhead per surface, high, but by itself not the cause of this issue.

I'll need to investigate further where our memory is going to get to 1.4 GB. My memory profiler does not appear to show that kind of consumption on any of the native memory heaps.

Comment 23

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Attachment: fix

This seems to fix the memory issues. The demo is still super-slow.

Comment 24

[Robert O'Callahan (:roc) (email my personal email if necessary)]

The slowness is quite likely to be some O(N^2) stuff in FrameLayerBuilder.

Comment 25

[Timothy Nikkel (:tnikkel)]

Comment on attachment 569571 [details] [diff] [review]
fix

Mention that the size is in pixels.

That 'if' is getting complicated.

Comment 26

[Robert O'Callahan (:roc) (email my personal email if necessary)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/9bbb30ec51f5

Comment 27

[Timothy Nikkel (:tnikkel)]

I meant the new constant should be mentioned to be in pixels. :) Either way is fine.

Comment 28

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Backed out:
https://hg.mozilla.org/integration/mozilla-inbound/rev/98013fe19dcb

Tests started unexpectedly passing, which reveals the problem that we use <canvas>es to force active layers in reftests, and this patch can defeat that if the canvas is too small. In fact I think we shouldn't be making canvases inactive with this patch.

Comment 29

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Attachment: fix v2

Move size check into nsDisplayItem::GetLayerState so it can be customized per-display-item-type.

Comment 30

[Robert O'Callahan (:roc) (email my personal email if necessary)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/70e60379729b

Comment 31

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Followup test fix:
http://hg.mozilla.org/integration/mozilla-inbound/rev/39fa89ebe5e2

Comment 32

[Marco Bonardo [:mak]]

https://hg.mozilla.org/mozilla-central/rev/70e60379729b
https://hg.mozilla.org/mozilla-central/rev/39fa89ebe5e2

Comment 33

[Ben Turner (not reading bugmail, use the needinfo flag!)]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #24)
> The slowness is quite likely to be some O(N^2) stuff in FrameLayerBuilder.

Just out of curiosity did something get filed on this?

Comment 34

[Robert O'Callahan (:roc) (email my personal email if necessary)]

No.