726764 - Massive memory leak in BrowserQuest HTML5 game with CoreGraphics Azure backend enabled

Status: VERIFIED FIXED

(Core :: Graphics, defect)

Description

[David Burns :automatedtester]

When playing around with the BrowserQuest HTML5 game that was sent to all today I noticed that my heap-unclassified starts growing quickly. On a different profile to details below after 20 mins it was over 1GB. Looking at https://etherpad.mozilla.org/browserquest-feedback there are others having the same issue.

Below is the memory details from a clean profile.

************ about:memory

Main Process

Explicit Allocations
318.04 MB (100.0%) -- explicit
├──173.61 MB (54.59%) ── heap-unclassified
├──112.95 MB (35.51%) -- js
│  ├───66.39 MB (20.88%) -- compartment(http://178.79.187.79/)
│  │   ├──51.64 MB (16.24%) -- gc-heap
│  │   │  ├──46.56 MB (14.64%) -- objects
│  │   │  │  ├──26.43 MB (08.31%) ── non-function
│  │   │  │  └──20.13 MB (06.33%) ── function
│  │   │  └───5.08 MB (01.60%) ++ (5 tiny)
│  │   ├───8.64 MB (02.72%) -- objects
│  │   │   ├──5.83 MB (01.83%) ── elements
│  │   │   └──2.81 MB (00.88%) ++ (2 tiny)
│  │   └───6.11 MB (01.92%) ++ (6 tiny)
│  ├───17.65 MB (05.55%) -- compartment([System Principal], 0x1178f1000)
│  │   ├──10.35 MB (03.25%) -- gc-heap
│  │   │  ├───4.32 MB (01.36%) -- objects
│  │   │  │   ├──3.29 MB (01.03%) ── function
│  │   │  │   └──1.03 MB (00.33%) ── non-function
│  │   │  ├───4.02 MB (01.26%) ++ shapes
│  │   │  └───2.01 MB (00.63%) ++ (5 tiny)
│  │   └───7.31 MB (02.30%) ++ (7 tiny)
│  ├───16.41 MB (05.16%) ── gc-heap-decommitted
│  ├────6.44 MB (02.03%) ++ (7 tiny)
│  └────6.05 MB (01.90%) -- runtime
│       ├──4.00 MB (01.26%) ── stack-committed
│       └──2.05 MB (00.64%) ++ (6 tiny)
├───15.92 MB (05.00%) -- images
│   ├──15.63 MB (04.91%) -- content
│   │  ├──15.63 MB (04.91%) -- used
│   │  │  ├──14.97 MB (04.71%) ── uncompressed-heap
│   │  │  └───0.66 MB (00.21%) ++ (2 tiny)
│   │  └───0.00 MB (00.00%) ++ unused
│   └───0.29 MB (00.09%) ++ chrome
├────8.68 MB (02.73%) ++ (9 tiny)
└────6.89 MB (02.17%) -- storage
     ├──6.26 MB (01.97%) ++ sqlite
     └──0.63 MB (00.20%) ++ prefixset

Other Measurements
   10.17 MB ── canvas-2d-pixel-bytes
    0.48 MB ── dom-total-window
  317.98 MB ── explicit
   15.26 MB ── gfx-surface-image
  229.17 MB ── heap-allocated
  237.16 MB ── heap-committed
      3.36% ── heap-committed-fragmentation
    1.54 MB ── heap-dirty
   15.83 MB ── heap-unallocated
          2 ── js-compartments-system
          3 ── js-compartments-user
   82.00 MB ── js-gc-heap
    3.58 MB ── js-main-runtime-analysis-temporary
    1.35 MB ── js-main-runtime-gc-heap-arena-unused
    0.00 MB ── js-main-runtime-gc-heap-chunk-clean-unused
    0.00 MB ── js-main-runtime-gc-heap-chunk-dirty-unused
   16.41 MB ── js-main-runtime-gc-heap-decommitted
     21.64% ── js-main-runtime-gc-heap-unused-fraction
    2.40 MB ── js-main-runtime-mjit
   60.37 MB ── js-main-runtime-objects
    4.64 MB ── js-main-runtime-scripts
    7.79 MB ── js-main-runtime-shapes
    5.27 MB ── js-main-runtime-strings
    0.75 MB ── js-main-runtime-type-inference
         21 ── page-faults-hard
    358,520 ── page-faults-soft
  414.50 MB ── resident
    6.26 MB ── storage-sqlite
    1.27 MB ── style-sheets-total-window
3,789.63 MB ── vsize


************ about:support


  Application Basics

        Name
        Firefox

        Version
        13.0a1

        User Agent
        Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0a1) Gecko/20120210 Firefox/13.0a1

        Profile Folder

          Show in Finder

        Enabled Plugins

          about:plugins

        Build Configuration

          about:buildconfig

        Crash Reports

          about:crashes

        Memory Use

          about:memory

  Extensions

        Name

        Version

        Enabled

        ID

        DivX Plus Web Player HTML5 <video>
        2.1.2.145
        false
        {23fcfd51-4958-4f00-80a3-ae97e717ed8b}

  Important Modified Preferences

      Name

      Value

        browser.cache.disk.capacity
        1048576

        browser.cache.disk.smart_size.first_run
        false

        browser.cache.disk.smart_size_cached_value
        1048576

        browser.places.smartBookmarksVersion
        2

        browser.startup.homepage_override.buildID
        20120210031150

        browser.startup.homepage_override.mstone
        rv:13.0a1

        extensions.lastAppVersion
        13.0a1

        network.cookie.prefsMigrated
        true

        places.history.expiration.transient_current_max_pages
        104858

        privacy.sanitize.migrateFx3Prefs
        true

  Graphics

        Vendor ID
        0x10de

        Device ID
        0x a29

        WebGL Renderer
        NVIDIA Corporation -- NVIDIA GeForce GT 330M OpenGL Engine -- 2.1 NVIDIA-1.6.40

        GPU Accelerated Windows
        1/1 OpenGL

        AzureBackend
        quartz

Comment 1

[Andrew McCreight [:mccr8]]

Here are the reports I saw about heap-unclassified in the etherpad:

"After plaing for a while my memory usage spiked up to 1,7GB and did not decrease when reloading or closing the tab containing the game. Not sure if there's some core problem here? about:memory wasn't very helpful (1,404.71 MB (77.28%) ── heap-unclassified)."

"I played a bit longer. Had real mem 1.4 gb, virtual mem 16gb, about:memory says 12gb in heap-unclassified. Closing tab or doing gc+cc doesn't reduce this. This is Aurora 12.0a2 on Mac OS X 10.7.2, Macbook Air. canvas-2d-pixel-bytes reports 0.4mb - but I'm guessing it's all canvas related."

"Mine got up to 19.2 GB in heap-unclassified, Activity Monitor on OS X showed 2.3GB in physical and 21 GB of VMEM in use.  Something's leaking.  It didn't go away closing the tab, had to restart Aurora to recover the memory."

Comment 2

[David Burns :automatedtester]

After a few minutes(10 minutes at most) of playing it in safe mode I got this.

Main Process

Explicit Allocations
381.65 MB (100.0%) -- explicit
├──246.43 MB (64.57%) ── heap-unclassified
├──102.97 MB (26.98%) -- js
│  ├───42.37 MB (11.10%) ── gc-heap-decommitted
│  ├───36.64 MB (09.60%) -- compartment(http://178.79.187.79/)
│  │   ├──26.91 MB (07.05%) -- gc-heap
│  │   │  ├──22.95 MB (06.01%) -- objects
│  │   │  │  ├──16.99 MB (04.45%) ── non-function
│  │   │  │  └───5.96 MB (01.56%) ── function
│  │   │  └───3.97 MB (01.04%) ++ (5 tiny)
│  │   ├───6.52 MB (01.71%) -- objects
│  │   │   ├──5.84 MB (01.53%) ── elements
│  │   │   └──0.68 MB (00.18%) ++ (2 tiny)
│  │   └───3.21 MB (00.84%) ++ (5 tiny)
│  ├───13.33 MB (03.49%) -- compartment([System Principal], 0x1176f1000)
│  │   ├───7.82 MB (02.05%) ++ gc-heap
│  │   └───5.50 MB (01.44%) ++ (6 tiny)
│  ├────5.99 MB (01.57%) -- runtime
│  │    ├──4.00 MB (01.05%) ── stack-committed
│  │    └──1.99 MB (00.52%) ++ (6 tiny)
│  └────4.64 MB (01.22%) ++ (7 tiny)
├───16.40 MB (04.30%) -- images
│   ├──15.97 MB (04.18%) -- content
│   │  ├──15.97 MB (04.18%) -- used
│   │  │  ├──15.36 MB (04.02%) ── uncompressed-heap
│   │  │  └───0.61 MB (00.16%) ++ (2 tiny)
│   │  └───0.00 MB (00.00%) ++ unused
│   └───0.43 MB (00.11%) ++ chrome
├───10.50 MB (02.75%) ++ (8 tiny)
└────5.34 MB (01.40%) -- storage
     ├──4.73 MB (01.24%) ++ sqlite
     └──0.62 MB (00.16%) ++ prefixset

Other Measurements
   10.27 MB ── canvas-2d-pixel-bytes
    0.51 MB ── dom-total-window
  381.65 MB ── explicit
   15.88 MB ── gfx-surface-image
  296.79 MB ── heap-allocated
  310.23 MB ── heap-committed
      4.32% ── heap-committed-fragmentation
    2.45 MB ── heap-dirty
   28.21 MB ── heap-unallocated
          2 ── js-compartments-system
          3 ── js-compartments-user
   80.00 MB ── js-gc-heap
    2.63 MB ── js-main-runtime-analysis-temporary
    0.70 MB ── js-main-runtime-gc-heap-arena-unused
    0.00 MB ── js-main-runtime-gc-heap-chunk-clean-unused
    0.00 MB ── js-main-runtime-gc-heap-chunk-dirty-unused
   42.37 MB ── js-main-runtime-gc-heap-decommitted
     53.83% ── js-main-runtime-gc-heap-unused-fraction
    0.00 MB ── js-main-runtime-mjit
   33.22 MB ── js-main-runtime-objects
    3.76 MB ── js-main-runtime-scripts
    5.95 MB ── js-main-runtime-shapes
    4.86 MB ── js-main-runtime-strings
    0.14 MB ── js-main-runtime-type-inference
         16 ── page-faults-hard
  8,343,810 ── page-faults-soft
  453.72 MB ── resident
    4.73 MB ── storage-sqlite
    1.31 MB ── style-sheets-total-window
3,751.05 MB ── vsize

Comment 3

[Nicholas Nethercote [inactive]]

I tried to replicate this, but I'm suffering from a bug that some other people have hit, which is that my character simply won't move :(  My current heap-unclassified:

├───16.49 MB (14.70%) ── heap-unclassified

14.70% is about as *low* as I've ever seen it.

Comment 4

[David Burns :automatedtester]

(In reply to Nicholas Nethercote [:njn] from comment #3)
> I tried to replicate this, but I'm suffering from a bug that some other
> people have hit, which is that my character simply won't move :(  

Refreshing the page gets you going again

Comment 5

[Nicholas Nethercote [inactive]]

> Refreshing the page gets you going again

Nope.  I've heard that some people hit this problem after playing for a while, and refreshing fixes that.  In contrast, my player won't move even at the start of the game, and refreshing doesn't help.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

After playing some minutes I ended-up in:

3,039,048,131 B (100.0%) -- explicit
├──2,842,934,728 B (93.55%) ── heap-unclassified

I will add a memchaser log from this whole session right away.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: MemChaser Log

Doesn't look like that using Resident in MemChaser was a good idea. We probably want to change it to explicit. I ended up with about 8GB of memory usage after about 15 minutes.

Comment 8

[David Burns :automatedtester]

(In reply to Nicholas Nethercote [:njn] from comment #5)
> > Refreshing the page gets you going again
> 
> Nope.  I've heard that some people hit this problem after playing for a
> while, and refreshing fixes that.  In contrast, my player won't move even at
> the start of the game, and refreshing doesn't help.

Refreshing doesnt release any resource for me. I can refresh a lot and the values still go up

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is a regression in Aurora and Nightly builds. The high memory usage doesn't exist in Release and Beta builds. I will work on the regression range.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Regression started between the builds 2012-01-17-03 and 2012-01-18-03 on mozilla-central:

Pushlog: 
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=34572943a3e4&tochange=f4049f65efc6

There are way too many changes during that day. I will have to start bisecting. Will be done in a couple of hours at latest.

Comment 11

[Tim Taubert [:ttaubert] (inactive)]

(In reply to Henrik Skupin (:whimboo) from comment #9)
> I will work on the regression range.

That's surely gonna be some hard work... ;)

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So this has been regressed on OS X because of the implementation of the CoreGraphics Azure backend on bug 692879.

Whenever you disable 'gfx.canvas.azure.enabled' the leak goes away. Moving bug into Core:Graphics.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We should track this for Firefox 12 and 13.

Comment 14

[Bas Schouten (:bas.schouten)]

I cannot reproduce this leak on D2D Azure Canvas on FX 9.

Comment 15

[Jeff Muizelaar [:jrmuizel]]

Attachment: Don't leak the path when getting it for text.

Comment 16

[Nicholas Nethercote [inactive]]

Jeff, is this something for which a memory reporter would be useful?  Or now that it's not leaking are the paths likely to be small and/or short-lived?

Comment 17

[Jeff Muizelaar [:jrmuizel]]

(In reply to Nicholas Nethercote [:njn] from comment #16)
> Jeff, is this something for which a memory reporter would be useful?  Or now
> that it's not leaking are the paths likely to be small and/or short-lived?

The paths are a CoreGraphics objects that we don't know the size of, they also tend to be short lived. A memory reporter is probably not going to be that useful.

Comment 18

[Joe Drew (not getting mail)]

Comment on attachment 597078 [details] [diff] [review]
Don't leak the path when getting it for text.

Review of attachment 597078 [details] [diff] [review]:
-----------------------------------------------------------------

your diffs should be -U 8!!

Comment 19

[Nicholas Nethercote [inactive]]

> The paths are a CoreGraphics objects that we don't know the size of

If they're heap-allocated and we have a pointer to them, we can use moz_malloc_usable_size() without having to know how big they are...

> , they
> also tend to be short lived. A memory reporter is probably not going to be
> that useful.

... but it doesn't sound worthwhile.  Thanks!

Comment 20

[Marco Bonardo [:mak]]

https://hg.mozilla.org/mozilla-central/rev/1aa0d78efa8d

Comment 21

[Jeff Muizelaar [:jrmuizel]]

David, can you retest this on tomorrow's nightly?

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Jeff, I will test once I was able to build with the latest code on m-c. Right now it fails due to bug 727595.

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Tested with latest changeset ae8cce613aa0 and the leak is gone. Thanks Jeff for the quick fix.

Looks like we can go ahead and land it for Aurora.

Comment 24

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 597078 [details] [diff] [review]
Don't leak the path when getting it for text.

[Approval Request Comment]
Regression caused by (bug #): 692879
User impact if declined: memory leaks when stroking text
Testing completed (on m-c, etc.): the problem is fixed on m-c
Risk to taking this patch (and alternatives if risky): very low risk
String changes made by this patch: none

Comment 25

[Alex Keybl [:akeybl]]

Comment on attachment 597078 [details] [diff] [review]
Don't leak the path when getting it for text.

[Triage Comment]
Low risk and fixes a major memory leak found in a web app being used for demos internally. Approving for Aurora.

Comment 26

[Jeff Muizelaar [:jrmuizel]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/f27b777981ff

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Verified fixed with Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:12.0a2) Gecko/20120226 Firefox/12.0a2 ID:20120226042010