976584 - [gaia-ui-endurance] Increase in marionette listener, strings, and runtime script sources memory consumption during endurance tests

Status: RESOLVED WORKSFORME

(Remote Protocol :: Marionette, defect, P2)

Description

[Robert Wood [:rwood]]

The parent b2g process memory use has increased when using the camera viewfinder, on Hamachi master. Fluctuations are common in the endurance tests, however it looks like it is remaining consistently at the higher level so I wanted to point it out in case there is a memory regression here.

Here is the latest test data for the camera viewfinder endurance test:

https://datazilla.mozilla.org/b2g/?branch=master&device=hamachi&range=90&test=endurance_camera_viewfinder&app_list=browser,calendar,camera,clock,contacts,fm_radio,gallery,messages,music,phone,settings,usage,video&app=camera&gaia_rev=49312fb89b90eca4&gecko_rev=b9bb3473d3f674da&plot=avg

This test just starts the camera app, leaves the viewfinder open for 30 seconds, then closes the app. The test is repeated 100x. The Y-axis is actually the b2g parent process RSS memory value.

More info about the gaia-ui endurance tests can be found at the link below. Note that currently they are run on hamachi master and v1.2 only.

https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance

Comment 1

[Jason Smith [:jsmith]]

I'm pretty sure this is a regression from the camera refactoring work in bug 909542 & bug 958200.

How much memory did we increase by in KB?

Comment 2

[Robert Wood [:rwood]]

Approx. 10,000 kb.

2-Feb: 68282
3-Feb: 64839
4-Feb: 64656

Then the rise:

5-Feb: 68656
6-Feb: 71884
7-Feb: 76760

8-Feb to latest 24-Feb stays consistently between 74996 and 77215.

Comment 3

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to Jason Smith [:jsmith] from comment #1)
> I'm pretty sure this is a regression from the camera refactoring work in bug
> 909542 & bug 958200.

The regression occurs between Feb 5 and Feb 8; bug 909542 and bug 958200 didn't land (and stick) until fully a week later, on Feb 15.

Guys, were there any changes in the Camera app in that above window that would cause a 10MiB increase in memory usage? (Conversely, were there any changes between Dec 17, and Dec 29, 2013, that would cause a similar drop?)

Comment 4

[Wilson Page [:wilsonpage]]

mikeh: It doesn't look like anything sizable landed between those dates:

https://github.com/mozilla-b2g/gaia/commit/45fec63fdaf7929ac5f3504ff6d8d1077ae674f5#diff-b8bbcd834d411eb51460810dafde25df

https://github.com/mozilla-b2g/gaia/commit/c0d7d914687f956a8373fdd09d4cf372b2bfb2ad#diff-b8bbcd834d411eb51460810dafde25df

https://github.com/mozilla-b2g/gaia/commit/bce7086866ffb3d1256a1614f1b6d9710b697722#diff-b8bbcd834d411eb51460810dafde25df

https://github.com/mozilla-b2g/gaia/commit/f0558e03a8aae2ef3e7e8a4c5506a6aa1947f678#diff-b8bbcd834d411eb51460810dafde25df

Comment 5

[Mike Habicher [:mikeh] (high bugzilla latency)]

Rob, does this memory increase only appear when running the Camera app?

Comment 6

[David Flanagan [:djf]]

Could this be related to all the graphics changes that have been going on?

Also, we've had some back and forth in our algorithm for selecting the resolution of the preview stream. Could we have switched from a smaller to a larger preview stream on those dates?

Maybe an about:memory analysis before and after this change would be informative.

Comment 7

[Mike Habicher [:mikeh] (high bugzilla latency)]

One thing to keep in mind here is that the reported memory increase is in the PARENT process, not in the Camera app. My understanding is that there are limited ways in which the latter can inflate the former; graphics may be one way, but I don't think the preview size could. A higher-resolution preview stream would require larger graphics buffers, but not 10MB worth.

Comment 8

[Justin D'Arcangelo [:justindarc]]

David: I was thinking the same thing when I looked at this yesterday. However, this was on Hamachi and the previewSize selection bug fixed in this regression date range did not have any effect on Hamachi.

Comment 9

[Ben Kelly [:bkelly, not reviewing]]

This sounds similar to the contact memory regression in bug 970360.  That was around the same timeframe and was due to keyboard moving back into the parent process.

Can some retest with the keyboard change backed out?  Can we dupe to bug 970360?

Comment 10

[Jason Smith [:jsmith]]

(In reply to Ben Kelly [:bkelly] from comment #9)
> This sounds similar to the contact memory regression in bug 970360.  That
> was around the same timeframe and was due to keyboard moving back into the
> parent process.
> 
> Can some retest with the keyboard change backed out?  Can we dupe to bug
> 970360?

Don't think so. Nothing in this test to my understanding involves the keyboard. bug 970360 on the other hand does involve the keyboard.

Comment 11

[Ben Kelly [:bkelly, not reviewing]]

Is there an about memory before/after?  I think the keyboard will still be resident in the parent process, won't it?

Comment 12

[Hema Koka [:hema]]

memory regression moving to blocker per triage on 2/26

Mike Lee,

We need your team's help here for investigation.

Thanks
Hema

Comment 13

[Robert Wood [:rwood]]

Attachment: http://people.mozilla.org/~rwood/bug976584/about_memory.zip

Link to about_memory dumps from before and after running the camera_viewfinder endurance test (100 iterations, on mar4 hamachi master)

Comment 14

[Ben Kelly [:bkelly, not reviewing]]

Seems like marionette is doing something bad with reloading scripts.  Large increase in strings, script sources, and the marionette-listener.js compartment:

21.93 MB (100.0%) -- explicit
├──23.50 MB (107.14%) -- js-non-window
│  ├──13.79 MB (62.88%) -- zones/zone(0xNNN)
│  │  ├───8.91 MB (40.62%) -- compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │   ├──5.01 MB (22.85%) ++ classes
│  │  │   ├──2.60 MB (11.87%) ── scripts/gc-heap [147]
│  │  │   └──1.29 MB (05.90%) ── sundries/malloc-heap [147]
│  │  ├───3.02 MB (13.78%) ++ strings
│  │  ├───1.04 MB (04.76%) ++ compartment([System Principal])
│  │  ├───0.91 MB (04.14%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  └──-0.09 MB (-0.41%) ++ (15 tiny)
│  ├───9.60 MB (43.77%) -- runtime
│  │   ├──9.34 MB (42.60%) -- script-sources
│  │   │  ├──9.21 MB (41.98%) -- source(scripts=148, dummy file)
│  │   │  │  ├──9.20 MB (41.94%) ── uncompressed [+]
│  │   │  │  └──0.01 MB (00.04%) ── misc [+]
│  │   │  ├──0.70 MB (03.19%) -- source(scripts=567, <non-notable files>)
│  │   │  │  ├──0.67 MB (03.04%) ── uncompressed [+]
│  │   │  │  └──0.03 MB (00.15%) ── misc [+]
│  │   │  └──-0.56 MB (-2.57%) ++ (4 tiny)
│  │   ├──0.27 MB (01.24%) ── script-data
│  │   └──-0.02 MB (-0.07%) ++ (2 tiny)
│  └───0.11 MB (00.50%) ── gc-heap/chunk-admin
└──-1.57 MB (-7.14%) ++ (15 tiny)

Comment 15

[Jonathan Griffin (:jgriffin)]

David, mdas, did we land any Marionette changes in the Feb 4-6 timeframe that could account for this?

Comment 16

[Malini Das [:mdas] - Away, not checking bugmail]

I don't see anything related in the logs. Most changes during that time were for the client, and nothing related to script loading. We may have to bisect this.

Comment 17

[Jonathan Griffin (:jgriffin)]

Rob, is the version of gaiatest that these tests use pinned, or do they use the latest in the gaia repo?  If the latter, this could be related to a change in gaiatest as well.

Comment 18

[Robert Wood [:rwood]]

No, the gaiatest version is not pinned

Comment 19

[David Burns :automatedtester]

According to mercurial these are the changes that landed in that time frame

changeset:   167285:7dbb42dd1cf4
user:        Ryan VanderMeulen <[e-mail]>
date:        Thu Feb 06 15:37:48 2014 -0500
summary:     Backed out changeset 5d1c157bbb46 (bug 940554) for changing the Marionette script timeout message in a TBPL-unfriendly way.

changeset:   167277:799c9a7716ef
user:        Andreas Tolfsen <[e-mail]>
date:        Thu Feb 06 14:57:29 2014 -0500
summary:     Bug 941158 - Honour expected failures in setUp or tearDown. r=jgriffin

changeset:   167240:5d1c157bbb46
user:        Andreas Tolfsen <[e-mail]>
date:        Thu Feb 06 09:46:28 2014 -0500
summary:     Bug 940554 - Fix Marionette's newSession to return capabilities. r=mdas

changeset:   167118:54c21f290a8d
user:        Wes Kocher <[e-mail]>
date:        Wed Feb 05 15:17:52 2014 -0800
summary:     Backed out changeset a246bbc0c61c (bug 941158) for marionette bustage

changeset:   167105:a246bbc0c61c
user:        Andreas Tolfsen <[e-mail]>
date:        Wed Feb 05 17:04:42 2014 -0500
summary:     Bug 941158 - Honour expected failures in setUp or tearDown. r=zac

changeset:   167104:cdc6fd62d987
user:        Andreas Tolfsen <[e-mail]>
date:        Wed Feb 05 17:04:42 2014 -0500
summary:     Bug 938228 - Fix marshaling of NodeList in executeScript. r=mdas

changeset:   167097:21826d6c7d6e
user:        David Burns <[e-mail]>
date:        Wed Feb 05 21:38:12 2014 +0000
summary:     Bug 874868: Make sure that Marionette takes a screenshot of the the entire content canvas; r=mdas

changeset:   166712:cdc2338e71a3
user:        Andrew Halberstadt <[e-mail]>
date:        Tue Feb 04 08:53:18 2014 -0500
summary:     Bug 949028 - Add check for crashes to geckoinstance.py, r=mdas

Comment 20

[David Burns :automatedtester]

My guess is that it is either bug 874868 or bug 938228, if its marionette code causing the issue

Comment 21

[Robert Wood [:rwood]]

That is a possibility that it is related to gaiatest, as:

Feb-6: tests used gaiatest 21.4, result: 71884
Feb-7: gaiatest upgraded to 21.5, result: 76760

However note there have been some more fluctuations recently (including a spike last night on several tests) and gaiatest did not change in that case.

https://datazilla.mozilla.org/b2g/?branch=master&device=hamachi&range=90&test=endurance_camera_viewfinder&app_list=browser,calendar,camera,clock,contacts,fm_radio,gallery,messages,music,phone,settings,usage,video&app=camera&gaia_rev=49312fb89b90eca4&gecko_rev=b9bb3473d3f674da&plot=avg

Comment 22

[Jonathan Griffin (:jgriffin)]

None of those look too suspicious.

Rob, do we have the about memory reports from the Feb 4 run? I want to confirm that we don't see the large marionette string consumption there.

We may want to do some bisection runs after that.  I'd recommend locally saving some builds from https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/mozilla-central-hamachi-eng/, since we're near the 30-day limit where those are purged; i.e., tomorrow the Feb 4 builds will be deleted.

Comment 23

[Nicholas Nethercote [inactive]]

Yeah, would be nice to know if this is Marionette or Gecko at fault.

Comment 24

[Jason Smith [:jsmith]]

Bouncing over to Marionette for now. If this ends up not being a Marionette bug, then bounce it back into the FxOS product.

Comment 25

[David Burns :automatedtester]

Attachment: about-memory.zip

adding example about:memory when running against 2014-02-04 using gaiatest 0.21.4, 0.21.5, 0.21.8 all pretty flat. Will carry on tomorrow (today really)

Comment 26

[Robert Wood [:rwood]]

Unfortunately no, we don't have about_memory reports from historical endurance test runs on jenkins. Since all the endurance tests are run by manifest, to get about_memory between the tests it would need to be integrated into the framework which I haven't done. Whenever an about_memory has been needed, I've just been kicking off single tests and grabbing the memory dump before and after.

Thanks :automatedtester re: comment 25. I can do some more test runs today also.

Comment 27

[David Burns :automatedtester]

Attachment: about-memory-gaiatest0.21.4.zip

Main Process is using 43.28MB using GaiaTest 0.21.4 and the build from 2014-02-03

43.28 MB (100.0%) -- explicit
├──17.03 MB (39.35%) -- js-non-window
│  ├───9.99 MB (23.09%) -- zones
│  │   ├──7.78 MB (17.97%) -- zone(0x43c02000)
│  │   │  ├──4.08 MB (09.42%) -- compartment([System Principal])
│  │   │  │  ├──1.95 MB (04.51%) -- objects
│  │   │  │  │  ├──1.08 MB (02.49%) -- gc-heap
│  │   │  │  │  │  ├──0.50 MB (01.16%) ── ordinary
│  │   │  │  │  │  ├──0.46 MB (01.07%) ── function
│  │   │  │  │  │  └──0.11 MB (00.26%) ++ (2 tiny)
│  │   │  │  │  ├──0.88 MB (02.02%) -- malloc-heap
│  │   │  │  │  │  ├──0.49 MB (01.14%) ── slots
│  │   │  │  │  │  └──0.38 MB (00.89%) ++ (2 tiny)
│  │   │  │  │  └──0.00 MB (00.00%) ── non-heap/code/asm.js
│  │   │  │  ├──1.19 MB (02.75%) -- shapes
│  │   │  │  │  ├──0.63 MB (01.47%) ++ gc-heap
│  │   │  │  │  └──0.55 MB (01.28%) ++ malloc-heap
│  │   │  │  ├──0.88 MB (02.04%) -- scripts
│  │   │  │  │  ├──0.70 MB (01.62%) ── gc-heap
│  │   │  │  │  └──0.18 MB (00.43%) ── malloc-heap/data
│  │   │  │  └──0.05 MB (00.12%) ++ (2 tiny)
│  │   │  ├──1.62 MB (03.75%) -- strings
│  │   │  │  ├──1.05 MB (02.42%) ++ notable
│  │   │  │  ├──0.54 MB (01.25%) ++ normal
│  │   │  │  └──0.04 MB (00.09%) ── short-gc-heap
│  │   │  ├──0.98 MB (02.25%) ── unused-gc-things
│  │   │  ├──0.66 MB (01.53%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://browser/content/shell.html)
│  │   │  └──0.44 MB (01.02%) ++ (14 tiny)
│  │   ├──2.01 MB (04.64%) -- zone(0x43c02800)
│  │   │  ├──1.98 MB (04.57%) -- strings
│  │   │  │  ├──1.71 MB (03.95%) -- normal
│  │   │  │  │  ├──1.30 MB (03.00%) ── malloc-heap
│  │   │  │  │  └──0.41 MB (00.95%) ── gc-heap
│  │   │  │  └──0.27 MB (00.63%) ── short-gc-heap
│  │   │  └──0.03 MB (00.07%) ++ (4 tiny)
│  │   └──0.20 MB (00.47%) ++ zone(0x43c04000)
│  ├───6.92 MB (15.98%) -- runtime
│  │   ├──3.07 MB (07.08%) ── script-sources
│  │   ├──2.33 MB (05.38%) ── script-data
│  │   ├──1.00 MB (02.31%) ── atoms-table
│  │   └──0.52 MB (01.21%) ++ (10 tiny)
│  └───0.13 MB (00.29%) ++ gc-heap
├───6.97 MB (16.11%) ── heap-unclassified
├───4.94 MB (11.40%) -- images
│   ├──4.93 MB (11.40%) -- content
│   │  ├──4.93 MB (11.40%) -- used
│   │  │  ├──4.48 MB (10.35%) ── uncompressed-heap
│   │  │  ├──0.46 MB (01.06%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───4.64 MB (10.73%) -- window-objects
│   ├──4.13 MB (09.54%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.39 MB (07.83%) -- active/window(app://system.gaiamobile.org/index.html)
│   │  │  ├──1.36 MB (03.15%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──0.61 MB (01.41%) ++ shapes
│   │  │  │  ├──0.60 MB (01.38%) ++ objects
│   │  │  │  └──0.16 MB (00.37%) ++ (5 tiny)
│   │  │  ├──0.75 MB (01.73%) ++ layout
│   │  │  ├──0.67 MB (01.55%) ── style-sheets
│   │  │  ├──0.59 MB (01.37%) ++ dom
│   │  │  └──0.01 MB (00.03%) ── property-tables
│   │  └──0.74 MB (01.71%) ++ js-zone(0x478d6000)
│   └──0.52 MB (01.19%) ++ top(chrome://browser/content/shell.html, id=1)
├───2.74 MB (06.34%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x43fa7c00)
│   ├──1.54 MB (03.57%) -- runtime
│   │  ├──0.90 MB (02.09%) ── script-sources
│   │  └──0.64 MB (01.48%) ++ (12 tiny)
│   ├──0.63 MB (01.46%) ++ (3 tiny)
│   └──0.57 MB (01.31%) -- zone(0x44fa6400)
│      ├──0.48 MB (01.12%) ++ compartment(web-worker)
│      └──0.08 MB (00.19%) ++ (2 tiny)
├───2.51 MB (05.80%) -- heap-overhead
│   ├──1.92 MB (04.44%) ── waste
│   ├──0.59 MB (01.35%) ── page-cache
│   └──0.00 MB (00.01%) ── bookkeeping
├───2.11 MB (04.88%) ++ (15 tiny)
├───0.88 MB (02.04%) ── xpti-working-set
├───0.80 MB (01.86%) -- gfx
│   ├──0.59 MB (01.36%) ── heap-textures
│   └──0.21 MB (00.50%) ++ (5 tiny)
└───0.64 MB (01.49%) -- storage/sqlite
    ├──0.56 MB (01.29%) ── other
    └──0.09 MB (00.20%) ++ (3 tiny)

Comment 28

[David Burns :automatedtester]

Attachment: about-memory-gaiatest0.21.8.zip

Memory usage on main process is 46.90 MB using GaiaTest 0.21.8 on the build from 2014-02-03


46.90 MB (100.0%) -- explicit
├──20.84 MB (44.43%) -- js-non-window
│  ├──13.20 MB (28.14%) -- zones
│  │  ├──10.98 MB (23.41%) -- zone(0x435ea000)
│  │  │  ├───4.97 MB (10.60%) -- compartment([System Principal])
│  │  │  │   ├──2.61 MB (05.56%) -- objects
│  │  │  │   │  ├──1.39 MB (02.96%) -- gc-heap
│  │  │  │   │  │  ├──0.70 MB (01.48%) ── ordinary
│  │  │  │   │  │  ├──0.54 MB (01.15%) ── function
│  │  │  │   │  │  └──0.15 MB (00.33%) ++ (2 tiny)
│  │  │  │   │  ├──1.22 MB (02.61%) -- malloc-heap
│  │  │  │   │  │  ├──0.63 MB (01.35%) ── slots
│  │  │  │   │  │  ├──0.54 MB (01.16%) ── elements/non-asm.js
│  │  │  │   │  │  └──0.05 MB (00.10%) ++ (2 tiny)
│  │  │  │   │  └──0.00 MB (00.00%) ── non-heap/code/asm.js
│  │  │  │   ├──1.37 MB (02.93%) -- shapes
│  │  │  │   │  ├──0.72 MB (01.54%) ++ gc-heap
│  │  │  │   │  └──0.65 MB (01.39%) ++ malloc-heap
│  │  │  │   ├──0.88 MB (01.88%) -- scripts
│  │  │  │   │  ├──0.70 MB (01.49%) ── gc-heap
│  │  │  │   │  └──0.18 MB (00.39%) ── malloc-heap/data
│  │  │  │   └──0.10 MB (00.22%) ++ (3 tiny)
│  │  │  ├───3.22 MB (06.86%) -- strings
│  │  │  │   ├──2.34 MB (04.99%) -- notable
│  │  │  │   │  ├──1.59 MB (03.39%) ++ (25 tiny)
│  │  │  │   │  └──0.75 MB (01.60%) -- string(length=32357, copies=12, "/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaLockScreen = {/n/n  unlock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.unlock(true);/n        waitFor(/n          function() {/n            finish(lockscreen.locked);/n          },/n          function() {/n            return !lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  },/n/n  lock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.lock(true);/n        waitFor(/n          function() {/n            finish(!lockscreen.locked);/n          },/n          function() {/n            return lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  }/n};/n/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaApps = {/n/n  normalizeName: function(name) {/n    return name.replace(/[- ]+/g, '').toLowerCase();/n  },/n/n  getInstalledApps: function() {/n    let req = navigator.mozApps.mgmt.getAll();/n    req.onsuccess = function() {/n      marionetteScriptFinished(req.result);/n    }/n  },/n/n  getRunningApps: function() {/n    let manager = window.wrappedJSObject.AppWindowManager || window.wrappedJSObject.WindowManager;/n    let runningApps = manager.getRunningApps();/n    // Return a simplified version of the runningApps object which can be/n    // JSON-serialized./n    let apps = {};/n    for (let app in runningApps) {/n        let anApp = {};/n        for (let key in runningApps[app]) {/n            if (['name', 'origin', 'manifest'].indexOf(key) > -1) {/n                anApp[key] = runningApps[app][key];/n            }/n        }/n        apps[app] = anApp;/n    }/n    return apps;/n  },/n/n  getRunningAppOrigin: function(name) {/n    let manager = window.wrappedJSObject.AppWindowManager || window.wrappedJSObject.WindowManager;/n    let runningApps = manager.getRunningApps();/n    let origin;/n/n    for (let property in runningApps) {/n      if (runningApps[property].name == name) {/n        origin = property;/n      }/n    }/n/n    return origin;/n  },/n/n  getPermission: function(appName, permissionName) {/n    GaiaApps.locateWithName(appName, function(app) {/n      console.log("Getting permission '" + permissionName + "' for " + appName);/n      var mozPerms = navigator.mozPermissionSettings;/n      var result = mozPerms.get(/n        permissionName, app.manifestURL, app.origin, false/n      );/n      marionetteScriptFinished(result);/n    });/n  },/n/n  setPermission: function(appName, permissionName, value) {/n    GaiaApps.locateWithName(appName, function(app) {/n      console.log("Setting permission '" + permissionName + "' for " +/n        appName + "to '" + value + "'");/n      var mozPerms = navigator.mozPermissionSettings;/n      mozPerms.set(/n        permissionName, value, app.manifestURL, app.origin, false/n      );/n      marionetteScriptFinished();/n    });/n  },/n/n  sendLocateResponse: function(aCallback, app, appName, entryPoint) {/n    var callback = aCallback || marionetteScriptFinished;/n    if (callback === marionetteScriptFinished) {/n " (truncated))
│  │  │  │   │     ├──0.75 MB (01.60%) ── malloc-heap
│  │  │  │   │     └──0.00 MB (00.00%) ── gc-heap
│  │  │  │   ├──0.83 MB (01.76%) -- normal
│  │  │  │   │  ├──0.63 MB (01.34%) ── malloc-heap
│  │  │  │   │  └──0.20 MB (00.43%) ── gc-heap
│  │  │  │   └──0.05 MB (00.11%) ── short-gc-heap
│  │  │  ├───1.12 MB (02.38%) -- compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://browser/content/shell.html)
│  │  │  │   ├──0.63 MB (01.35%) ++ (4 tiny)
│  │  │  │   └──0.48 MB (01.03%) ++ objects
│  │  │  ├───0.73 MB (01.56%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │  ├───0.68 MB (01.46%) ── unused-gc-things
│  │  │  └───0.26 MB (00.55%) ++ (14 tiny)
│  │  ├───2.01 MB (04.30%) -- zone(0x435ea800)
│  │  │   ├──1.98 MB (04.23%) -- strings
│  │  │   │  ├──1.71 MB (03.65%) -- normal
│  │  │   │  │  ├──1.30 MB (02.77%) ── malloc-heap
│  │  │   │  │  └──0.41 MB (00.88%) ── gc-heap
│  │  │   │  └──0.27 MB (00.58%) ── short-gc-heap
│  │  │   └──0.03 MB (00.07%) ++ (4 tiny)
│  │  └───0.20 MB (00.43%) ++ zone(0x435ec000)
│  ├───7.50 MB (15.99%) -- runtime
│  │   ├──3.57 MB (07.61%) ── script-sources
│  │   ├──2.34 MB (05.00%) ── script-data
│  │   ├──1.00 MB (02.13%) ── atoms-table
│  │   └──0.59 MB (01.25%) ++ (10 tiny)
│  └───0.14 MB (00.30%) ++ gc-heap
├───7.37 MB (15.73%) ── heap-unclassified
├───5.00 MB (10.66%) -- window-objects
│   ├──4.48 MB (09.54%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.73 MB (07.96%) -- active/window(app://system.gaiamobile.org/index.html)
│   │  │  ├──1.68 MB (03.59%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──0.75 MB (01.60%) -- objects
│   │  │  │  │  ├──0.52 MB (01.10%) ++ gc-heap
│   │  │  │  │  └──0.23 MB (00.50%) ++ (2 tiny)
│   │  │  │  ├──0.65 MB (01.38%) ++ shapes
│   │  │  │  └──0.29 MB (00.61%) ++ (5 tiny)
│   │  │  ├──0.76 MB (01.62%) ++ layout
│   │  │  ├──0.67 MB (01.43%) ── style-sheets
│   │  │  ├──0.61 MB (01.30%) ++ dom
│   │  │  └──0.01 MB (00.02%) ── property-tables
│   │  └──0.74 MB (01.59%) ++ js-zone(0x466c4000)
│   └──0.52 MB (01.11%) ++ top(chrome://browser/content/shell.html, id=1)
├───4.99 MB (10.65%) -- images
│   ├──4.99 MB (10.65%) -- content
│   │  ├──4.99 MB (10.65%) -- used
│   │  │  ├──4.52 MB (09.64%) ── uncompressed-heap
│   │  │  ├──0.47 MB (01.01%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───2.81 MB (05.98%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x440b7c00)
│   ├──1.61 MB (03.43%) -- runtime
│   │  ├──0.90 MB (01.93%) ── script-sources
│   │  └──0.70 MB (01.50%) ++ (12 tiny)
│   ├──0.63 MB (01.35%) ++ (3 tiny)
│   └──0.57 MB (01.21%) -- zone(0x450e6000)
│      ├──0.48 MB (01.03%) ++ compartment(web-worker)
│      └──0.08 MB (00.18%) ++ (2 tiny)
├───1.95 MB (04.15%) ++ (15 tiny)
├───1.88 MB (04.00%) -- heap-overhead
│   ├──1.36 MB (02.90%) ── waste
│   ├──0.51 MB (01.09%) ── page-cache
│   └──0.00 MB (00.01%) ── bookkeeping
├───0.88 MB (01.88%) ── xpti-working-set
├───0.64 MB (01.38%) -- storage/sqlite
│   ├──0.56 MB (01.19%) ── other
│   └──0.09 MB (00.19%) ++ (3 tiny)
└───0.53 MB (01.14%) ++ xpconnect

Comment 29

[David Burns :automatedtester]

I restarted the phone between runs to make sure it cleared everything.

It appears that between GaiaTest 0.21.4 and GaiaTest 0.21.8 we are using 3.62 MB more on a test that iterates 100 times. I have attached the about:memory reports and can't see anything obvious about Marionette so don't think its that.

Comment 30

[David Burns :automatedtester]

running same tests againt 2014-02-04 build now

Comment 31

[Jonathan Griffin (:jgriffin)]

(In reply to David Burns :automatedtester from comment #29)
> I restarted the phone between runs to make sure it cleared everything.
> 
> It appears that between GaiaTest 0.21.4 and GaiaTest 0.21.8 we are using
> 3.62 MB more on a test that iterates 100 times. I have attached the
> about:memory reports and can't see anything obvious about Marionette so
> don't think its that.

Well, on the latter report, there is 0.73 MB from compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361)) which isn't present in the former, and 0.75 MB from one of the gaia atoms, which also isn't present in the former.  Will be interesting to see if this pattern continues with the 2014-02-04 build.

Comment 32

[David Burns :automatedtester]

Attachment: about-memory-2014-02-04-gaiatest0.21.8.zip

In a run of 30 I managed to get 3.23 MB of injected on 2014-02-04 build with 0.21.8

├──3.23 MB (06.68%) -- notable
│  │  │  │   │  ├──1.54 MB (03.19%) ++ (32 tiny)
│  │  │  │   │  ├──1.13 MB (02.33%) -- string(length=32357, copies=18, "/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaLockScreen = {/n/n  unlock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.unlock(true);/n        waitFor(/n          function() {/n            finish(lockscreen.locked);/n          },/n          function() {/n            return !lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  },/n/n  lock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.lock(true);/n        waitFor(/n          function() {/n            finish(!lockscreen.locked);/n          },/n          function() {/n            return lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  }/n};/n/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaApps = {/n/n  normalizeName: function(name) {/n    return name.replace(/[- ]+/g, '').toLowerCase();/n  },/n/n  getInstalledApps: function() {/n    let req = navigator.mozApps.mgmt.getAll();/n    req.onsuccess = function() {/n      marionetteScriptFinished(req.result);/n    }/n  },/n/n  getRunningApps: function() {/n    let manager = window.wrappedJSObject.AppWindowManager || window.wrappedJSObject.WindowManager;/n    let runningApps = manager.getRunningApps();/n    // Return a simplified version of the runningApps object which can be/n    // JSON-serialized./n    let apps = {};/n    for (let app in runningApps) {/n        let anApp = {};/n        for (let key in runningApps[app]) {/n            if (['name', 'origin', 'manifest'].indexOf(key) > -1) {/n                anApp[key] = runningApps[app][key];/n            }/n        }/n        apps[app] = anApp;/n    }/n    return apps;/n  },/n/n  getRunningAppOrigin: function(name) {/n    let manager = window.wrappedJSObject.AppWindowManager || window.wrappedJSObject.WindowManager;/n    let runningApps = manager.getRunningApps();/n    let origin;/n/n    for (let property in runningApps) {/n      if (runningApps[property].name == name) {/n        origin = property;/n      }/n    }/n/n    return origin;/n  },/n/n  getPermission: function(appName, permissionName) {/n    GaiaApps.locateWithName(appName, function(app) {/n      console.log("Getting permission '" + permissionName + "' for " + appName);/n      var mozPerms = navigator.mozPermissionSettings;/n      var result = mozPerms.get(/n        permissionName, app.manifestURL, app.origin, false/n      );/n      marionetteScriptFinished(result);/n    });/n  },/n/n  setPermission: function(appName, permissionName, value) {/n    GaiaApps.locateWithName(appName, function(app) {/n      console.log("Setting permission '" + permissionName + "' for " +/n        appName + "to '" + value + "'");/n      var mozPerms = navigator.mozPermissionSettings;/n      mozPerms.set(/n        permissionName, value, app.manifestURL, app.origin, false/n      );/n      marionetteScriptFinished();/n    });/n  },/n/n  sendLocateResponse: function(aCallback, app, appName, entryPoint) {/n    var callback = aCallback || marionetteScriptFinished;/n    if (callback === marionetteScriptFinished) {/n " (truncated))
│  │  │  │   │  │  ├──1.13 MB (02.32%) ── malloc-heap
│  │  │  │   │  │  └──0.00 MB (00.00%) ── gc-heap
│  │  │  │   │  └──0.56 MB (01.16%) -- string(length=32526, copies=9, "/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaLockScreen = {/n/n  unlock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.unlock(true);/n        waitFor(/n          function() {/n            finish(lockscreen.locked);/n          },/n          function() {/n            return !lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  },/n/n  lock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.lock(true);/n        waitFor(/n          function() {/n            finish(!lockscreen.locked);/n          },/n          function() {/n            return lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  }/n};/n/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaApps = {/n/n  normalizeName: function(name) {/n    return name.replace(/[- ]+/g, '').toLowerCase();/n  },/n/n  getInstalledApps: function() {/n    let req = navigator.mozApps.mgmt.getAll();/n    req.onsuccess = function() {/n      marionetteScriptFinished(req.result);/n    }/n  },/n/n  getRunningApps: function() {/n    let manager = window.wrappedJSObject.AppWindowManager || window.wrappedJSObject.WindowManager;/n    let runningApps = manager.getRunningApps();/n    // Return a simplified version of the runningApps object which can be/n    // JSON-serialized./n    let apps = {};/n    for (let app in runningApps) {/n        let anApp = {};/n        for (let key in runningApps[app]) {/n            if (['name', 'origin', 'manifest'].indexOf(key) > -1) {/n                anApp[key] = runningApps[app][key];/n            }/n        }/n        apps[app] = anApp;/n    }/n    return apps;/n  },/n/n  getRunningAppOrigin: function(name) {/n    let manager = window.wrappedJSObject.AppWindowManager || window.wrappedJSObject.WindowManager;/n    let runningApps = manager.getRunningApps();/n    let origin;/n/n    for (let property in runningApps) {/n      if (runningApps[property].name == name) {/n        origin = property;/n      }/n    }/n/n    return origin;/n  },/n/n  getPermission: function(appName, permissionName) {/n    GaiaApps.locateWithName(appName, function(app) {/n      console.log("Getting permission '" + permissionName + "' for " + appName);/n      var mozPerms = navigator.mozPermissionSettings;/n      var result = mozPerms.get(/n        permissionName, app.manifestURL, app.origin, false/n      );/n      marionetteScriptFinished(result);/n    });/n  },/n/n  setPermission: function(appName, permissionName, value) {/n    GaiaApps.locateWithName(appName, function(app) {/n      console.log("Setting permission '" + permissionName + "' for " +/n        appName + "to '" + value + "'");/n      var mozPerms = navigator.mozPermissionSettings;/n      mozPerms.set(/n        permissionName, value, app.manifestURL, app.origin, false/n      );/n      marionetteScriptFinished();/n    });/n  },/n/n  sendLocateResponse: function(aCallback, app, appName, entryPoint) {/n    var callback = aCallback || marionetteScriptFinished;/n    if (callback === marionetteScriptFinished) {/n " (truncated))
│  │  │  │   │     ├──0.56 MB (01.16%) ── malloc-heap
│  │  │  │   │     └──0.00 MB (00.00%) ── gc-heap

Comment 33

[David Burns :automatedtester]

Attachment: about-memory-2014-02-04-gaiatest0.21.4.zip

on GaiaTest 0.21.4 We dont seem to have this situation

Comment 34

[David Burns :automatedtester]

The Gaia Atoms appear to be what we are injecting and taking up the memory

Comment 35

[David Burns :automatedtester]

:njn

About:Memory keeps mentioning http://mxr.mozilla.org/mozilla-central/source/testing/marionette/marionette-listener.js#361 in the compartments. Is this just that its doing the work at the moment or do you think that area has a memory leak? Pointers appreciated. If this isn't marionette where do you suggest I look next?

Comment 36

[Robert Wood [:rwood]]

I believe it IS the keyboard afterall that is causing the increase memory use, at least some of it.

While running tests I noticed this: On the Feb-7th build, the keyboard IS showing up unexpectedly during this test. When the camera app is minimized, the homescreen "search" bar is displayed and the cursor is placed inside the search bar which causes the keyboard to pop up briefly until the next test iteration (and the camera app is started again, then the keyboard disappears).

On the Feb-6 build, before the memory increase, the keyboard does NOT pop-up after the camera app is minimized (the camera app is minimized and the search bar appears but the keyboard doesn't pop-up). As mentioned in comment 10, the keyboard was moved from OOP back into the main b2g process at this same time, so I'm guessing that is causing the extra memory use.

Note on the datazilla graph for camera_viewfinder there is a dip in memory use on Feb-27. I verified that in the build used for that test, the keyboard does NOT pop up after minimizing the camera. However for the point in datazilla immediately before that, which is much higher, the keyboard DOES appear after minimizing the camera app.

I'm guessing that explains the main spike. There is another larger spike on the graph for the most recent test run; however we will see after a few more days if that level remains.

Comment 37

[Nicholas Nethercote [inactive]]

> About:Memory keeps mentioning
> http://mxr.mozilla.org/mozilla-central/source/testing/marionette/marionette-
> listener.js#361 in the compartments. Is this just that its doing the work at
> the moment or do you think that area has a memory leak?

I don't know enough about Marionette to make any useful comments. Sorry.

Comment 38

[David Burns :automatedtester]

(In reply to Robert Wood [:rwood] from comment #36)
> I believe it IS the keyboard afterall that is causing the increase memory
> use, at least some of it.
> 
> While running tests I noticed this: On the Feb-7th build, the keyboard IS
> showing up unexpectedly during this test. When the camera app is minimized,
> the homescreen "search" bar is displayed and the cursor is placed inside the
> search bar which causes the keyboard to pop up briefly until the next test
> iteration (and the camera app is started again, then the keyboard
> disappears).
> 
> On the Feb-6 build, before the memory increase, the keyboard does NOT pop-up
> after the camera app is minimized (the camera app is minimized and the
> search bar appears but the keyboard doesn't pop-up). As mentioned in comment
> 10, the keyboard was moved from OOP back into the main b2g process at this
> same time, so I'm guessing that is causing the extra memory use.
> 
> Note on the datazilla graph for camera_viewfinder there is a dip in memory
> use on Feb-27. I verified that in the build used for that test, the keyboard
> does NOT pop up after minimizing the camera. However for the point in
> datazilla immediately before that, which is much higher, the keyboard DOES
> appear after minimizing the camera app.
> 
> I'm guessing that explains the main spike. There is another larger spike on
> the graph for the most recent test run; however we will see after a few more
> days if that level remains.

I wonder if we are dealing with 2 potential leaks here...

Comment 39

[Jonathan Griffin (:jgriffin)]

The only change to the handling of atoms in the relevant timeframe is https://github.com/mozilla-b2g/gaia/commit/7984fc887aeb564238a3fef81df72ad35a3666d8#diff-6d544bbb7c759ffb68f344f348f7740e; there's nothing there that looks like it could possibly be leaky.

The atoms which are leaking are scripts which get executed in a sandbox at http://mxr.mozilla.org/mozilla-central/source/testing/marionette/marionette-listener.js#502, and none of that code changed in this timeframe either.

So, this is pretty puzzling.

Comment 40

[Robert Wood [:rwood]]

Attachment: feb27_about_memory_viewfinder.zip

Now looking at the latest spike (started feb.27) which is consistent across several endurance tests.

feb-27 hamachi central, after camera_viewfinder 25x

46.89 MB (100.0%) -- explicit
├──19.65 MB (41.90%) -- js-non-window
│  ├──11.46 MB (24.44%) -- zones
│  │  ├───9.14 MB (19.50%) -- zone(0x4352f800)
│  │  │   ├──4.20 MB (08.96%) ++ compartment([System Principal])
│  │  │   ├──2.49 MB (05.31%) -- strings
│  │  │   │  ├──1.83 MB (03.90%) -- notable
│  │  │   │  │  ├──1.13 MB (02.40%) ++ (18 tiny)
│  │  │   │  │  └──0.70 MB (01.50%) -- string(length=28946, copies=12, "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHgAAAB4CAYAAAA5ZDbSAAAgAElEQVR4Xu2dB7zcVZ32z7Q7t6SHhCQkGFroWJCqCIoNFRtFFEVZUbGvu+q67qrsrm3FtXd5FRURxbKKWFAXEFQ6SK+BkIT0RnLb3Cnv833OOXPn3iQYpJh3P+/FeO7/P3Nn/vN/zvP7Pb9yzhTC///5X30HCv+rP93//3Dh/1WAC+GEE4rGb9W+f/kzLJg9+pw5Uwv76s9ueWBdq43/nctGf9/apJhxS3zO+ec39f9/+fnbyeT6yzdnO7nQcMI5+1QrhTcXCuEFpWJx16J+yf/0a8fvnNdF62STBxh1WNRcbm7h0xYNVSsUWy0/L2gMzZaGiGGz1eTQx3Ue0kEjtO5sNJo/Hxlpfjmc/5q7t5dbtKXr2P4BPuGEUrHy4o8IzPeUCsViuQNYAB4PrtFNwOoXgwrVDfZWfjrBLQNs+peBzSA3mAQJ5KYmRaPZqjcb9X+rn3fKR7ZXVm/vABeKJ337W6VS6TVtxgq0skB8KHAzcw007NUAOzkf7Xr8MbMTU81kAxsZze/FZgd79XskNyDHf/U0NpqNzze/d8o7tkcmb9cAV0455+3NevhcBrfNXgAW0CWhVSzot0jjZJZ1MploxgxqS89p6UQjfeJS24tG85zBzWzO54qNZojMjf8yyHVRGRb7vMDv7u06beNZJ/6f7Q3k7RLgQ7/9p72LjfDeqy++97XNerNgYItFg1oUkBx3muaSzjX1eKffNWsBVf/qmblmdP7I0edikgGtKF/bCTIk5hwAZp+MqRbeW2RxtaccnnzkLj/Sq3/q9685+I/bC9DbFcBP++YVh7ZKhX8VlC9c/cD6cOefl4ulOkp+t9P/glVZj2X28pym2Nw0kyO4fqxDbPmBzk+cTbKAK4wD2SY6MRtTLX/bweIkuJgAOp9N9b5P3SlMnTmBc3/U8z/6x9cdcuHfGujtAuDDzvrjk0XNj0hDHSOMhEEh3PSn+8KmDcObg7sF8wyrYbDNsf51T+oKk3foDqEkUqZzRlbPG+Vvh8+1HxaTBdbaFUNheFNN/jf5Y4Fos416TuaYpxvUbKY1ATiePrMv7P3UuVmI6yWbV440Wx+48tRDf/O3AvpvCvBBX7x4VrGr56OFYuF1kBG2AXBjuB6u+N3dW2avQCp1KGl+b+KME1N3WjA5zNE/A5sVlugJm9sftgPl7IoBrQBgMsFLblsXli3caJDHm+l6h9hq++EEMG9w+HP30GfQZOP1PAkcWl3YaNT+4ao3HnHn4w303wbgM84oHjjreW8tlQqEPxPtGmEXFlSorFmyIdx+47KHDXDvlGpY8LQd9RoJXL8g2PN/EeQt/SQCO9b170LllkuWi8kjDwtg/n4/mekpMyeOAdhuvNkSmZufWLl06MP3nfHMoccL6Mcd4P0//8e9qpXwDc3yw7jhRbStwU3CSQDfLXCXLd2wVYA3U9CJwTvuPjnMXjDFkwRXDIv9AdsgxxAoAx2TGTwxJjIcIsE6DYtvXBtWL9qEmd2iiXaYNM5E8xrzdpkS5u81C4Mvk44YiJPGkwcz3wx36PVPu/ZNh17+eID8eAJcOPALfzhdYP6X/vVEQRuzTuMBvvJ/7g41mWn86dZE1vgQiedOfcLEMG+/6W22jgcZkw3onT85Y2WQjW/85d7rVoX1Dwy04+FtEVn8bXVCNRx4xC4GuJgUdxboBlvvoddifnzsmhVDZ4Qznll/LIF+XAB+0qcvnhIqXWcrE/USi6gELuNYgFthSALnussWGnT7ZP3rDJPGJzk8ARBZTIZKKex11By9ldQVJjmZ6gyq39Ygx1uKco7yKvnLSPAwPFAPt126LBTqjS2HScS/SVVnBQ1iiDDM+8FH7xEq3WXI7wmSAcbH2wOkc8L5iuHB+ok3/+ORix8rkB9zgA/47B/2E7t+LGD3cPYpARzBTSY0+V9U0dK7V4dFd62yMwY4m/EMchZXD5Ho6J1UDTvtPzX0TZGKzn6X90p3cDyDEUL8xDx0CBvXDIVFMs8j/SNbYW8EaHyiIydDAHT3fWaF2fOnWmB1mmcFY8nPR/HFezcarVXi8yv+/M6nX/xYgPyYArz/py8/plgq/KAUChNgTcpFxHDGQGcAI6vxUddeclcYGWnEE7DSzx2NhfG/D5WqZDLUyW6VY1IkpyvHpykzqDlVaSQazi+Pxr8pFML0tpMcHeztTFdG0x6Z3aWkx0FH7u73HgNw8vHpqWZ2zI61VLsovOWmdx7+tUcb5McM4H0+dfnphULxi8mCtvP/oyZ51FTnc0vuWhkW370m2lEDO2qmt5Ts+EvFhgzqmEJD5ydOMdJoejJlr/T2OfbN4I6vKLXTlAY1pjNzkQJzMH/fHcOc+dNsGqIy7zDP+altgK3leOJ//vnvn/bPyWs8Klg/JgDv+8nL3yf/97GoZuNbZNO8NTP94Lr+cMtVi6KXtP2OIEchNjb2BXhy0OCfwuex5cLkk+MbR8+7pVIhD1NkACxSlrlcSPwb73dkbgR3NEXZ6XcTA9vsbRcs9LZPPHx+6J3UY8s03jwzgZLLthbw++n/ROWzbr7igdPD+SfKjD3yn0cd4H0+cdnHCqXi+0pcdTLBBpoPsBUz3b92INx6rcCl4JrNalZE43xxZ1XJQAOS32dsTdjg8VqQJ02yzkpSNtHtW6i7PRbY+IzRkuGWAY6ToD0D2qGWU51dxbDfwQJ5Ynd7klh08XSLtA7St393Fevcm65YdsqjAfKjCvA+/3npGYVi6UO+2YmBMSfcgVv+XTefD7lq4cpw/z0SVUY/sTcxFzLv2KqFlw3fHvZuLQszw4bQF4ZCV1AI5Xg23iCAjGMHhKJmS7RFSfsx+8MYAzM6d80DnedlanmthiaalTn+mMyazkuXS1jxeDEMNEthuNgTVjQnhDuKc8IF1T3DIh3nODpJZV9cQRN95z13DLPmT3eItrn/TebZVjwqbvvyRuvcW997xKsfqbl+1ADe66OXvluK6MzIpqiWzeBxZtphyIP94cGV65WxWhtqI2KJ7Xf+F/9uz+aD4bO1H4Z9elagPdvmOoPaFHgjyXxmJjSagj351XSfYgjkeyYjmSaEcxsGnZSiwCuWPNm4BJ6PsINe6i7Q2AilshyBnlch+BarKyXBrfOekJSX9Py7BqeH95SPC9eWd/DfZiRjlUrlxGop7DBvWpgwc3LomdQbLYuV9Kh5zkItJ0jk279y2z89482PxFA/bIDnPfFtz2tWq1/1zDfTdHMqXX2F7uoOsZITk/4tnQdo/y6+kS9uiqXDwwo//JxSaqnh9whuLNQXw/NHFoX3Df8w9BaHQ90F3Fi94TcJ3fgjFkXkdJf057F4HwFq/4z7dDw1g5wwDCVfl5jJ+wssGBuJrcmi44LAbcFcjv28kjCth3IJC9AIFeed4yQZDuXwmerLw3937ZbYrEmAtXA2TP94HVO0Gapd5aghLKVjNaug1yulCWF1zwwYqa1vDtc2FPVYIcVdpVr/yfff8JU/bAvwDxvg+Xu+6rz65Fmv4K62cg3Woz6oBZLOA6yrOx1Ato87AR0FNYO8V31d+PD6b8sUD1iZutZr8wsxYrjkWm0biEgYyAW49vWdGHNJ6ZwzW9xI/TGgdY42wzpfB2QmpEHTcT6vsazPw7GgkZPQDTeB0+QQAAi+oWZX+Leprw3XVWZ48mVwDTTPtayO58uxEuFzmelxIthJO05u/70nQTxfXnnHuffd9/OTHwuAi7MWHL+yPHnn6VHhRjC52xHMdMzvmc1cfmanZnl+XnscV6j//Kpzw/zWEivbilhCXAo7gEmZMGRPzDPblI4CoZyTAfEN43q4IRnQNI05zj8wFBDNVEauM/2tlKyZnUeDnsBtg67jRkOME6NHWipKyEurQcsTbXFzRnjbjNfpvF41gcI4BsQEsq+3YyIwCTKQ6AxPhsRcM1ifc+SBG1euWPLbHR91gPdecOKzVzebv6lOmRf9zzaymDttk5xqthncyHomQ0xvvXjwznDa4M8sNspMBn2+isYGszaZUm4wQHjM5ztABZjob+PIe" (truncated))
│  │  │   │  │     ├──0.70 MB (01.50%) ── malloc-heap
│  │  │   │  │     └──0.00 MB (00.00%) ── gc-heap
│  │  │   │  ├──0.58 MB (01.25%) ++ normal
│  │  │   │  └──0.07 MB (00.16%) ── short-gc-heap
│  │  │   ├──0.75 MB (01.60%) ── unused-gc-things
│  │  │   ├──0.67 MB (01.43%) -- compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  │   │  ├──0.27 MB (00.57%) ++ objects
│  │  │   │  ├──0.20 MB (00.43%) ++ scripts
│  │  │   │  ├──0.19 MB (00.40%) ++ shapes
│  │  │   │  ├──0.02 MB (00.03%) ── cross-compartment-wrapper-table
│  │  │   │  └──0.00 MB (00.00%) ── sundries/malloc-heap
│  │  │   ├──0.66 MB (01.40%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │   └──0.37 MB (00.79%) ++ (14 tiny)
│  │  ├───2.11 MB (04.50%) -- zone(0x4352d800)
│  │  │   ├──2.08 MB (04.43%) -- strings
│  │  │   │  ├──1.79 MB (03.83%) -- normal
│  │  │   │  │  ├──1.36 MB (02.91%) ── malloc-heap
│  │  │   │  │  └──0.43 MB (00.92%) ── gc-heap
│  │  │   │  └──0.29 MB (00.61%) ── short-gc-heap
│  │  │   └──0.03 MB (00.06%) ++ (4 tiny)
│  │  └───0.21 MB (00.44%) ++ zone(0x4352f400)
│  ├───8.03 MB (17.13%) -- runtime
│  │   ├──3.90 MB (08.32%) ── script-sources
│  │   ├──2.48 MB (05.29%) ── script-data
│  │   ├──1.08 MB (02.31%) ── atoms-table
│  │   └──0.57 MB (01.21%) ++ (10 tiny)
│  └───0.16 MB (00.33%) ++ gc-heap
├───6.38 MB (13.60%) ── heap-unclassified
├───5.55 MB (11.84%) -- window-objects
│   ├──5.01 MB (10.69%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.97 MB (08.46%) -- active
│   │  │  ├──3.53 MB (07.53%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.48 MB (03.15%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──0.63 MB (01.35%) ++ objects
│   │  │  │  │  ├──0.62 MB (01.32%) ++ shapes
│   │  │  │  │  └──0.23 MB (00.48%) ++ (5 tiny)
│   │  │  │  ├──0.77 MB (01.63%) ++ layout
│   │  │  │  ├──0.69 MB (01.48%) ── style-sheets
│   │  │  │  ├──0.59 MB (01.25%) ++ dom
│   │  │  │  └──0.01 MB (00.02%) ── property-tables
│   │  │  └──0.43 MB (00.92%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──0.85 MB (01.82%) ++ js-zone(0x476ee800)
│   │  └──0.19 MB (00.41%) ++ js-zone(0x4558fc00)
│   └──0.54 MB (01.16%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.46 MB (09.51%) -- images
│   ├──4.46 MB (09.51%) -- content
│   │  ├──4.46 MB (09.51%) -- used
│   │  │  ├──3.87 MB (08.25%) ── uncompressed-heap
│   │  │  ├──0.59 MB (01.26%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───3.21 MB (06.85%) -- heap-overhead
│   ├──2.70 MB (05.75%) ── waste
│   ├──0.51 MB (01.09%) ── page-cache
│   └──0.00 MB (00.01%) ── bookkeeping
├───3.13 MB (06.68%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x455c6400)
│   ├──1.79 MB (03.81%) -- runtime
│   │  ├──0.95 MB (02.02%) ── script-sources
│   │  └──0.84 MB (01.79%) ++ (12 tiny)
│   ├──0.68 MB (01.45%) ++ (3 tiny)
│   └──0.66 MB (01.42%) -- zone(0x457c5c00)
│      ├──0.49 MB (01.05%) ++ compartment(web-worker)
│      └──0.17 MB (00.37%) ++ (4 tiny)
├───1.85 MB (03.94%) ++ (14 tiny)
├───1.27 MB (02.71%) -- storage/sqlite
│   ├──1.08 MB (02.30%) ── other
│   └──0.19 MB (00.41%) ++ (5 tiny)
├───0.87 MB (01.85%) ── xpti-working-set
└───0.52 MB (01.11%) ++ xpconnect

Comment 41

[Robert Wood [:rwood]]

Attachment: mar6_about_memory_viewfinder.zip

mar-6 hamachi central, after camera_viewfinder 25x

61.78 MB (100.0%) -- explicit
├──33.88 MB (54.85%) -- js-non-window
│  ├──19.10 MB (30.92%) -- zones
│  │  ├──16.73 MB (27.09%) -- zone(0x43564800)
│  │  │  ├───6.89 MB (11.15%) -- compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │  │   ├──3.87 MB (06.27%) -- classes
│  │  │  │   │  ├──3.36 MB (05.44%) -- class(<non-notable classes>)
│  │  │  │   │  │  ├──1.76 MB (02.85%) -- objects
│  │  │  │   │  │  │  ├──1.28 MB (02.07%) ── gc-heap [114]
│  │  │  │   │  │  │  └──0.48 MB (00.78%) ++ malloc-heap
│  │  │  │   │  │  └──1.60 MB (02.59%) -- shapes
│  │  │  │   │  │     ├──1.26 MB (02.03%) -- gc-heap
│  │  │  │   │  │     │  ├──0.87 MB (01.41%) ── tree [114]
│  │  │  │   │  │     │  └──0.39 MB (00.62%) ++ (2 tiny)
│  │  │  │   │  │     └──0.35 MB (00.56%) ++ malloc-heap
│  │  │  │   │  └──0.51 MB (00.82%) ++ class(Function)
│  │  │  │   ├──2.02 MB (03.27%) ── scripts/gc-heap [114]
│  │  │  │   └──1.00 MB (01.62%) ── sundries/malloc-heap [114]
│  │  │  ├───4.21 MB (06.81%) -- compartment([System Principal])
│  │  │  │   ├──3.14 MB (05.08%) -- classes
│  │  │  │   │  ├──1.70 MB (02.75%) ++ (13 tiny)
│  │  │  │   │  └──1.44 MB (02.33%) -- class(Object)
│  │  │  │   │     ├──0.89 MB (01.44%) -- objects
│  │  │  │   │     │  ├──0.64 MB (01.04%) ++ malloc-heap
│  │  │  │   │     │  └──0.25 MB (00.41%) ── gc-heap
│  │  │  │   │     └──0.55 MB (00.88%) ++ shapes
│  │  │  │   ├──0.85 MB (01.38%) -- scripts
│  │  │  │   │  ├──0.67 MB (01.09%) ── gc-heap
│  │  │  │   │  └──0.18 MB (00.29%) ── malloc-heap/data
│  │  │  │   └──0.21 MB (00.34%) ++ (4 tiny)
│  │  │  ├───3.61 MB (05.84%) -- strings
│  │  │  │   ├──1.53 MB (02.48%) ++ (16 tiny)
│  │  │  │   ├──1.11 MB (01.80%) -- string(length=28946, copies=19, "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHgAAAB4CAYAAAA5ZDbSAAAgAElEQVR4Xu2dB7zcVZ32z7Q7t6SHhCQkGFroWJCqCIoNFRtFFEVZUbGvu+q67qrsrm3FtXd5FRURxbKKWFAXEFQ6SK+BkIT0RnLb3Cnv833OOXPn3iQYpJh3P+/FeO7/P3Nn/vN/zvP7Pb9yzhTC///5X30HCv+rP93//3Dh/1WAC+GEE4rGb9W+f/kzLJg9+pw5Uwv76s9ueWBdq43/nctGf9/apJhxS3zO+ec39f9/+fnbyeT6yzdnO7nQcMI5+1QrhTcXCuEFpWJx16J+yf/0a8fvnNdF62STBxh1WNRcbm7h0xYNVSsUWy0/L2gMzZaGiGGz1eTQx3Ue0kEjtO5sNJo/Hxlpfjmc/5q7t5dbtKXr2P4BPuGEUrHy4o8IzPeUCsViuQNYAB4PrtFNwOoXgwrVDfZWfjrBLQNs+peBzSA3mAQJ5KYmRaPZqjcb9X+rn3fKR7ZXVm/vABeKJ337W6VS6TVtxgq0skB8KHAzcw007NUAOzkf7Xr8MbMTU81kAxsZze/FZgd79XskNyDHf/U0NpqNzze/d8o7tkcmb9cAV0455+3NevhcBrfNXgAW0CWhVSzot0jjZJZ1MploxgxqS89p6UQjfeJS24tG85zBzWzO54qNZojMjf8yyHVRGRb7vMDv7u06beNZJ/6f7Q3k7RLgQ7/9p72LjfDeqy++97XNerNgYItFg1oUkBx3muaSzjX1eKffNWsBVf/qmblmdP7I0edikgGtKF/bCTIk5hwAZp+MqRbeW2RxtaccnnzkLj/Sq3/q9685+I/bC9DbFcBP++YVh7ZKhX8VlC9c/cD6cOefl4ulOkp+t9P/glVZj2X28pym2Nw0kyO4fqxDbPmBzk+cTbKAK4wD2SY6MRtTLX/bweIkuJgAOp9N9b5P3SlMnTmBc3/U8z/6x9cdcuHfGujtAuDDzvrjk0XNj0hDHSOMh" (truncated))
│  │  │  │   │  ├──1.11 MB (01.80%) ── malloc-heap
│  │  │  │   │  └──0.00 MB (00.00%) ── gc-heap
│  │  │  │   └──0.96 MB (01.56%) -- string(<non-notable strings>)
│  │  │  │      ├──0.73 MB (01.17%) ── malloc-heap
│  │  │  │      └──0.24 MB (00.38%) ── gc-heap
│  │  │  ├───1.10 MB (01.79%) ++ (16 tiny)
│  │  │  └───0.93 MB (01.50%) -- compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  │      ├──0.68 MB (01.10%) ++ classes
│  │  │      └──0.25 MB (00.40%) ++ (4 tiny)
│  │  ├───2.16 MB (03.49%) -- zone(0x43562c00)
│  │  │   ├──2.11 MB (03.42%) -- strings/string(<non-notable strings>)
│  │  │   │  ├──1.38 MB (02.24%) ── malloc-heap
│  │  │   │  └──0.73 MB (01.18%) ── gc-heap
│  │  │   └──0.04 MB (00.07%) ++ (4 tiny)
│  │  └───0.21 MB (00.33%) ++ zone(0x43564400)
│  ├──14.56 MB (23.58%) -- runtime
│  │  ├──10.37 MB (16.79%) -- script-sources
│  │  │  ├───7.14 MB (11.56%) -- source(scripts=115, dummy file)
│  │  │  │   ├──7.14 MB (11.55%) ── uncompressed
│  │  │  │   └──0.01 MB (00.01%) ── misc
│  │  │  ├───2.54 MB (04.11%) ++ (63 tiny)
│  │  │  └───0.69 MB (01.11%) -- source(scripts=500, <non-notable files>)
│  │  │      ├──0.65 MB (01.06%) ── uncompressed
│  │  │      └──0.03 MB (00.05%) ── misc
│  │  ├───2.54 MB (04.12%) ── script-data
│  │  ├───1.08 MB (01.75%) ── atoms-table
│  │  └───0.57 MB (00.92%) ++ (10 tiny)
│  └───0.22 MB (00.35%) ++ gc-heap
├───6.46 MB (10.46%) ── heap-unclassified
├───5.45 MB (08.82%) -- window-objects
│   ├──4.89 MB (07.91%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.91 MB (06.32%) -- active
│   │  │  ├──3.47 MB (05.62%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.44 MB (02.33%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──1.10 MB (01.77%) ++ classes
│   │  │  │  │  └──0.34 MB (00.55%) ++ (6 tiny)
│   │  │  │  ├──0.75 MB (01.21%) ++ layout
│   │  │  │  ├──0.69 MB (01.12%) ── style-sheets
│   │  │  │  └──0.60 MB (00.97%) ++ (2 tiny)
│   │  │  └──0.43 MB (00.70%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──0.79 MB (01.28%) ++ js-zone(0x457fc800)
│   │  └──0.19 MB (00.31%) ++ js-zone(0x45459c00)
│   └──0.56 MB (00.91%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.06 MB (06.57%) -- images
│   ├──4.06 MB (06.57%) -- content
│   │  ├──4.06 MB (06.57%) -- used
│   │  │  ├──3.36 MB (05.44%) ── uncompressed-heap
│   │  │  ├──0.70 MB (01.13%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───3.33 MB (05.39%) -- heap-overhead
│   ├──2.55 MB (04.13%) ── waste
│   ├──0.77 MB (01.25%) ── page-cache
│   └──0.00 MB (00.01%) ── bookkeeping
├───3.24 MB (05.25%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x452c3000)
│   ├──1.79 MB (02.90%) -- runtime
│   │  ├──0.95 MB (01.54%) -- script-sources
│   │  │  ├──0.89 MB (01.45%) -- source(scripts=1, resource://gre/modules/ril_worker.js)
│   │  │  │  ├──0.89 MB (01.45%) ── uncompressed
│   │  │  │  └──0.00 MB (00.00%) ── misc
│   │  │  └──0.06 MB (00.09%) ++ (3 tiny)
│   │  └──0.84 MB (01.36%) ++ (12 tiny)
│   ├──0.73 MB (01.18%) ++ zone(0x454a8000)
│   └──0.72 MB (01.17%) ++ (3 tiny)
├───1.83 MB (02.96%) ++ (14 tiny)
├───1.38 MB (02.24%) -- xpconnect
│   ├──1.01 MB (01.63%) ── proto-iface-cache
│   └──0.37 MB (00.61%) ++ (3 tiny)
├───1.27 MB (02.06%) -- storage/sqlite
│   ├──1.08 MB (01.75%) ── other
│   └──0.19 MB (00.31%) ++ (5 tiny)
└───0.87 MB (01.40%) ── xpti-working-set

Comment 42

[Robert Wood [:rwood]]

The differences in comment 40 and comment 41 are increases in the anonymous-sandbox marionette listener, strings, and runtime script sources.

Comment 43

[Jonathan Griffin (:jgriffin)]

Does this only happen with the camera viewfinder test, and not with other camera tests?  If so, it may help to start tweaking the viewfinder test to see if changes to it make the leak go away...e.g., if the time.sleep(30) is reduced to time.sleep(5), for example (the most obvious difference between this test and other similar ones).

Comment 44

[Robert Wood [:rwood]]

I just finished the same test but for add_contact endurance test, and the same trend shows.

feb-27 hamachi central, after add_contact 25x

45.38 MB (100.0%) -- explicit
├──17.27 MB (38.05%) -- js-non-window
│  ├──10.00 MB (22.04%) -- zones
│  │  ├───7.71 MB (17.00%) -- zone(0x43bcb800)
│  │  │   ├──3.60 MB (07.94%) -- compartment([System Principal])
│  │  │   │  ├──1.52 MB (03.36%) -- objects
│  │  │   │  │  ├──0.99 MB (02.17%) -- gc-heap
│  │  │   │  │  │  ├──0.51 MB (01.12%) ++ (3 tiny)
│  │  │   │  │  │  └──0.47 MB (01.05%) ── ordinary
│  │  │   │  │  ├──0.54 MB (01.18%) ++ malloc-heap
│  │  │   │  │  └──0.00 MB (00.00%) ── non-heap/code/asm.js
│  │  │   │  ├──1.20 MB (02.65%) -- shapes
│  │  │   │  │  ├──0.64 MB (01.40%) ++ gc-heap
│  │  │   │  │  └──0.56 MB (01.24%) ++ malloc-heap
│  │  │   │  ├──0.83 MB (01.82%) -- scripts
│  │  │   │  │  ├──0.65 MB (01.44%) ── gc-heap
│  │  │   │  │  └──0.17 MB (00.38%) ── malloc-heap/data
│  │  │   │  └──0.05 MB (00.12%) ++ (2 tiny)
│  │  │   ├──1.88 MB (04.15%) -- strings
│  │  │   │  ├──1.45 MB (03.19%) -- notable
│  │  │   │  │  ├──1.05 MB (02.32%) -- string(length=28946, copies=18, "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHgAAAB4CAYAAAA5ZDbSAAAgAElEQVR4Xu2dB7zcVZ32z7Q7t6SHhCQkGFroWJCqCIoNFRtFFEVZUbGvu+q67qrsrm3FtXd5FRURxbKKWFAXEFQ6SK+BkIT0RnLb3Cnv833OOXPn3iQYpJh3P+/FeO7/P3Nn/vN/zvP7Pb9yzhTC///5X30HCv+rP93//3Dh/1WAC+GEE4rGb9W+f/kzLJg9+pw5Uwv76s9ueWBdq43/nctGf9/apJhxS3zO+ec39f9/+fnbyeT6yzdnO7nQcMI5+1QrhTcXCuEFpWJx16J+yf/0a8fvnNdF62STBxh1WNRcbm7h0xYNVSsUWy0/L2gMzZaGiGGz1eTQx3Ue0kEjtO5sNJo/Hxlpfjmc/5q7t5dbtKXr2P4BPuGEUrHy4o8IzPeUCsViuQNYAB4PrtFNwOoXgwrVDfZWfjrBLQNs+peBzSA3mAQJ5KYmRaPZqjcb9X+rn3fKR7ZXVm/vABeKJ337W6VS6TVtxgq0skB8KHAzcw007NUAOzkf7Xr8MbMTU81kAxsZze/FZgd79XskNyDHf/U0NpqNzze/d8o7tkcmb9cAV0455+3NevhcBrfNXgAW0CWhVSzot0jjZJZ1MploxgxqS89p6UQjfeJS24tG85zBzWzO54qNZojMjf8yyHVRGRb7vMDv7u06beNZJ/6f7Q3k7RLgQ7/9p72LjfDeqy++97XNerNgYItFg1oUkBx3muaSzjX1eKffNWsBVf/qmblmdP7I0edikgGtKF/bCTIk5hwAZp+MqRbeW2RxtaccnnzkLj/Sq3/q9685+I/bC9DbFcBP++YVh7ZKhX8VlC9c/cD6cOefl4ulOkp+t9P/glVZj2X28pym2Nw0kyO4fqxDbPmBzk+cTbKAK4wD2SY6MRtTLX/bweIkuJgAOp9N9b5P3SlMnTmBc3/U8z/6x9cdcuHfGujtAuDDzvrjk0XNj0hDHSOMhEEh3PSn+8KmDcObg7sF8wyrYbDNsf51T+oKk3foDqEkUqZzRlbPG+Vvh8+1HxaTBdbaFUNheFNN/jf5Y4Fos416TuaYpxvUbKY1ATiePrMv7P3UuVmI6yWbV440Wx+48tRDf/O3AvpvCvBBX7x4VrGr56OFYuF1kBG2AXBjuB6u+N3dW2avQCp1KGl+b+KME1N3WjA5zNE/A5sVlugJm9sftgPl7IoBrQBgMsFLblsXli3caJDHm+l6h9hq++EEMG9w+HP30GfQZOP1PAkcWl3YaNT+4ao3HnHn4w303wbgM84oHjjreW8tlQqEPxPtGmEXFlSorFmyIdx+47KHDXDvlGpY8LQd9RoJXL8g2PN/EeQt/SQCO9b170LllkuWi8kjDwtg/n4/mekpMyeOAdhuvNkSmZufWLl06MP3nfHMoccL6Mcd4P0//8e9qpXwDc3yw7jhRbStwU3CSQDfLXCXLd2wVYA3U9CJwTvuPjnMXjDFkwRXDIv9AdsgxxAoAx2TGTwxJjIcIsE6DYtvXBtWL9qEmd2iiXaYNM5E8xrzdpkS5u81C4Mvk44YiJPGkwcz3wx36PVPu/ZNh17+eID8eAJcOPALfzhdYP6X/vVEQRuzTuMBvvJ/7g41mWn86dZE1vgQiedOfcLEMG+/6W22jgcZkw3onT85Y2WQjW/85d7rVoX1Dwy04+FtEVn8bXVCNRx4xC4GuJgUdxboBlvvoddifnzsmhVDZ4Qznll/LIF+XAB+0qcvnhIqXWcrE/USi6gELuNYgFthSALnussWGnT7ZP3rDJPGJzk8ARBZTIZKKex11By9ldQVJjmZ6gyq39Ygx1uKco7yKvnLSPAwPFAPt126LBTqjS2HScS/SVVnBQ1iiDDM+8FH7xEq3WXI7wmSAcbH2wOkc8L5iuHB+ok3/+ORix8rkB9zgA/47B/2E7t+LGD3cPYpARzBTSY0+V9U0dK7V4dFd62yMwY4m/EMchZXD5Ho6J1UDTvtPzX0TZGKzn6X90p3cDyDEUL8xDx0CBvXDIVFMs8j/SNbYW8EaHyiIydDAHT3fWaF2fOnWmB1mmcFY8nPR/HFezcarVXi8yv+/M6nX/xYgPyYArz/py8/plgq/KAUChNgTcpFxHDGQGcAI6vxUddeclcYGWnEE7DSzx2NhfG/D5WqZDLUyW6VY1IkpyvHpykzqDlVaSQazi+Pxr8pFML0tpMcHeztTFdG0x6Z3aWkx0FH7u73HgNw8vHpqWZ2zI61VLsovOWmdx7+tUcb5McM4H0+dfnphULxi8mCtvP/oyZ51FTnc0vuWhkW370m2lEDO2qmt5Ts+EvFhgzqmEJD5ydOMdJoejJlr/T2OfbN4I6vKLXTlAY1pjNzkQJzMH/fHcOc+dNsGqIy7zDP+altgK3leOJ//vnvn/bPyWs8Klg/JgDv+8nL3yf/97GoZuNbZNO8NTP94Lr+cMtVi6KXtP2OIEchNjb2BXhy0OCfwuex5cLkk+MbR8+7pVIhD1NkACxSlrlcSPwb73dkbgR3NEXZ6XcTA9vsbRcs9LZPPHx+6J3UY8s03jwzgZLLthbw++n/ROWzbr7igdPD+SfKjD3yn0cd4H0+cdnHCqXi+0pcdTLBBpoPsBUz3b92INx6rcCl4JrNalZE43xxZ1XJQAOS32dsTdjg8VqQJ02yzkpSNtHtW6i7PRbY+IzRkuGWAY6ToD0D2qGWU51dxbDfwQJ5Ynd7klh08XSLtA7St393Fevcm65YdsqjAfKjCvA+/3npGYVi6UO+2YmBMSfcgVv+XTefD7lq4cpw/z0SVUY/sTcxFzLv2KqFlw3fHvZuLQszw4bQF4ZCV1AI5Xg23iCAjGMHhKJmS7RFSfsx+8MYAzM6d80DnedlanmthiaalTn+mMyazkuXS1jxeDEMNEthuNgTVjQnhDuKc8IF1T3DIh3nODpJZV9cQRN95z13DLPmT3eItrn/TebZVjwqbvvyRuvcW997xKsfqbl+1ADe66OXvluK6MzIpqiWzeBxZtphyIP94cGV65WxWhtqI2KJ7Xf+F/9uz+aD4bO1H4Z9elagPdvmOoPaFHgjyXxmJjSagj351XSfYgjkeyYjmSaEcxsGnZSiwCuWPNm4BJ6PsINe6i7Q2AilshyBnlch+BarKyXBrfOekJSX9Py7BqeH95SPC9eWd/DfZiRjlUrlxGop7DBvWpgwc3LomdQbLYuV9Kh5zkItJ0jk279y2z89482PxFA/bIDnPfFtz2tWq1/1zDfTdHMqXX2F7uoOsZITk/4tnQdo/y6+kS9uiqXDwwo//JxSaqnh9whuLNQXw/NHFoX3Df8w9BaHQ90F3Fi94TcJ3fgjFkXkdJf057F4HwFq/4z7dDw1g5wwDCVfl5jJ+wssGBuJrcmi44LAbcFcjv28kjCth3IJC9AIFeed4yQZDuXwmerLw3937ZbYrEmAtXA2TP94HVO0Gapd5aghLKVjNaug1yulCWF1zwwYqa1vDtc2FPVYIcVdpVr/yfff8JU/bAvwDxvg+Xu+6rz65Fmv4K62cg3Woz6oBZLOA6yrOx1Ato87AR0FNYO8V31d+PD6b8sUD1iZutZr8wsxYrjkWm0biEgYyAW49vWdGHNJ6ZwzW9xI/TGgdY42wzpfB2QmpEHTcT6vsazPw7GgkZPQDTeB0+QQAAi+oWZX+Leprw3XVWZ48mVwDTTPtayO58uxEuFzmelxIthJO05u/70nQTxfXnnHuffd9/OTHwuAi7MWHL+yPHnn6VHhRjC52xHMdMzvmc1cfmanZnl+XnscV6j//Kpzw/zWEivbilhCXAo7gEmZMGRPzDPblI4CoZyTAfEN43q4IRnQNI05zj8wFBDNVEauM/2tlKyZnUeDnsBtg67jRkOME6NHWipKyEurQcsTbXFzRnjbjNfpvF41gcI4BsQEsq+3YyIwCTKQ6AxPhsRcM1ifc+SBG1euWPLbHR91gPdecOKzVzebv6lOmRf9zzaymDttk5xqthncyHomQ0xvvXjwznDa4M8sNspMBn2+isYGszaZUm4wQHjM5ztABZjob+PIe" (truncated))
│  │  │   │  │  │  ├──1.05 MB (02.32%) ── malloc-heap
│  │  │   │  │  │  └──0.00 MB (00.00%) ── gc-heap
│  │  │   │  │  └──0.39 MB (00.86%) ++ (7 tiny)
│  │  │   │  └──0.44 MB (00.96%) ++ (2 tiny)
│  │  │   ├──1.40 MB (03.09%) ── unused-gc-things
│  │  │   ├──0.48 MB (01.06%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  │   └──0.34 MB (00.76%) ++ (13 tiny)
│  │  ├───2.08 MB (04.59%) -- zone(0x43bc9400)
│  │  │   ├──2.05 MB (04.52%) -- strings
│  │  │   │  ├──1.77 MB (03.89%) -- normal
│  │  │   │  │  ├──1.34 MB (02.96%) ── malloc-heap
│  │  │   │  │  └──0.43 MB (00.94%) ── gc-heap
│  │  │   │  └──0.28 MB (00.62%) ── short-gc-heap
│  │  │   └──0.03 MB (00.07%) ++ (4 tiny)
│  │  └───0.21 MB (00.46%) ++ zone(0x43bcb400)
│  ├───7.09 MB (15.63%) -- runtime
│  │   ├──3.21 MB (07.07%) ── script-sources
│  │   ├──2.41 MB (05.32%) ── script-data
│  │   ├──1.08 MB (02.39%) ── atoms-table
│  │   └──0.39 MB (00.85%) ++ (10 tiny)
│  └───0.17 MB (00.38%) ++ gc-heap
├───6.52 MB (14.37%) ── heap-unclassified
├───5.85 MB (12.89%) -- window-objects
│   ├──5.28 MB (11.65%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──4.10 MB (09.04%) -- active
│   │  │  ├──3.43 MB (07.56%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.39 MB (03.06%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──0.62 MB (01.36%) ++ objects
│   │  │  │  │  ├──0.61 MB (01.33%) ++ shapes
│   │  │  │  │  └──0.16 MB (00.36%) ++ (5 tiny)
│   │  │  │  ├──0.77 MB (01.69%) ++ layout
│   │  │  │  ├──0.69 MB (01.53%) ── style-sheets
│   │  │  │  ├──0.57 MB (01.26%) ++ dom
│   │  │  │  └──0.01 MB (00.02%) ── property-tables
│   │  │  └──0.67 MB (01.49%) -- window(app://keyboard.gaiamobile.org/index.html#en)
│   │  │     ├──0.49 MB (01.08%) ++ js-compartment(app://keyboard.gaiamobile.org/index.html#en)
│   │  │     └──0.18 MB (00.40%) ++ (3 tiny)
│   │  ├──0.97 MB (02.13%) ++ js-zone(0x48afc400)
│   │  └──0.21 MB (00.47%) ++ js-zone(0x475f3000)
│   └──0.57 MB (01.25%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.75 MB (10.46%) -- heap-overhead
│   ├──4.01 MB (08.84%) ── waste
│   ├──0.73 MB (01.61%) ── page-cache
│   └──0.00 MB (00.01%) ── bookkeeping
├───3.13 MB (06.90%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x44ecbc00)
│   ├──1.79 MB (03.94%) -- runtime
│   │  ├──0.95 MB (02.08%) ── script-sources
│   │  └──0.84 MB (01.85%) ++ (12 tiny)
│   ├──0.69 MB (01.51%) ++ (3 tiny)
│   └──0.66 MB (01.45%) -- zone(0x454ba000)
│      ├──0.49 MB (01.08%) ++ compartment(web-worker)
│      └──0.16 MB (00.36%) ++ (4 tiny)
├───3.10 MB (06.83%) -- images
│   ├──3.10 MB (06.83%) -- content
│   │  ├──3.10 MB (06.83%) -- used
│   │  │  ├──2.69 MB (05.93%) ── uncompressed-heap
│   │  │  └──0.41 MB (00.90%) ++ (2 tiny)
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───1.95 MB (04.30%) ++ (14 tiny)
├───1.27 MB (02.80%) -- storage/sqlite
│   ├──1.08 MB (02.39%) ── other
│   └──0.19 MB (00.41%) ++ (5 tiny)
├───0.87 MB (01.91%) ── xpti-working-set
└───0.68 MB (01.49%) ── freetype

Comment 45

[Robert Wood [:rwood]]

mar-6 hamachi central, after add_contact 25x

52.94 MB (100.0%) -- explicit
├──24.85 MB (46.94%) -- js-non-window
│  ├──13.78 MB (26.03%) -- zones
│  │  ├──11.44 MB (21.60%) -- zone(0x43564c00)
│  │  │  ├───3.69 MB (06.96%) -- compartment([System Principal])
│  │  │  │   ├──2.66 MB (05.02%) -- classes
│  │  │  │   │  ├──1.34 MB (02.52%) -- class(Object)
│  │  │  │   │  │  ├──0.80 MB (01.52%) -- objects
│  │  │  │   │  │  │  ├──0.60 MB (01.13%) ++ malloc-heap
│  │  │  │   │  │  │  └──0.21 MB (00.39%) ── gc-heap
│  │  │  │   │  │  └──0.53 MB (01.01%) ++ shapes
│  │  │  │   │  └──1.32 MB (02.49%) ++ (11 tiny)
│  │  │  │   ├──0.85 MB (01.61%) -- scripts
│  │  │  │   │  ├──0.67 MB (01.27%) ── gc-heap
│  │  │  │   │  └──0.18 MB (00.34%) ── malloc-heap/data
│  │  │  │   └──0.18 MB (00.34%) ++ (3 tiny)
│  │  │  ├───3.53 MB (06.66%) -- compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │  │   ├──1.95 MB (03.68%) -- classes
│  │  │  │   │  ├──1.91 MB (03.61%) -- class(<non-notable classes>)
│  │  │  │   │  │  ├──1.05 MB (01.99%) -- objects
│  │  │  │   │  │  │  ├──0.80 MB (01.51%) ── gc-heap [60]
│  │  │  │   │  │  │  └──0.25 MB (00.48%) ++ malloc-heap
│  │  │  │   │  │  └──0.86 MB (01.62%) -- shapes
│  │  │  │   │  │     ├──0.68 MB (01.28%) ++ gc-heap
│  │  │  │   │  │     └──0.18 MB (00.33%) ++ malloc-heap
│  │  │  │   │  └──0.04 MB (00.07%) ++ class(Function)
│  │  │  │   ├──1.06 MB (02.01%) ── scripts/gc-heap [60]
│  │  │  │   └──0.52 MB (00.98%) ── sundries/malloc-heap [60]
│  │  │  ├───2.09 MB (03.94%) -- strings
│  │  │  │   ├──1.11 MB (02.10%) -- string(length=28946, copies=19, "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHgAAAB4CAYAAAA5ZDbSAAAgAElEQVR4Xu2dB7zcVZ32z7Q7t6SHhCQkGFroWJCqCIoNFRtFFEVZUbGvu+q67qrsrm3FtXd5FRURxbKKWFAXEFQ6SK+BkIT0RnLb3Cnv833OOXPn3iQYpJh3P+/FeO7/P3Nn/vN/zvP7Pb9yzhTC///5X30HCv+rP93//3Dh/1WAC+GEE4rGb9W+f/kzLJg9+pw5Uwv76s9ueWBdq43/nctGf9/apJhxS3zO+ec39f9/+fnbyeT6yzdnO7nQcMI5+1QrhTcXCuEFpWJx16J+yf/0a8fvnNdF62STBxh1WNRcbm7h0xYNVSsUWy0/L2gMzZaGiGGz1eTQx3Ue0kEjtO5sNJo/Hxlpfjmc/5q7t5dbtKXr2P4BPuGEUrHy4o8IzPeUCsViuQNYAB4PrtFNwOoXgwrVDfZWfjrBLQNs+peBzSA3mAQJ5KYmRaPZqjcb9X+rn3fKR7ZXVm/vABeKJ337W6VS6TVtxgq0skB8KHAzcw007NUAOzkf7Xr8MbMTU81kAxsZze/FZgd79XskNyDHf/U0NpqNzze/d8o7tkcmb9cAV0455+3NevhcBrfNXgAW0CWhVSzot0jjZJZ1MploxgxqS89p6UQjfeJS24tG85zBzWzO54qNZojMjf8yyHVRGRb7vMDv7u06beNZJ/6f7Q3k7RLgQ7/9p72LjfDeqy++97XNerNgYItFg1oUkBx3muaSzjX1eKffNWsBVf/qmblmdP7I0edikgGtKF/bCTIk5hwAZp+MqRbeW2RxtaccnnzkLj/Sq3/q9685+I/bC9DbFcBP++YVh7ZKhX8VlC9c/cD6cOefl4ulOkp+t9P/glVZj2X28pym2Nw0kyO4fqxDbPmBzk+cTbKAK4wD2SY6MRtTLX/bweIkuJgAOp9N9b5P3SlMnTmBc3/U8z/6x9cdcuHfGujtAuDDzvrjk0XNj0hDHSOMh" (truncated))
│  │  │  │   │  ├──1.11 MB (02.10%) ── malloc-heap
│  │  │  │   │  └──0.00 MB (00.00%) ── gc-heap
│  │  │  │   ├──0.57 MB (01.07%) ++ string(<non-notable strings>)
│  │  │  │   └──0.41 MB (00.77%) ++ (3 tiny)
│  │  │  ├───1.08 MB (02.04%) ── unused-gc-things
│  │  │  ├───0.63 MB (01.19%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  │  └───0.43 MB (00.81%) ++ (13 tiny)
│  │  ├───2.13 MB (04.03%) -- zone(0x43562c00)
│  │  │   ├──2.10 MB (03.97%) -- strings/string(<non-notable strings>)
│  │  │   │  ├──1.38 MB (02.60%) ── malloc-heap
│  │  │   │  └──0.73 MB (01.37%) ── gc-heap
│  │  │   └──0.03 MB (00.06%) ++ (4 tiny)
│  │  └───0.21 MB (00.39%) ++ zone(0x43564800)
│  ├──10.87 MB (20.53%) -- runtime
│  │  ├───6.93 MB (13.08%) -- script-sources
│  │  │   ├──3.77 MB (07.11%) -- source(scripts=61, dummy file)
│  │  │   │  ├──3.76 MB (07.10%) ── uncompressed
│  │  │   │  └──0.00 MB (00.01%) ── misc
│  │  │   ├──2.49 MB (04.70%) ++ (61 tiny)
│  │  │   └──0.67 MB (01.27%) -- source(scripts=386, <non-notable files>)
│  │  │      ├──0.64 MB (01.21%) ── uncompressed
│  │  │      └──0.03 MB (00.05%) ── misc
│  │  ├───2.47 MB (04.67%) ── script-data
│  │  ├───1.08 MB (02.05%) ── atoms-table
│  │  └───0.39 MB (00.73%) ++ (10 tiny)
│  └───0.20 MB (00.38%) ++ gc-heap
├───6.70 MB (12.66%) ── heap-unclassified
├───5.82 MB (11.00%) -- window-objects
│   ├──5.28 MB (09.97%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.97 MB (07.49%) -- active
│   │  │  ├──3.30 MB (06.23%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.28 MB (02.42%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──1.02 MB (01.92%) ++ classes
│   │  │  │  │  └──0.26 MB (00.49%) ++ (4 tiny)
│   │  │  │  ├──0.75 MB (01.42%) ++ layout
│   │  │  │  ├──0.69 MB (01.30%) ── style-sheets
│   │  │  │  ├──0.57 MB (01.08%) ++ dom
│   │  │  │  └──0.01 MB (00.02%) ── property-tables
│   │  │  └──0.67 MB (01.26%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──1.09 MB (02.06%) -- js-zone(0x45ab2000)
│   │  │  ├──0.62 MB (01.16%) ── unused-gc-things
│   │  │  └──0.48 MB (00.90%) ++ (6 tiny)
│   │  └──0.22 MB (00.41%) ++ js-zone(0x460f9c00)
│   └──0.55 MB (01.03%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.47 MB (08.44%) -- heap-overhead
│   ├──3.95 MB (07.47%) ── waste
│   └──0.51 MB (00.97%) ++ (2 tiny)
├───3.30 MB (06.23%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x450f6000)
│   ├──1.92 MB (03.62%) -- runtime
│   │  ├──0.97 MB (01.83%) ++ (12 tiny)
│   │  └──0.95 MB (01.79%) -- script-sources
│   │     ├──0.89 MB (01.69%) -- source(scripts=1, resource://gre/modules/ril_worker.js)
│   │     │  ├──0.89 MB (01.69%) ── uncompressed
│   │     │  └──0.00 MB (00.00%) ── misc
│   │     └──0.06 MB (00.10%) ++ (3 tiny)
│   ├──0.69 MB (01.31%) ++ zone(0x45aa5000)
│   └──0.69 MB (01.30%) ++ (3 tiny)
├───2.52 MB (04.77%) -- images
│   ├──2.52 MB (04.77%) -- content
│   │  ├──2.52 MB (04.77%) -- used
│   │  │  ├──2.13 MB (04.02%) ── uncompressed-heap
│   │  │  └──0.39 MB (00.75%) ++ (2 tiny)
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───1.60 MB (03.03%) ++ (13 tiny)
├───1.27 MB (02.40%) -- storage/sqlite
│   ├──1.08 MB (02.04%) ── other
│   └──0.19 MB (00.35%) ++ (5 tiny)
├───0.87 MB (01.64%) ── xpti-working-set
├───0.86 MB (01.63%) -- xpconnect
│   ├──0.57 MB (01.08%) ── proto-iface-cache
│   └──0.29 MB (00.55%) ++ (3 tiny)
└───0.68 MB (01.28%) ── freetype

Comment 46

[Jonathan Griffin (:jgriffin)]

Can you run this against builds from feb 28 - mar 5, so we can see where the problem pops up?

Comment 47

[Robert Wood [:rwood]]

Attachment: 20140228160203_about_memory_viewfinder.zip.zip

The first build where this memory increase (in marionette listener sandbox, strings, and runtime script sources) first appears is:

https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2g_ril/hamachi-eng-mozilla-central-20140228160203-ril01.02.00.019.102.zip

build id 20140228160203
git commit info 2014-02-28 01:15:20 3e9005ca

This about_memory info can be compared with the output in comment 40 (which was just two builds earlier than this one).

feb-28 hamachi-central, after camera_viewfinder 25x (gaiatest still 0.21.8)

62.76 MB (100.0%) -- explicit
├──35.41 MB (56.41%) -- js-non-window
│  ├──20.65 MB (32.90%) -- zones
│  │  ├──18.33 MB (29.20%) -- zone(0x43ca1800)
│  │  │  ├───6.96 MB (11.09%) -- compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │  │   ├──3.92 MB (06.24%) -- classes
│  │  │  │   │  ├──3.41 MB (05.43%) -- class(<non-notable classes>)
│  │  │  │   │  │  ├──1.79 MB (02.84%) -- objects
│  │  │  │   │  │  │  ├──1.30 MB (02.07%) ── gc-heap [115]
│  │  │  │   │  │  │  └──0.48 MB (00.77%) ++ malloc-heap
│  │  │  │   │  │  └──1.63 MB (02.59%) -- shapes
│  │  │  │   │  │     ├──1.28 MB (02.03%) -- gc-heap
│  │  │  │   │  │     │  ├──0.88 MB (01.41%) ── tree [115]
│  │  │  │   │  │     │  └──0.39 MB (00.62%) ++ (2 tiny)
│  │  │  │   │  │     └──0.35 MB (00.56%) ++ malloc-heap
│  │  │  │   │  └──0.51 MB (00.81%) ++ class(Function)
│  │  │  │   ├──2.04 MB (03.25%) ── scripts/gc-heap [115]
│  │  │  │   └──1.01 MB (01.61%) ── sundries/malloc-heap [115]
│  │  │  ├───5.22 MB (08.32%) -- strings
│  │  │  │   ├──1.76 MB (02.80%) -- string(length=28946, copies=30, "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHgAAAB4CAYAAAA5ZDbSAAAgAElEQVR4Xu2dB7zcVZ32z7Q7t6SHhCQkGFroWJCqCIoNFRtFFEVZUbGvu+q67qrsrm3FtXd5FRURxbKKWFAXEFQ6SK+BkIT0RnLb3Cnv833OOXPn3iQYpJh3P+/FeO7/P3Nn/vN/zvP7Pb9yzhTC///5X30HCv+rP93//3Dh/1WAC+GEE4rGb9W+f/kzLJg9+pw5Uwv76s9ueWBdq43/nctGf9/apJhxS3zO+ec39f9/+fnbyeT6yzdnO7nQcMI5+1QrhTcXCuEFpWJx16J+yf/0a8fvnNdF62STBxh1WNRcbm7h0xYNVSsUWy0/L2gMzZaGiGGz1eTQx3Ue0kEjtO5sNJo/Hxlpfjmc/5q7t5dbtKXr2P4BPuGEUrHy4o8IzPeUCsViuQNYAB4PrtFNwOoXgwrVDfZWfjrBLQNs+peBzSA3mAQJ5KYmRaPZqjcb9X+rn3fKR7ZXVm/vABeKJ337W6VS6TVtxgq0skB8KHAzcw007NUAOzkf7Xr8MbMTU81kAxsZze/FZgd79XskNyDHf/U0NpqNzze/d8o7tkcmb9cAV0455+3NevhcBrfNXgAW0CWhVSzot0jjZJZ1MploxgxqS89p6UQjfeJS24tG85zBzWzO54qNZojMjf8yyHVRGRb7vMDv7u06beNZJ/6f7Q3k7RLgQ7/9p72LjfDeqy++97XNerNgYItFg1oUkBx3muaSzjX1eKffNWsBVf/qmblmdP7I0edikgGtKF/bCTIk5hwAZp+MqRbeW2RxtaccnnzkLj/Sq3/q9685+I/bC9DbFcBP++YVh7ZKhX8VlC9c/cD6cOefl4ulOkp+t9P/glVZj2X28pym2Nw0kyO4fqxDbPmBzk+cTbKAK4wD2SY6MRtTLX/bweIkuJgAOp9N9b5P3SlMnTmBc3/U8z/6x9cdcuHfGujtAuDDzvrjk0XNj0hDHSOMh" (truncated))
│  │  │  │   │  ├──1.76 MB (02.80%) ── malloc-heap
│  │  │  │   │  └──0.00 MB (00.00%) ── gc-heap
│  │  │  │   ├──1.71 MB (02.73%) ++ (18 tiny)
│  │  │  │   ├──1.00 MB (01.60%) -- string(<non-notable strings>)
│  │  │  │   │  ├──0.74 MB (01.18%) ── malloc-heap
│  │  │  │   │  └──0.26 MB (00.42%) ── gc-heap
│  │  │  │   └──0.75 MB (01.20%) -- string(length=32357, copies=12, "/* This Source Code Form is subject to the terms of the Mozilla Public/n * License, v. 2.0. If a copy of the MPL was not distributed with this file,/n * You can obtain one at http://mozilla.org/MPL/2.0/. *//n/n'use strict';/n/nvar GaiaLockScreen = {/n/n  unlock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen;/n    let setlock = window.wrappedJSObject.SettingsListener.getSettingsLock();/n    let obj = {'screen.timeout': 0};/n    setlock.set(obj);/n/n    window.wrappedJSObject.ScreenManager.turnScreenOn();/n/n    waitFor(/n      function() {/n        lockscreen.unlock(true);/n        waitFor(/n          function() {/n            finish(lockscreen.locked);/n          },/n          function() {/n            return !lockscreen.locked;/n          }/n        );/n      },/n      function() {/n        return !!lockscreen;/n      }/n    );/n  },/n/n  lock: function() {/n    let lockscreen = window.wrappedJSObject.lockScreen || window.wrappedJSObject.LockScreen" (truncated))
│  │  │  │      ├──0.75 MB (01.19%) ── malloc-heap
│  │  │  │      └──0.00 MB (00.00%) ── gc-heap
│  │  │  ├───4.17 MB (06.64%) -- compartment([System Principal])
│  │  │  │   ├──3.13 MB (04.98%) -- classes
│  │  │  │   │  ├──1.82 MB (02.90%) ++ (13 tiny)
│  │  │  │   │  └──1.31 MB (02.08%) -- class(Object)
│  │  │  │   │     ├──0.75 MB (01.20%) ++ objects
│  │  │  │   │     └──0.55 MB (00.88%) ++ shapes
│  │  │  │   ├──0.83 MB (01.32%) -- scripts
│  │  │  │   │  ├──0.65 MB (01.04%) ── gc-heap
│  │  │  │   │  └──0.17 MB (00.28%) ── malloc-heap/data
│  │  │  │   └──0.21 MB (00.34%) ++ (4 tiny)
│  │  │  ├───1.07 MB (01.71%) -- compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  │  │   ├──0.80 MB (01.27%) ++ classes
│  │  │  │   └──0.28 MB (00.44%) ++ (4 tiny)
│  │  │  └───0.91 MB (01.44%) ++ (16 tiny)
│  │  ├───2.11 MB (03.37%) -- zone(0x43c9f800)
│  │  │   ├──2.08 MB (03.31%) -- strings/string(<non-notable strings>)
│  │  │   │  ├──1.36 MB (02.17%) ── malloc-heap
│  │  │   │  └──0.72 MB (01.14%) ── gc-heap
│  │  │   └──0.03 MB (00.05%) ++ (4 tiny)
│  │  └───0.21 MB (00.33%) ++ zone(0x43ca1400)
│  ├──14.54 MB (23.16%) -- runtime
│  │  ├──10.41 MB (16.59%) -- script-sources
│  │  │  ├───7.21 MB (11.48%) -- source(scripts=116, dummy file)
│  │  │  │   ├──7.20 MB (11.47%) ── uncompressed
│  │  │  │   └──0.01 MB (00.01%) ── misc
│  │  │  ├───2.52 MB (04.01%) ++ (62 tiny)
│  │  │  └───0.69 MB (01.10%) -- source(scripts=497, <non-notable files>)
│  │  │      ├──0.66 MB (01.05%) ── uncompressed
│  │  │      └──0.03 MB (00.05%) ── misc
│  │  ├───2.47 MB (03.94%) ── script-data
│  │  ├───1.08 MB (01.73%) ── atoms-table
│  │  └───0.57 MB (00.90%) ++ (10 tiny)
│  └───0.22 MB (00.35%) ++ gc-heap
├───6.59 MB (10.50%) ── heap-unclassified
├───5.54 MB (08.83%) -- window-objects
│   ├──4.99 MB (07.96%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.96 MB (06.32%) -- active
│   │  │  ├──3.53 MB (05.63%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.48 MB (02.35%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──1.12 MB (01.78%) ++ classes
│   │  │  │  │  └──0.36 MB (00.57%) ++ (6 tiny)
│   │  │  │  ├──0.76 MB (01.22%) ++ layout
│   │  │  │  ├──0.69 MB (01.10%) ── style-sheets
│   │  │  │  └──0.60 MB (00.95%) ++ (2 tiny)
│   │  │  └──0.43 MB (00.69%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──0.84 MB (01.34%) ++ js-zone(0x48adf400)
│   │  └──0.19 MB (00.30%) ++ js-zone(0x45bf7000)
│   └──0.55 MB (00.87%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.51 MB (07.19%) -- images
│   ├──4.51 MB (07.19%) -- content
│   │  ├──4.51 MB (07.19%) -- used
│   │  │  ├──3.93 MB (06.26%) ── uncompressed-heap
│   │  │  └──0.59 MB (00.93%) ++ (2 tiny)
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───3.31 MB (05.27%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x451efc00)
│   ├──1.92 MB (03.06%) -- runtime
│   │  ├──0.97 MB (01.54%) ++ (12 tiny)
│   │  └──0.95 MB (01.51%) -- script-sources
│   │     ├──0.89 MB (01.43%) -- source(scripts=1, resource://gre/modules/ril_worker.js)
│   │     │  ├──0.89 MB (01.43%) ── uncompressed
│   │     │  └──0.00 MB (00.00%) ── misc
│   │     └──0.06 MB (00.09%) ++ (3 tiny)
│   ├──0.70 MB (01.12%) ++ zone(0x454d8c00)
│   └──0.69 MB (01.09%) ++ (3 tiny)
├───2.03 MB (03.23%) -- heap-overhead
│   ├──1.45 MB (02.31%) ── waste
│   └──0.58 MB (00.92%) ++ (2 tiny)
├───1.83 MB (02.92%) ++ (14 tiny)
├───1.41 MB (02.24%) -- xpconnect
│   ├──1.02 MB (01.62%) ── proto-iface-cache
│   └──0.39 MB (00.62%) ++ (3 tiny)
├───1.27 MB (02.03%) -- storage/sqlite
│   ├──1.08 MB (01.72%) ── other
│   └──0.19 MB (00.30%) ++ (5 tiny)
└───0.87 MB (01.38%) ── xpti-working-set

Comment 48

[Jonathan Griffin (:jgriffin)]

This corresponds to https://hg.mozilla.org/mozilla-central/pushloghtml?startID=26315&endID=26316.

There aren't many gecko commits here (most are gaia commits).  There is a sandbox-related commit, though, https://hg.mozilla.org/mozilla-central/rev/c1a0493fa09e.

Jed, could your patch be causing Marionette to leak strings evaled in a sandbox?

Comment 49

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

(In reply to Jonathan Griffin (:jgriffin) from comment #48)
> Jed, could your patch be causing Marionette to leak strings evaled in a
> sandbox?

Unrelated.  The "sandboxing" affected by that patch is a Linux kernel feature to restrict the system calls that a process can perform; it shares a name with Cu.Sandbox but nothing else, as far as I know.

Comment 50

[Nicholas Nethercote [inactive]]

> The differences in comment 40 and comment 41 are increases in the
> anonymous-sandbox marionette listener, strings, and runtime script sources.

BTW, do you know about about:memory's diff feature? Use the "Load and diff..." button. It might make these comparisons easier to read.

Comment 51

[David Burns :automatedtester]

seemed to have narrowed down to build https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/mozilla-central-hamachi-eng/20140228053232/ which I think relates to http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=5acf5aefe27a which is a merge... 

will carry on bisecting

Comment 52

[Robert Wood [:rwood]]

Just FYI, the same spike is seen and staying consistent for the launch_phone test:

https://datazilla.mozilla.org/b2g/?branch=master&device=hamachi&range=90&test=endurance_launch_phone&app_list=browser,calendar,camera,clock,contacts,fm_radio,gallery,messages,music,phone,settings,usage,video&app=camera&gaia_rev=a71c4bf0a0e8ef94&gecko_rev=7312341e00e27854&plot=avg

Comment 53

[Ben Kelly [:bkelly, not reviewing]]

This appears to be the same problem as bug 982237.  We are leaking the sandboxes themselves:

115 (86.47%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [115]

Let me see if the gc log points at the same issue here.

Comment 54

[Ben Kelly [:bkelly, not reviewing]]

Looks like a different issue.

It seems the window is reference the sandbox the SpecialPower.executeSoon()?

    --[SpecialPowers]--> 0x4426cb50 [Proxy <no private>]
    --[private]--> 0x4a5e6e80 [Object <no private>]
    --[type]--> 0x49523f20 [type_object]
    --[type_proto]--> 0x4a5f3580 [Object <no private>]
    --[type]--> 0x49523bc0 [type_object]
    --[type_proto]--> 0x4a5f3430 [Object <no private>]
    --[executeSoon]--> 0x4a512ec0 [Function SpecialPowersAPI.prototype.executeSoon]
    --[fun_callscope]--> 0x44120280 [ContentFrameMessageManager 443572b0]
    --[sandbox]--> 0x4a58e280 [Proxy <no private>]
    --[private]--> 0x48fc88b0 [Sandbox 4aac7864]

Comment 55

[Ben Kelly [:bkelly, not reviewing]]

Its not immediately obvious where this executeSoon() is being called from, though.

Comment 56

[Jonathan Griffin (:jgriffin)]

(In reply to Ben Kelly [:bkelly] from comment #55)
> Its not immediately obvious where this executeSoon() is being called from,
> though.

That's a mystery.  Neither Marionette nor gaia-ui-tests utilize executeSoon.

Comment 57

[Ben Kelly [:bkelly, not reviewing]]

Hmm, maybe I was looking at the wrong sandbox in the report.  There are a lot of them!

Comment 58

[Ben Kelly [:bkelly, not reviewing]]

I'm sorry, but I don't think we have evidence these are directly related yet.  For one, this is happening in mozilla-center while bug 981871 is in v1.3.

Comment 59

[Ben Kelly [:bkelly, not reviewing]]

So I see a lot of references to sandboxes from console storage via script sources:

via jar:file:///system/b2g/omni.ja!/components/ConsoleAPIStorage.js :
0x49923f00 [script jar:file:///system/b2g/omni.ja!/components/]
    --[objects[2]]--> 0x499442a0 [Function CS_init]
    --[script]--> 0x49948100 [script jar:file:///system/b2g/omni.ja!/components/]
    --[enclosing]--> 0x49944240 [Function]
    --[shape]--> 0x494dd6d0 [shape]
    --[base]--> 0x499471a0 [base_shape]
    --[parent]--> 0x499a0bd0 [FakeBackstagePass <no private>]
    --[NSGetFactory]--> 0x49944580 [Function NSGetFactory]
    --[fun_callscope]--> 0x4992ea90 [Call <no private>]
    --[classes]--> 0x4992eac0 [Object <no private>]
    --[{96cf7855-dfa9-4c6d-8276-f9705b4890f2}]--> 0x4992eb20 [Object <no private>]
    --[createInstance]--> 0x49944560 [Function XPCOMUtils__getFactory/factory.createInst]
    --[fun_callscope]--> 0x4992eaf0 [Call <no private>]
    --[component]--> 0x49944340 [Function ConsoleAPIStorageService]
    --[prototype]--> 0x4992e8b0 [Object <no private>]
    --[clearEvents]--> 0x499444e0 [Function CS_clearEvents]
    --[fun_callscope]--> 0x49929e20 [Call <no private>]
    --[_consoleStorage]--> 0x495d8cc0 [Map 44c51260]
    --[value]--> 0x49929f60 [Array <no private>]
    --[objectElements[25]]--> 0x44196850 [Proxy <no private>]
    --[private]--> 0x4a5ceec0 [Object <no private>]
    --[arguments]--> 0x441967f0 [Array <no private>]
    --[type]--> 0x4a580860 [type_object]
    --[type_proto]--> 0x49bcc6a0 [Array <no private>]
    --[type]--> 0x4a580620 [type_object]
    --[type_proto]--> 0x4a5cea30 [Object <no private>]
    --[constructor]--> 0x4a617ec0 [Function Object]
    --[getOwnPropertyDescriptor]--> 0x4a61c120 [Function getOwnPropertyDescriptor]
    --[type]--> 0x4a580520 [type_object]
    --[type_proto]--> 0x4a617ea0 [Function]
    --[script]--> 0x495c4e80 [script (null):0]
    --[sourceObject]--> 0x49bcc580 [ScriptSource <no private>]
    --[shape]--> 0x499da2f8 [shape]
    --[base]--> 0x4990b858 [base_shape]
    --[parent]--> 0x44120ee0 [Sandbox 44c676c4]

It looks like script sources might have been touched in bug 935203 the day before this bug was opened.

Jim, do you think this could be related to your script source changes?

Comment 60

[Ben Kelly [:bkelly, not reviewing]]

Sorry Jim, I just noticed that the spike in the graph originally was 2/8, although we think that is keyboard.  The new spike is at 2/28 which is a few days after your change.

Comment 61

[Ben Kelly [:bkelly, not reviewing]]

Kyle noted on IRC that bug 965860 landed right before the pushlog in comment 48.  Perhaps its related given the many references we see in the gc/cc logs.

Comment 62

[Ben Kelly [:bkelly, not reviewing]]

David, can you try backing out bug 965860 commits and see if it helps?  Do you have a local build you were bisecting from?

Comment 63

[David Burns :automatedtester]

(In reply to Ben Kelly [:bkelly] from comment #62)
> David, can you try backing out bug 965860 commits and see if it helps?  Do
> you have a local build you were bisecting from?

I will try, I am currently blocked on bug 980899 so can't build anything at the moment for my hamachi, I had been bisecting with tinderbox builds before that.

Comment 64

[Jim Blandy :jimb]

Attachment: In the JS engine, don't retain introduction script data for Debugger.

I saw that the needinfo was cancelled, but I poked around with this a bit anyway; I know you folks are sweating the memory consumption, so I wanted to be sure I wasn't making your life harder.

While it is true that bug 969786 does add some edges to the GC graph, so that the JS engine has a new reason to retain certain data, I believe in most cases that data would have been retained anyway for other reasons. I couldn't figure out how to get graphs out of the graph server, so I did a very rough test: I looked at desktop FF's memory consumption with and without the attached patch, which simply neglects to add that new reference. (Only the debugger ever uses it, and it's optional even there, so omitting it doesn't affect ordinary operation.) I visited cnn.com, arstechinca.com, and boingboing.com. It didn't seem to have any significant effect.

However, I've attached the patch anyway, in case you'd like to try it out with your tests.

Details: when we do an 'eval' or something like that, the newly compiled code now holds a reference to the code that caused it to be compiled. That is, the eval'ed code points back to the eval call.

Comment 65

[Jim Blandy :jimb]

Looking at: https://areweslimyet.com/

It doesn't seem like there was a significant increase in Gecko memory consumption over that period.

Comment 66

[Jonathan Griffin (:jgriffin)]

(In reply to Jim Blandy :jimb from comment #64)

> Details: when we do an 'eval' or something like that, the newly compiled
> code now holds a reference to the code that caused it to be compiled. That
> is, the eval'ed code points back to the eval call.

How does this work with evalInSandbox, which Marionette uses?

Comment 67

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Jim Blandy :jimb from comment #65)
> Looking at: https://areweslimyet.com/
> 
> It doesn't seem like there was a significant increase in Gecko memory
> consumption over that period.

I think we're far enough off the beaten path here with Sandboxes and whatnot that that doesn't say much.

Comment 68

[Jonathan Griffin (:jgriffin)]

(In reply to David Burns :automatedtester from comment #63)
> (In reply to Ben Kelly [:bkelly] from comment #62)
> > David, can you try backing out bug 965860 commits and see if it helps?  Do
> > you have a local build you were bisecting from?
> 
> I will try, I am currently blocked on bug 980899 so can't build anything at
> the moment for my hamachi, I had been bisecting with tinderbox builds before
> that.

I'll make a build and send you the url.

Comment 69

[Jonathan Griffin (:jgriffin)]

(In reply to Jim Blandy :jimb from comment #64)
> Created attachment 8391981 [details] [diff] [review]
> 
> However, I've attached the patch anyway, in case you'd like to try it out
> with your tests.
> 

I've made a build with this patch applied:  http://people.mozilla.org/~jgriffin/hamachi1/

Comment 70

[David Burns :automatedtester]

flashing

unzip gecko.zip, then 'adb remount', then 'adb shell stop b2g', then 'adb push b2g /system/b2g'

then unzip gaia.zip and use adb push to install the files, then 'adb shell start b2g'

Comment 71

[David Burns :automatedtester]

Attachment: jgriffins first hamachi build about-memory

Comment 72

[Jim Blandy :jimb]

(In reply to Jonathan Griffin (:jgriffin) from comment #66)
> How does this work with evalInSandbox, which Marionette uses?

At the moment, xpc::EvalInSandbox does not call setIntroductionInfo on the CompileOptions instance that it passes to JS::Evaluate, so that link, from compiled script to introducing script, is not created for such scripts.

Even if it did, it would only affect storage use if the script calling evalInSandbox would otherwise be garbage. This is usually not the case; for example, if the evalInSandbox call appears in a global function, then the function is holding that script alive anyway.

Comment 73

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #68)
> (In reply to David Burns :automatedtester from comment #63)
> > (In reply to Ben Kelly [:bkelly] from comment #62)
> > > David, can you try backing out bug 965860 commits and see if it helps?  Do
> > > you have a local build you were bisecting from?
> > 
> > I will try, I am currently blocked on bug 980899 so can't build anything at
> > the moment for my hamachi, I had been bisecting with tinderbox builds before
> > that.
> 
> I'll make a build and send you the url.

Here's a build with the console commits backed out: http://people.mozilla.org/~jgriffin/hamachi2/

Comment 74

[David Burns :automatedtester]

Attachment: JGriffin's second hamachi build about:memory

Comment 75

[Jonathan Griffin (:jgriffin)]

build from http://hg.mozilla.org/mozilla-central/rev/1320ac79187f (before jimb's patch):  http://people.mozilla.org/~jgriffin/hamachi/1320ac79187f/

I'll make a post a few other builds in the regression range.

Comment 76

[Jonathan Griffin (:jgriffin)]

build from http://hg.mozilla.org/mozilla-central/rev/0be8765182a3 (after jimb's patch, before the console patch): http://people.mozilla.org/~jgriffin/hamachi/0be8765182a3/

Comment 77

[Robert Wood [:rwood]]

I notice that the lock_screen endurance test doesn't have the memory increase spike which is consistently seen in the other tests. The difference between this test and the others is that the lock_screen test doesn't launch an application. Not sure if that is relevant, but *perhaps* the leak has to do with launching or closing an application.

Comment 78

[Ben Kelly [:bkelly, not reviewing]]

It appears we are still leaking with both Jim's patch and the console changes backed out.

You can quickly see this by looking at the number of compartments:

155 (100.0%) -- js-main-runtime-compartments
├──145 (93.55%) -- user
│  ├──137 (88.39%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [137]

Comment 79

[Ben Kelly [:bkelly, not reviewing]]

This is the reference chain I am seeing now with the memory report in attachment 8392597 [details].  Kyle, do you see anything in here?  Seems to be an observer reference hanging around?

via nsXPCWrappedJS::mJSObj :
0x4ad69e50 [Object <no private>]
    --[shape]--> 0x4ad68b50 [shape]
    --[base]--> 0x4ad6a768 [base_shape]
    --[parent]--> 0x4ad4cee0 [FakeBackstagePass <no private>]
    --[NSGetFactory]--> 0x4ad6c5e0 [Function NSGetFactory]
    --[fun_callscope]--> 0x4ad69d30 [Call <no private>]
    --[classes]--> 0x4ad69d60 [Object <no private>]
    --[{96cf7855-dfa9-4c6d-8276-f9705b4890f2}]--> 0x4ad69dc0 [Object <no private>]
    --[createInstance]--> 0x4ad6c5c0 [Function XPCOMUtils__getFactory/factory.createInst]
    --[fun_callscope]--> 0x4ad69d90 [Call <no private>]
    --[component]--> 0x4ad6c3a0 [Function ConsoleAPIStorageService]
    --[prototype]--> 0x4ad69b50 [Object <no private>]
    --[observe]--> 0x4ad6c4c0 [Function CS_observe]
    --[fun_callscope]--> 0x4ad6b330 [Call <no private>]
    --[_consoleStorage]--> 0x4ad48240 [Map 44453f00]
    --[value]--> 0x4ad6b470 [Array <no private>]
    --[objectElements[196]]--> 0x49132610 [Proxy <no private>]
    --[private]--> 0x44524590 [Object <no private>]
    --[arguments]--> 0x491325e0 [Array <no private>]
    --[type]--> 0x4c151200 [type_object]
    --[type_proto]--> 0x4c11b280 [Array <no private>]
    --[constructor]--> 0x49c5f280 [Function Array]
    --[splice]--> 0x491311f0 [Function splice]
    --[type]--> 0x4c176360 [type_object]
    --[type_proto]--> 0x4c0316a0 [Function]
    --[script]--> 0x4c094b00 [script (null):0]
    --[sourceObject]--> 0x4c11b0d0 [ScriptSource <no private>]
    --[shape]--> 0x46a795e0 [shape]
    --[base]--> 0x4bf23998 [base_shape]
    --[parent]--> 0x4c1523a0 [Sandbox 455e2104]

Comment 80

[Andrew McCreight [:mccr8]]

What is keeping 0x4ad69e50 alive in the CC graph?

Comment 81

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Andrew McCreight [:mccr8] from comment #80)
> What is keeping 0x4ad69e50 alive in the CC graph?

0x44936748 [nsXPCWrappedJS (nsIConsoleAPIStorage)]
    --[mJSObj]--> 0x4ad69e50 [JS Object (Object)]

    Root 0x44936748 is a ref counted object with 1 unknown edge(s).
    known edges:
       0x44936748 [nsXPCWrappedJS (nsIConsoleAPIStorage)] --[self]--> 0x44936748
       0x45bc5800 [Console] --[mStorage]--> 0x44936748

0x449366c8 [nsXPCWrappedJS (nsIObserver)]
    --[mJSObj]--> 0x4ad69e50 [JS Object (Object)]

    Root 0x449366c8 is a ref counted object with 3 unknown edge(s).
    known edges:
       0x44936748 [nsXPCWrappedJS (nsIConsoleAPIStorage)] --[root]--> 0x449366c8
       0x449366c8 [nsXPCWrappedJS (nsIObserver)] --[self]--> 0x449366c8

Comment 82

[Ben Kelly [:bkelly, not reviewing]]

Is this a case of code in the sandboxed test doing something like this?

  var foo = "bar";
  dump(foo);

It appears that |sendConsoleAPIMessage()| stores:

  let consoleEvent = {
    // ...
    arguments: aArgs,
  };

In the console API storage service by calling |recordEvent()|.  This function then places that event object in an array in its |_consoleStorage| map.

Comment 83

[Ben Kelly [:bkelly, not reviewing]]

Or other uses of console where the value passed in is owned by the sandbox.

Comment 84

[Ben Kelly [:bkelly, not reviewing]]

Hmm, maybe something like this:

  getFiles: function(aType, aCallback) {
    // ...
    console.log('getting', aType);
    // ...
  }

From here:

  https://github.com/mozilla-b2g/gaia/blob/master/tests/atoms/gaia_data_layer.js#L423

I guess its not clear to me how to tell what data will belong to the sandbox vs the normal window, etc.

Comment 85

[Ben Kelly [:bkelly, not reviewing]]

That code was changed on Jan 27 in this commit:

  https://github.com/mozilla-b2g/gaia/commit/46dc0aa9c4cf3d2beef0f18f7fcbb96c8dc5ba09

It moved from:

  mediaTypes.forEach(function(aType) {
    console.log('getting', aType);
  });

To literals like:

  function getFiles(aType) {
    console.log('getting', aType);
  }

  getFiles('pictures');

Comment 86

[Ben Kelly [:bkelly, not reviewing]]

That date seems wrong for this bug, though.

This landed on Feb 27, though:

  https://github.com/mozilla-b2g/gaia/commit/e5248fadbc2fbb58fc4e00b1a35b86adbf0062c8#diff-d44c0a8c1ad5aef288a0690ce29b8296

And does something with the "consoleActor".

Seems unlikely its getting run as part of the b2g-endurance tests, though.

Comment 87

[Ben Kelly [:bkelly, not reviewing]]

Have we ruled out a gaia change here?  Can we try rolling back gaia to before Feb 27 and see if the latest memory spike goes away?

Comment 88

[Jonathan Griffin (:jgriffin)]

(In reply to Ben Kelly [:bkelly] from comment #86)
> That date seems wrong for this bug, though.
> 
> This landed on Feb 27, though:
> 
>  
> https://github.com/mozilla-b2g/gaia/commit/
> e5248fadbc2fbb58fc4e00b1a35b86adbf0062c8#diff-
> d44c0a8c1ad5aef288a0690ce29b8296
> 
> And does something with the "consoleActor".
> 
> Seems unlikely its getting run as part of the b2g-endurance tests, though.

It isn't; that's only part of the MarionetteJS tests.

Comment 89

[Jonathan Griffin (:jgriffin)]

(In reply to Robert Wood [:rwood] from comment #77)
> I notice that the lock_screen endurance test doesn't have the memory
> increase spike which is consistently seen in the other tests. The difference
> between this test and the others is that the lock_screen test doesn't launch
> an application. Not sure if that is relevant, but *perhaps* the leak has to
> do with launching or closing an application.

The last time we saw a problem like this it turned out to be a leak in a WebAPI that we used to launch apps...so this is giving me deja vu.  See bug 897684.

One way to help verify this Rob, would be to take one of the simple app launch tests and refactor it so it doesn't use the gaia atoms...instead is just launches the app by clicking on the Nth icon on the homescreen.  This would help us distinguish some problem in the atom vs some internal problem with app launching.

Comment 90

[Jonathan Griffin (:jgriffin)]

The gaia applaunch atom hasn't changed in 2 months, so we can rule out that.

Comment 91

[Robert Wood [:rwood]]

About_memory for camera_viewfinder 25x, hamachi with build from JGriffin (build link in comment 75) gaiatest 0.21.8.

45.19 MB (100.0%) -- explicit
├──17.88 MB (39.57%) -- js-non-window
│  ├──10.14 MB (22.44%) -- zones
│  │  ├───7.92 MB (17.52%) -- zone(0x42aa6400)
│  │  │   ├──3.70 MB (08.18%) -- compartment([System Principal])
│  │  │   │  ├──1.64 MB (03.64%) -- objects
│  │  │   │  │  ├──1.02 MB (02.25%) -- gc-heap
│  │  │   │  │  │  ├──0.55 MB (01.21%) ++ (3 tiny)
│  │  │   │  │  │  └──0.47 MB (01.05%) ── ordinary
│  │  │   │  │  ├──0.63 MB (01.39%) ++ malloc-heap
│  │  │   │  │  └──0.00 MB (00.00%) ── non-heap/code/asm.js
│  │  │   │  ├──1.22 MB (02.70%) -- shapes
│  │  │   │  │  ├──0.63 MB (01.40%) ++ gc-heap
│  │  │   │  │  └──0.59 MB (01.31%) ++ malloc-heap
│  │  │   │  ├──0.77 MB (01.71%) -- scripts
│  │  │   │  │  ├──0.61 MB (01.35%) ── gc-heap
│  │  │   │  │  └──0.16 MB (00.35%) ── malloc-heap/data
│  │  │   │  └──0.06 MB (00.13%) ++ (2 tiny)
│  │  │   ├──1.86 MB (04.11%) -- strings
│  │  │   │  ├──1.36 MB (03.01%) ++ notable
│  │  │   │  └──0.49 MB (01.09%) ++ (2 tiny)
│  │  │   ├──1.00 MB (02.21%) ── unused-gc-things
│  │  │   ├──0.71 MB (01.57%) ++ (15 tiny)
│  │  │   └──0.66 MB (01.46%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  ├───2.02 MB (04.46%) -- zone(0x42aa4800)
│  │  │   ├──1.98 MB (04.38%) -- strings
│  │  │   │  ├──1.70 MB (03.77%) -- normal
│  │  │   │  │  ├──1.29 MB (02.85%) ── malloc-heap
│  │  │   │  │  └──0.41 MB (00.91%) ── gc-heap
│  │  │   │  └──0.28 MB (00.62%) ── short-gc-heap
│  │  │   └──0.04 MB (00.08%) ++ (4 tiny)
│  │  └───0.21 MB (00.46%) ++ zone(0x42aa6000)
│  ├───7.59 MB (16.79%) -- runtime
│  │   ├──3.65 MB (08.07%) ── script-sources
│  │   ├──2.35 MB (05.21%) ── script-data
│  │   ├──1.08 MB (02.40%) ── atoms-table
│  │   └──0.51 MB (01.12%) ++ (10 tiny)
│  └───0.16 MB (00.35%) ++ gc-heap
├───6.26 MB (13.85%) ── heap-unclassified
├───5.04 MB (11.15%) -- window-objects
│   ├──4.48 MB (09.92%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.52 MB (07.79%) -- active
│   │  │  ├──3.14 MB (06.95%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.32 MB (02.93%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──0.60 MB (01.32%) ++ shapes
│   │  │  │  │  ├──0.56 MB (01.24%) ++ objects
│   │  │  │  │  └──0.17 MB (00.37%) ++ (5 tiny)
│   │  │  │  ├──0.77 MB (01.69%) ++ layout
│   │  │  │  ├──0.67 MB (01.48%) ── style-sheets
│   │  │  │  └──0.38 MB (00.85%) ++ (2 tiny)
│   │  │  └──0.38 MB (00.85%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──0.81 MB (01.80%) ++ js-zone(0x479ca800)
│   │  └──0.15 MB (00.33%) ++ js-zone(0x4030c400)
│   └──0.56 MB (01.23%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.49 MB (09.94%) -- images
│   ├──4.49 MB (09.94%) -- content
│   │  ├──4.49 MB (09.94%) -- used
│   │  │  ├──3.87 MB (08.56%) ── uncompressed-heap
│   │  │  ├──0.62 MB (01.37%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───3.43 MB (07.60%) -- heap-overhead
│   ├──3.01 MB (06.66%) ── waste
│   └──0.42 MB (00.93%) ++ (2 tiny)
├───2.95 MB (06.54%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x4519bc00)
│   ├──1.72 MB (03.82%) -- runtime
│   │  ├──0.95 MB (02.09%) ── script-sources
│   │  └──0.78 MB (01.72%) ++ (12 tiny)
│   ├──0.66 MB (01.45%) ++ (3 tiny)
│   └──0.57 MB (01.27%) -- zone(0x456cac00)
│      ├──0.48 MB (01.07%) ++ compartment(web-worker)
│      └──0.09 MB (00.20%) ++ (4 tiny)
├───2.24 MB (04.95%) ++ (15 tiny)
├───1.28 MB (02.84%) -- storage/sqlite
│   ├──1.08 MB (02.40%) ── other
│   └──0.20 MB (00.45%) ++ (5 tiny)
├───0.87 MB (01.92%) ── xpti-working-set
└───0.74 MB (01.64%) -- dmd
    ├──0.49 MB (01.09%) ++ stack-traces
    └──0.25 MB (00.55%) ── block-table

Comment 92

[Ben Kelly [:bkelly, not reviewing]]

Just to follow-up on comment 82, it would be interesting to try a test that converts these:

tests/atoms/gaia_data_layer.js:      console.error('error saving contact', req.error.name);
tests/atoms/gaia_data_layer.js:      console.error('error finding contacts', req.error.name);
tests/atoms/gaia_data_layer.js:      console.error('error finding contacts', req.error.name);
tests/atoms/gaia_data_layer.js:      console.log('error getting setting', req.error.name);
tests/atoms/gaia_data_layer.js:      console.log('error changing setting', req.error.name);
tests/atoms/gaia_data_layer.js:        console.log('error connecting to network', req.error.name);
tests/atoms/gaia_data_layer.js:      console.log('error getting known networks', req.error.name);
tests/atoms/gaia_data_layer.js:    console.log('getting', aType);
tests/atoms/gaia_data_layer.js:      console.error('failed to enumerate ' + aType, req.error.name);

To use string concatenation instead of passing as a raw argument value.  So:

  console.log('getting ' + aType);

Instead of:

  console.log('getting', aType);

Jonathon, is this something you could try?  I know we still don't know what changed, but the gc/cc logs seem to suggest that this could be related.

Comment 93

[Robert Wood [:rwood]]

About_memory for camera_viewfinder 25x, hamachi with build from JGriffin (build link in comment 76) gaiatest 0.21.8.

47.55 MB (100.0%) -- explicit
├──19.70 MB (41.43%) -- js-non-window
│  ├──11.32 MB (23.81%) -- zones
│  │  ├───9.07 MB (19.07%) -- zone(0x42ab1800)
│  │  │   ├──4.08 MB (08.57%) -- compartment([System Principal])
│  │  │   │  ├──1.96 MB (04.13%) -- objects
│  │  │   │  │  ├──1.16 MB (02.44%) -- gc-heap
│  │  │   │  │  │  ├──0.59 MB (01.25%) ++ (3 tiny)
│  │  │   │  │  │  └──0.57 MB (01.19%) ── ordinary
│  │  │   │  │  ├──0.80 MB (01.69%) ++ malloc-heap
│  │  │   │  │  └──0.00 MB (00.00%) ── non-heap/code/asm.js
│  │  │   │  ├──1.27 MB (02.68%) -- shapes
│  │  │   │  │  ├──0.68 MB (01.43%) ++ gc-heap
│  │  │   │  │  └──0.60 MB (01.26%) ++ malloc-heap
│  │  │   │  ├──0.77 MB (01.62%) -- scripts
│  │  │   │  │  ├──0.61 MB (01.28%) ── gc-heap
│  │  │   │  │  └──0.16 MB (00.34%) ── malloc-heap/data
│  │  │   │  └──0.07 MB (00.15%) ++ (3 tiny)
│  │  │   ├──2.37 MB (04.98%) -- strings
│  │  │   │  ├──1.69 MB (03.54%) ++ notable
│  │  │   │  ├──0.62 MB (01.30%) -- normal
│  │  │   │  │  ├──0.50 MB (01.05%) ── malloc-heap
│  │  │   │  │  └──0.12 MB (00.25%) ── gc-heap
│  │  │   │  └──0.06 MB (00.14%) ── short-gc-heap
│  │  │   ├──0.90 MB (01.89%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │   ├──0.69 MB (01.44%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  │   ├──0.65 MB (01.36%) ── unused-gc-things
│  │  │   └──0.40 MB (00.83%) ++ (15 tiny)
│  │  ├───2.05 MB (04.30%) -- zone(0x42aafc00)
│  │  │   ├──1.98 MB (04.16%) -- strings
│  │  │   │  ├──1.70 MB (03.57%) -- normal
│  │  │   │  │  ├──1.29 MB (02.71%) ── malloc-heap
│  │  │   │  │  └──0.41 MB (00.87%) ── gc-heap
│  │  │   │  └──0.28 MB (00.59%) ── short-gc-heap
│  │  │   └──0.07 MB (00.14%) ++ (4 tiny)
│  │  └───0.21 MB (00.43%) ++ zone(0x42ab1400)
│  ├───8.22 MB (17.29%) -- runtime
│  │   ├──4.21 MB (08.85%) ── script-sources
│  │   ├──2.36 MB (04.97%) ── script-data
│  │   ├──1.08 MB (02.28%) ── atoms-table
│  │   └──0.57 MB (01.19%) ++ (10 tiny)
│  └───0.16 MB (00.33%) ++ gc-heap
├───6.69 MB (14.08%) ── heap-unclassified
├───5.19 MB (10.91%) -- window-objects
│   ├──4.64 MB (09.77%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.68 MB (07.74%) -- active
│   │  │  ├──3.30 MB (06.94%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.47 MB (03.10%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──0.61 MB (01.29%) ++ objects
│   │  │  │  │  ├──0.61 MB (01.27%) ++ shapes
│   │  │  │  │  └──0.25 MB (00.53%) ++ (5 tiny)
│   │  │  │  ├──0.77 MB (01.61%) ++ layout
│   │  │  │  ├──0.67 MB (01.40%) ── style-sheets
│   │  │  │  └──0.40 MB (00.84%) ++ (2 tiny)
│   │  │  └──0.38 MB (00.80%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──0.82 MB (01.73%) ++ js-zone(0x494fb400)
│   │  └──0.14 MB (00.30%) ++ js-zone(0x45e77400)
│   └──0.54 MB (01.15%) ++ top(chrome://b2g/content/shell.html, id=1)
├───4.65 MB (09.78%) -- images
│   ├──4.65 MB (09.78%) -- content
│   │  ├──4.65 MB (09.78%) -- used
│   │  │  ├──3.87 MB (08.14%) ── uncompressed-heap
│   │  │  ├──0.78 MB (01.65%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───3.09 MB (06.50%) -- heap-overhead
│   ├──2.50 MB (05.26%) ── waste
│   ├──0.59 MB (01.23%) ── page-cache
│   └──0.00 MB (00.01%) ── bookkeeping
├───2.96 MB (06.22%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x452f8000)
│   ├──1.73 MB (03.64%) -- runtime
│   │  ├──0.95 MB (02.00%) ── script-sources
│   │  └──0.78 MB (01.64%) ++ (12 tiny)
│   ├──0.66 MB (01.38%) ++ (3 tiny)
│   └──0.57 MB (01.21%) -- zone(0x452cf000)
│      ├──0.48 MB (01.02%) ++ compartment(web-worker)
│      └──0.09 MB (00.19%) ++ (4 tiny)
├───1.84 MB (03.87%) ++ (14 tiny)
├───1.28 MB (02.70%) -- storage/sqlite
│   ├──1.08 MB (02.28%) ── other
│   └──0.20 MB (00.42%) ++ (5 tiny)
├───0.87 MB (01.82%) ── xpti-working-set
├───0.73 MB (01.53%) -- dmd
│   ├──0.48 MB (01.00%) ++ stack-traces
│   └──0.25 MB (00.53%) ── block-table
└───0.54 MB (01.14%) ++ xpconnect

Comment 94

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Bug 985827 will probably help here too, and with anything that uses marionette.

Comment 95

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #94)
> Bug 985827 will probably help here too, and with anything that uses
> marionette.

I checked the logs and we only have something like 32 SettingsManagers.  So its happening a bit, but not to the degree we are leaking sandboxes.

Comment 96

[Zac C (:zac)]

(In reply to Ben Kelly [:bkelly] from comment #92)
> To use string concatenation instead of passing as a raw argument value.  So:
> 
>   console.log('getting ' + aType);
> 
> Instead of:
> 
>   console.log('getting', aType);
> 
> Jonathon, is this something you could try?  I know we still don't know what
> changed, but the gc/cc logs seem to suggest that this could be related.

I'm not in a position to (easily) run this test scenario but this is a change I can make on master asap as a matter of good practice if you think worthwhile?

Comment 97

[Jonathan Griffin (:jgriffin)]

Rob's latest two reports make it seem pretty likely that jimb's patch is the culprit, although the size of the sandbox leak isn't as large as some of the later reports.  I'll make another build to verify.

Meanwhile, Rob, can you make the changes to gaiatest in your virtualenv indicated in comment #92 and run the tests again?

Comment 98

[Jonathan Griffin (:jgriffin)]

Zac, yes, I think that would help, since the leak is also affecting the latest master.

Comment 99

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Jonathan Griffin (:jgriffin) from comment #97)
> Rob's latest two reports make it seem pretty likely that jimb's patch is the
> culprit, although the size of the sandbox leak isn't as large as some of the
> later reports.  I'll make another build to verify.

I'm sorry, I'm not following how those reports show a significant improvement.  Can you explain what numbers you are comparing?

Comment 100

[Jonathan Griffin (:jgriffin)]

The latter has

│  │  │   ├──0.90 MB (01.89%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))

whereas the former doesn't have this.  It's definitely not as much as some of the other reports, so I'm not quite sure whether this is exhibiting the leak or not.

Comment 101

[Ben Kelly [:bkelly, not reviewing]]

I think you need to really look in the "Other Measurements" section of the report for something like this:

158 (100.0%) -- js-main-runtime-compartments
├──148 (93.67%) -- user
│  ├──139 (87.97%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [139]
│  └────9 (05.70%) ++ (9 tiny)

Which shows the exact count of the compartments.  I don't see that part included in comment 91 or comment 93, though.

Comment 102

[Robert Wood [:rwood]]

In response to comment 89, I modified the camera_viewfinder test so it was semi-automated; instead of having the test use the gaiatest atoms camera_app.launch() and gaiatest.close_app() to start and shutdown the camera app, I ran the endurance test but manually physically clicked the hamachi screen to start and shutdown the camera app at the appropriate place during the test. Used the hamachi build where the leak was first seen (link in comment 47), 25x, same gaiatest 0.21.8.

Result:

45.41 MB (100.0%) -- explicit
├──17.55 MB (38.66%) -- js-non-window
│  ├───9.49 MB (20.90%) -- zones
│  │   ├──7.20 MB (15.85%) -- zone(0x403bbc00)
│  │   │  ├──3.51 MB (07.73%) -- compartment([System Principal])
│  │   │  │  ├──2.52 MB (05.56%) -- classes
│  │   │  │  │  ├──1.10 MB (02.42%) -- class(Object)
│  │   │  │  │  │  ├──0.56 MB (01.24%) ++ objects
│  │   │  │  │  │  └──0.53 MB (01.18%) ++ shapes
│  │   │  │  │  ├──0.97 MB (02.13%) ++ (8 tiny)
│  │   │  │  │  └──0.46 MB (01.01%) ++ class(Function)
│  │   │  │  ├──0.82 MB (01.80%) -- scripts
│  │   │  │  │  ├──0.65 MB (01.42%) ── gc-heap
│  │   │  │  │  └──0.17 MB (00.38%) ── malloc-heap/data
│  │   │  │  └──0.17 MB (00.38%) ++ (4 tiny)
│  │   │  ├──1.15 MB (02.53%) -- strings
│  │   │  │  ├──0.68 MB (01.49%) ++ (9 tiny)
│  │   │  │  └──0.48 MB (01.05%) ++ string(<non-notable strings>)
│  │   │  ├──0.98 MB (02.16%) ── unused-gc-things
│  │   │  ├──0.65 MB (01.43%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │   │  ├──0.53 MB (01.17%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │   │  └──0.37 MB (00.82%) ++ (14 tiny)
....
28 (100.0%) -- js-main-runtime-compartments
├──19 (67.86%) -- user
│  ├──11 (39.29%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [11]

With the same setup but running the test fully automated (using the gaitest atoms camera_app.launch and gaiatest.close_app), this is the result:

59.13 MB (100.0%) -- explicit
├──31.67 MB (53.56%) -- js-non-window
│  ├──17.04 MB (28.81%) -- zones
│  │  ├──14.73 MB (24.91%) -- zone(0x43368c00)
│  │  │  ├───6.82 MB (11.54%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
....
131 (100.0%) -- js-main-runtime-compartments
├──122 (93.13%) -- user
│  ├──113 (86.26%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [113]
│  └────9 (06.87%) ++ (9 tiny)
└────9 (06.87%) ++ system

So looks like the memory leak is associated with the gaiatest atoms used for launching and closing the app.

(Even though the version of gaiatest itself did not change when this memory leak first occurred, so perhaps in some code the atoms are calling but not the actual atoms themselves).

Comment 103

[Jonathan Griffin (:jgriffin)]

Rob, here's another build:  http://people.mozilla.org/~jgriffin/hamachi/ec85e0c7c060/

This is the same as the one from comment #93, along with the console changes.

Comment 104

[Robert Wood [:rwood]]

Thanks Jonathan. The leak does show up with that build. Result:

58.74 MB (100.0%) -- explicit
├──30.86 MB (52.53%) -- js-non-window
│  ├──16.36 MB (27.84%) -- zones
│  │  ├──14.10 MB (24.01%) -- zone(0x42aaec00)
│  │  │  ├───6.82 MB (11.61%) ++ compartment(app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361))
│  │  │  ├───3.42 MB (05.83%) -- compartment([System Principal])
│  │  │  │   ├──1.45 MB (02.46%) -- objects
│  │  │  │   │  ├──0.89 MB (01.51%) ++ gc-heap
│  │  │  │   │  └──0.56 MB (00.95%) ++ (2 tiny)
....
130 (100.0%) -- js-main-runtime-compartments
├──121 (93.08%) -- user
│  ├──113 (86.92%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [113]
│  └────8 (06.15%) ++ (8 tiny)
└────9 (06.92%) ++ system

Comment 105

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Robert Wood [:rwood] from comment #104)
> Thanks Jonathan. The leak does show up with that build. Result:

:-(

Can I get a full memory report with gc/cc logs?

Comment 106

[Jonathan Griffin (:jgriffin)]

The only difference between comment #104 and comment #93 is the commits in bug 965860.

Comment 107

[Ben Kelly [:bkelly, not reviewing]]

Oh, I thought "the console changes" were the replacement of console.log(a, b) with console.log(a + b);

Comment 108

[Jonathan Griffin (:jgriffin)]

No, we don't have data for that scenario, yet.

Comment 109

[Jim Blandy :jimb]

I'm not sure I'm following the back-and-forth here; but if the build linked in comment 69 doesn't have the leak, then we can be sure that it's the introduction script retention that's at fault. That's an easy fix.

Comment 110

[Robert Wood [:rwood]]

Attachment: about_memory_concatenation.zip

Re: comment 92

I made the concatenation changes as suggested in comment 92 (and made sure all the console lines used concatenation in all the /tests/atoms). Then ran the test again - but the memory leak still appeared. Attaching the about_memory for that test run, showing the memory leak still there.

However: I then just commented out all of the 'console.log' and 'console.error' code from the /tests/atoms, ran the test again, and the memory leak disappeared. So it looks like it is related to the console code. I put the console lines back in, and ran the test again just to verify, and the leak was back as expected.

Used the hamachi build from comment 47, camera_viewfinder 25x.

This is the good result with the 'console.log' and 'console.error' lines commented out of the gaia/tests/atoms:

46.23 MB (100.0%) -- explicit
├──18.36 MB (39.72%) -- js-non-window
│  ├──10.41 MB (22.53%) -- zones
│  │  ├───8.09 MB (17.49%) -- zone(0x43567c00)
│  │  │   ├──3.49 MB (07.55%) -- compartment([System Principal])
│  │  │   │  ├──2.51 MB (05.42%) -- classes
│  │  │   │  │  ├──1.15 MB (02.49%) -- class(Object)
│  │  │   │  │  │  ├──0.58 MB (01.26%) ++ objects
│  │  │   │  │  │  └──0.57 MB (01.23%) ++ shapes
│  │  │   │  │  ├──0.88 MB (01.90%) ++ (9 tiny)
│  │  │   │  │  └──0.48 MB (01.03%) ++ class(Function)
│  │  │   │  ├──0.83 MB (01.80%) -- scripts
│  │  │   │  │  ├──0.66 MB (01.42%) ── gc-heap
│  │  │   │  │  └──0.17 MB (00.38%) ── malloc-heap/data
│  │  │   │  └──0.15 MB (00.33%) ++ (3 tiny)
│  │  │   ├──2.20 MB (04.76%) -- strings
│  │  │   │  ├──1.63 MB (03.53%) ++ (9 tiny)
│  │  │   │  └──0.57 MB (01.23%) ++ string(<non-notable strings>)
│  │  │   ├──1.02 MB (02.22%) ── unused-gc-things
│  │  │   ├──0.71 MB (01.54%) ++ (16 tiny)
│  │  │   └──0.66 MB (01.42%) ++ compartment([System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html)
│  │  ├───2.12 MB (04.59%) -- zone(0x43565800)
│  │  │   ├──2.07 MB (04.49%) -- strings/string(<non-notable strings>)
│  │  │   │  ├──1.36 MB (02.94%) ── malloc-heap
│  │  │   │  └──0.72 MB (01.55%) ── gc-heap
│  │  │   └──0.05 MB (00.10%) ++ (4 tiny)
│  │  └───0.21 MB (00.45%) ++ zone(0x43567800)
│  ├───7.81 MB (16.89%) -- runtime
│  │   ├──3.61 MB (07.81%) -- script-sources
│  │   │  ├──2.93 MB (06.33%) ++ (63 tiny)
│  │   │  └──0.68 MB (01.48%) -- source(scripts=279, <non-notable files>)
│  │   │     ├──0.66 MB (01.42%) ── uncompressed
│  │   │     └──0.03 MB (00.06%) ── misc
│  │   ├──2.47 MB (05.34%) ── script-data
│  │   ├──1.08 MB (02.34%) ── atoms-table
│  │   └──0.65 MB (01.40%) ++ (10 tiny)
│  └───0.14 MB (00.30%) ++ gc-heap
├───6.57 MB (14.21%) ── heap-unclassified
├───5.50 MB (11.90%) -- window-objects
│   ├──4.96 MB (10.73%) -- top(app://system.gaiamobile.org/index.html, id=3)
│   │  ├──3.93 MB (08.49%) -- active
│   │  │  ├──3.49 MB (07.56%) -- window(app://system.gaiamobile.org/index.html)
│   │  │  │  ├──1.41 MB (03.05%) -- js-compartment(app://system.gaiamobile.org/index.html)
│   │  │  │  │  ├──1.08 MB (02.34%) ++ classes
│   │  │  │  │  └──0.33 MB (00.71%) ++ (6 tiny)
│   │  │  │  ├──0.76 MB (01.65%) ++ layout
│   │  │  │  ├──0.69 MB (01.49%) ── style-sheets
│   │  │  │  ├──0.62 MB (01.34%) ++ dom
│   │  │  │  └──0.01 MB (00.02%) ── property-tables
│   │  │  └──0.43 MB (00.94%) ++ window(app://keyboard.gaiamobile.org/index.html#en)
│   │  ├──0.85 MB (01.83%) ++ js-zone(0x480dec00)
│   │  └──0.19 MB (00.41%) ++ js-zone(0x45737400)
│   └──0.54 MB (01.16%) ++ top(chrome://b2g/content/shell.html, id=1)
├───5.05 MB (10.93%) -- images
│   ├──5.05 MB (10.93%) -- content
│   │  ├──5.05 MB (10.93%) -- used
│   │  │  ├──3.93 MB (08.49%) ── uncompressed-heap
│   │  │  ├──1.12 MB (02.43%) ── raw
│   │  │  └──0.00 MB (00.00%) ── uncompressed-nonheap
│   │  └──0.00 MB (00.00%) ++ unused
│   └──0.00 MB (00.00%) ++ chrome
├───3.25 MB (07.03%) -- heap-overhead
│   ├──2.89 MB (06.26%) ── waste
│   └──0.36 MB (00.78%) ++ (2 tiny)
├───3.10 MB (06.71%) -- workers/workers()/worker(resource://gre/modules/ril_worker.js, 0x454ea000)
│   ├──1.79 MB (03.88%) -- runtime
│   │  ├──0.95 MB (02.05%) -- script-sources
│   │  │  ├──0.89 MB (01.94%) -- source(scripts=1, resource://gre/modules/ril_worker.js)
│   │  │  │  ├──0.89 MB (01.93%) ── uncompressed
│   │  │  │  └──0.00 MB (00.00%) ── misc
│   │  │  └──0.06 MB (00.12%) ++ (3 tiny)
│   │  └──0.84 MB (01.82%) ++ (12 tiny)
│   ├──0.66 MB (01.43%) ++ (3 tiny)
│   └──0.65 MB (01.40%) -- zone(0x456e3000)
│      ├──0.49 MB (01.07%) ++ compartment(web-worker)
│      └──0.16 MB (00.34%) ++ (5 tiny)
├───2.24 MB (04.85%) ++ (15 tiny)
├───1.28 MB (02.78%) -- storage/sqlite
│   ├──1.08 MB (02.34%) ── other
│   └──0.20 MB (00.44%) ++ (5 tiny)
└───0.87 MB (01.88%) ── xpti-working-set
....
24 (100.0%) -- js-main-runtime-compartments
├──15 (62.50%) -- user
│  ├───6 (25.00%) ── app://system.gaiamobile.org/index.html, [anonymous sandbox] (from: chrome://marionette/content/marionette-listener.js:361) [6]
│  ├───1 (04.17%) ── app://keyboard.gaiamobile.org/index.html#en
│  ├───1 (04.17%) ── app://system.gaiamobile.org/index.html
│  ├───1 (04.17%) ── moz-nullprincipal:{216c82d6-34a3-432c-b70b-f058ce0e5fd7}
│  ├───1 (04.17%) ── moz-nullprincipal:{2f69c59b-3ed3-4343-aca6-efeb6193f4b1}
│  ├───1 (04.17%) ── moz-nullprincipal:{47ece69c-9b17-47d0-8be7-aa92be40c86f}
│  ├───1 (04.17%) ── moz-nullprincipal:{9ed562b5-c9d0-48a7-82f5-555c5056846d}
│  ├───1 (04.17%) ── moz-nullprincipal:{b2da115a-591e-4a82-a774-546a69037dcc}
│  ├───1 (04.17%) ── moz-nullprincipal:{cdcf9a45-9a2a-498d-b41b-24f948ea98a8}
│  └───1 (04.17%) ── moz-nullprincipal:{dbde1572-0f6e-49b8-9ee4-b6b4c4410a72}
└───9 (37.50%) -- system
    ├──1 (04.17%) ── [System Principal]
    ├──1 (04.17%) ── [System Principal], XPConnect Junk Compartment
    ├──1 (04.17%) ── [System Principal], [anonymous sandbox] (from: chrome://marionette/content/marionette-server.js:653)
    ├──1 (04.17%) ── [System Principal], about:blank
    ├──1 (04.17%) ── [System Principal], chrome://global/content/bindings/scrollbar.xml
    ├──1 (04.17%) ── [System Principal], inProcessTabChildGlobal?ownedBy=app://system.gaiamobile.org/index.html
    ├──1 (04.17%) ── [System Principal], inProcessTabChildGlobal?ownedBy=chrome://b2g/content/shell.html
    ├──1 (04.17%) ── atoms
    └──1 (04.17%) ── null-principal

Comment 111

[Jonathan Griffin (:jgriffin)]

(In reply to Jim Blandy :jimb from comment #109)
> I'm not sure I'm following the back-and-forth here; but if the build linked
> in comment 69 doesn't have the leak, then we can be sure that it's the
> introduction script retention that's at fault. That's an easy fix.

Nope, it seems we've narrowed it down to bug 965860 (the console changes).

Comment 112

[Ben Kelly [:bkelly, not reviewing]]

Err... I thought we tried backing out bug 965860 and the leak persisted.  Hmm, but now I don't see the about-memory for the build in comment 76.

Comment 113

[Jonathan Griffin (:jgriffin)]

I did, but the backout wasn't clean, and it's possible I did something wrong with the merges.

Comment 114

[Ben Kelly [:bkelly, not reviewing]]

I guess it seems possible something else changed that is effecting the console path.  For example, maybe the magic javascript |arguments| array is getting owned by the sandbox now where previously it did not.

Comment 115

[Jonathan Griffin (:jgriffin)]

Is there anything else you'd like from us to be able to resolve this?

Comment 116

[Ben Kelly [:bkelly, not reviewing]]

Andrea, have you had any luck investigating this?  I seem to recall you said you had some cycles to investigate console's impact here.  Thanks!

Comment 117

[Andrea Marchesini [:baku]]

Yep. So in order to reproduce this issue, should I just open and close the camera app X times, correct?

Comment 118

[Ben Kelly [:bkelly, not reviewing]]

I think you actually need to run the b2g-endurance tests as its an interaction with the marionette sandbox.  I think rwood should be able to point you towards instructions on running the tests.  (You hopefully don't have to wait for them to complete as they are quite long...)

Comment 119

[Robert Wood [:rwood]]

Hi Andrea, more info about the endurance tests, including how to run them, can be found here:

https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance

Note that the endurance tests currently run on hamachi master with v1.3 (not on 1.1 as it says in MDN; I need to update that).

You can run any of these endurance tests to demonstrate the leak:

https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/tests/endurance/test_endurance_camera_viewfinder.py

https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/tests/endurance/test_endurance_gallery_flick.py

https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/tests/endurance/test_endurance_launch_phone.py

For this issue, I was mainly running camera_viewfinder, 25 iterations, and grabbing the about_memory after.

Thanks, and if you have any questions feel free to ping me in #ateam

Comment 120

[Robert Wood [:rwood]]

Correction - just hamachi *master* not 1.3 :)

Comment 121

[Jonathan Griffin (:jgriffin)]

(In reply to Andrea Marchesini (:baku) from comment #117)
> Yep. So in order to reproduce this issue, should I just open and close the
> camera app X times, correct?

Any luck in reproducing this?  Or, has this problem disappeared?

Comment 122

[Robert Wood [:rwood]]

The endurance tests on hamachi have been offline for over the past week because of build issues, so unfortunately there's no recent data. The leak is still showing consistently in the data prior to that.

Comment 123

[Ben Kelly [:bkelly, not reviewing]]

Andrea, I have time to look at this now.  I assume you've been busy with other work here, so just stealing the bug.  Feel free to steal back if you are currently working it. :-)

Comment 124

[Ben Kelly [:bkelly, not reviewing]]

Reproduced locally, let me see what I can find.

Comment 125

[Ben Kelly [:bkelly, not reviewing]]

Hmm, now I'm not seeing the leak.  Perhaps I had an old gecko on the device last night when I got the leak to show.  Will investigate more, but this may have gotten fixed somewhere along the line in gecko.

Comment 126

[Ben Kelly [:bkelly, not reviewing]]

So it leaks in b2g30_v1_4 and doesn't leak in today's mozilla-central.  This is just a few days, so I'm going to bisect it.  I'm curious what got fixed.

Comment 127

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Ben Kelly [:bkelly] from comment #126)
> So it leaks in b2g30_v1_4 and doesn't leak in today's mozilla-central.  This
> is just a few days, so I'm going to bisect it.  I'm curious what got fixed.

Err... b2g30_v1_4 came from mozilla-aurora, not mozilla-central.  So make the 7 weeks of changes.  Still might be worth knowing the fix, though, in case it should be uplifted.

Comment 128

[Ben Kelly [:bkelly, not reviewing]]

Well, I'm giving up on finding which revision fixed this.  I tried bisecting but there is a large swath of revs where the test basically fails to open the app.  Given that the current head has no leak I don't think its worth doing that much archaeology here.