1149755 - DevHud App Memory: Relying on sizeOfTab does not give accurate memory measurements for b2g apps

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Developer Tools, defect)

Description

[:Harald Kirschner :digitarald]

Reported values are not accurate according to erahm as sizeOfTab doesn't report correctly on b2g.

Comment 1

[:Harald Kirschner :digitarald]

Background: I am using the devhud adb logcat output to show real-time memory graphs: https://github.com/digitarald/fxos-remote-dev-hud

ni? :erahm to add his findings and :janx to be aware.

Comment 2

[Eric Rahm [:erahm]]

I should note the values aren't incorrect, they're just incomplete. They don't give an accurate reflection of the overall memory usage for the entire process.

As in, the value returned in the |jsStringsSize| param is almost certainly not an accurate representation of the actual memory used by all JS strings in the app process. We can take a look at an about:memory report for a j2me app as an example:

> 57.32 MB (100.0%) -- explicit
> ├──35.95 MB (62.72%) -- window-objects/top(app://j2mejs.mozilla.org/index.html, id=1)
> │  ├──28.08 MB (48.98%) -- active
> │  │  ├──27.71 MB (48.33%) -- window(app://j2mejs.mozilla.org/main.html)
> │  │  └───0.37 MB (00.64%) ++ window(app://j2mejs.mozilla.org/index.html)
> │  └───7.88 MB (13.74%) -- js-zone(0xb3527800)
> │      ├──3.99 MB (06.95%) -- strings
> │      │  ├──3.60 MB (06.28%) -- string(<non-notable strings>)
> │      │  └──0.39 MB (00.68%) ++ (17 tiny)
> │      ├──3.31 MB (05.77%) ── unused-gc-things
> │      └──0.58 MB (01.02%) ++ (5 tiny)
> ├───7.08 MB (12.36%) -- js-non-window
> │   ├──3.95 MB (06.89%) -- zones
> │   │  ├──1.92 MB (03.35%) -- zone(0xb6a57000)
> │   │  │  ├──1.67 MB (02.91%) ++ strings/string(<non-notable strings>)
> │   │  │  └──0.25 MB (00.44%) ++ (4 tiny)

So we can see there are 3.99MiB of strings that are roughly associated w/ the window (probably the 'tab' in this case, as you see there are 2 windows so there's an additional possibility that we're skipping one) and 1.67MiB of strings associated w/ the non-window zone, which I believe is not included in the |sizeOfTab| calculations (I need to verify this assertion, the person who wrote the code is away for a month).

Now I could be wrong about that specifically, but the overall point stands that this doesn't given an accurate representation of memory use of the *entire* process (something like 'explicit' from about:memory is probably a better measure, or the JS summary in the 'other' category).

Comment 3

[Jordan Santell [:jsantell] [@jsantell]]

To clear up some things:

* Is the DevHud App using the memory actor?
* is the Memory Reporter Manager's `resident`[0] attribute what you're looking for for this? Wouldn't this value be incompatible with all the other size values reported by sizeOfTab?

[0] https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIMemoryReporterManager#Attributes

Comment 4

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to Jordan Santell [:jsantell] [@jsantell] from comment #3)
> To clear up some things:
> 
> * Is the DevHud App using the memory actor?

Yes, the HUD[1] uses the memory actor.

[1]: https://dxr.mozilla.org/mozilla-central/source/b2g/chrome/content/devtools/hud.js#484

Comment 5

[Jordan Santell [:jsantell] [@jsantell]]

How can we get the same information as sizeOfTab on b2g? Is there some memory module for that?

Comment 6

[Jan Keromnes [:janx]]

Thanks for reporting this, Harald!

Yes, Eric is right, the "App memory" value is incomplete because sizeOfTab is itself incomplete. The option is only there because it was the only way to show some sort of deep-dive into the memory components (even if incomplete, it might help to quickly notice big leaks in JS strings / DOM / other). I know "App memory" has helped find at least one major leak in the most popular Firefox OS app to date, ConnectA2.

However, "App memory" is not the recommended way to look at memory, instead one should use the complete aggregate process memory called "Unique set size", the first available option, as documented under "Memory" in [1].

In general, I feel like this is a problem with sizeOfTab. We should either fix sizeOfTab to be more complete and relevant, or make the Developer HUD use something better for detailed memory footprints.

Harald, I don't think you're looking for Jordan's suggested Resident Set Size, because the Unique Set Size you're importing at [2] is already a good indicator of high-level process memory. Rather, for a complete deep-dive into the different components of a memory footprint, I see two potential options:

- You could grab and parse an about:memory dump from the device, using something like the B2G/tools/get_about_memory.py script. [3]

- We could use the new devtools memory inspection tools. I believe they're still under construction. Nick, what's the status on these? Could the Developer HUD use them now to display a more complete set of components in an app's memory footprint? This would be helpful to both Harald's graphing tool and regular Developer HUD users.

Jordan, I'm not sure what you're looking for with sizeOfTab on b2g, so I'll let Nicholas reply to your needinfo.

[1] Developer HUD documentation: https://developer.mozilla.org/en-US/Firefox_OS/Debugging/Developer_settings#Developer_HUD

[2] Memory events supported by Harald's Remote Dev HUD: https://github.com/digitarald/fxos-remote-dev-hud/blob/master/server.js#L193

[3] B2G's get_about_memory.py tool: https://github.com/mozilla-b2g/B2G/blob/master/tools/get_about_memory.py

Comment 7

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Jan Keromnes [:janx] from comment #6)
> - We could use the new devtools memory inspection tools. I believe they're
> still under construction. Nick, what's the status on these? Could the
> Developer HUD use them now to display a more complete set of components in
> an app's memory footprint? This would be helpful to both Harald's graphing
> tool and regular Developer HUD users.

We are waiting to ship a dedicated memory tool until heap snapshots and a few platform features are implemented.

https://bugzilla.mozilla.org/show_bug.cgi?id=961331 is a good bug to track for memory + devtools platform features.

Comment 8

[Jan Keromnes [:janx]]

Thanks Nick! Added as a dependency to this bug.

Comment 9

[Nicholas Nethercote [inactive]]

(In reply to Jordan Santell [:jsantell] [@jsantell] from comment #5)
> How can we get the same information as sizeOfTab on b2g? Is there some
> memory module for that?

sizeOfTab can be used in B2G. But as noted above it's an approximation.

Comment 10

[BMO Automation]

Firefox OS is not being worked on