1516493 - AWSY base is not correctly avoiding measuring the preallocated content process

Status: RESOLVED FIXED

(Testing :: AWSY, defect)

Description

[Cameron McCormack (:heycam)]

In test_base_memory_usage.py is this:

  # We don't want to measure the preallocated process, so we load enough
  # tabs so that it is no longer launched.
  process_count = self.marionette.get_pref('dom.ipc.processCount')
  self._urls = ['about:blank'] * process_count

In the logs of the awsy-base jobs on the build machines is this:

  TEST-START | awsy/test_base_memory_usage.py TestMemoryUsage.test_open_tabs
  areweslimyet run by 4 pages, 1 iterations, 10 perTabPause, 60 settleWaitTime
  setting up!
  areweslimyet run by 4 pages, 1 iterations, 10 perTabPause, 60 settleWaitTime, 8 content processes
  done setting up!
  loading about:blank
  loaded!
  switching to tab
  switched to tab
  loading about:blank
  loaded!
  switching to tab
  switched to tab
  loading about:blank
  loaded!
  switching to tab
  switched to tab
  loading about:blank
  loaded!
  starting gc...
  gc done!
  starting checkpoint TabsOpenForceGC...
  checkpoint created, stored in /builds/worker/workspace/build/tests/results/memory-report-TabsOpenForceGC-0.json.gz

So we only ever load 4 about:blank pages, even though dom.ipc.processCount is 8.  I think this is due to the

  "entities": 4,

line in base-testvars.json.  The result is that the preallocated process gets included in the mean memory measurements, and it skews the test results lower, since in that process various things (such as all of the user agent style sheets) have not been loaded, as no document has been loaded in it.

But also, when I open the memory report file that is generated, I see that three of the about:blank pages have been loaded into the Privileged Content process, and only one in a non-preallocated content process.  So the mean is calculated only of the preallocated content process (that doesn't have about:blank loaded in it) and one regular content process (that does).

I'm not sure what the rules are for which process gets chosen to load a given URL, but I guess we need to load different kinds of URLs to cause them to open in different content processes.  Maybe we can load empty pages from https: URLs from different origins?

For the purpose of avoiding the preallocated process, we should probably just set dom.ipc.processPrelaunch.enabled to false in base-prefs.json.

Comment 1

[Kris Maglione [:kmag]]

(In reply to Cameron McCormack (:heycam) (away Jan 1) from comment #0)
> So we only ever load 4 about:blank pages, even though dom.ipc.processCount
> is 8.  I think this is due to the
> 
>   "entities": 4,
> 
> line in base-testvars.json.  The result is that the preallocated process
> gets included in the mean memory measurements, and it skews the test results
> lower, since in that process various things (such as all of the user agent
> style sheets) have not been loaded, as no document has been loaded in it.

Ideally, this wouldn't matter much, since we currently use the median value for all content processes, which would tend to ignore the pre-allocated content process. But given that most of our about:blank pages are currently loaded in the privileged content process, we probably wind up taking the average of that one real content process and the pre-allocated process, at the moment.

Either way, yeah, it would be great to avoid this situation.

> But also, when I open the memory report file that is generated, I see that
> three of the about:blank pages have been loaded into the Privileged Content
> process, and only one in a non-preallocated content process.  So the mean is
> calculated only of the preallocated content process (that doesn't have
> about:blank loaded in it) and one regular content process (that does).

This is bug 1515201.

> For the purpose of avoiding the preallocated process, we should probably just set dom.ipc.processPrelaunch.enabled to false in base-prefs.json.

I'm kind of on the fence about this. It sounds good in theory, but it's very possible that content processes that started off pre-allocated wind up using a different amount of memory than those that don't. Not that we adequately control for that at the moment, anyway...

But it may make more sense to just release the pre-allocated process before taking measurements: https://searchfox.org/mozilla-central/rev/d6f2d4bb73446534a9231faefae05934fcd05911/dom/chrome-webidl/MessageManager.webidl#547-551

Comment 2

[Cameron McCormack (:heycam)]

(In reply to Kris Maglione [:kmag] from comment #1)
> > But also, when I open the memory report file that is generated, I see that
> > three of the about:blank pages have been loaded into the Privileged Content
> > process, and only one in a non-preallocated content process.  So the mean is
> > calculated only of the preallocated content process (that doesn't have
> > about:blank loaded in it) and one regular content process (that does).
> 
> This is bug 1515201.

OK, that eliminates the need for the patch I was just working on. :-)

> I'm kind of on the fence about this. It sounds good in theory, but it's very
> possible that content processes that started off pre-allocated wind up using
> a different amount of memory than those that don't. Not that we adequately
> control for that at the moment, anyway...

That's a fair concern.

> But it may make more sense to just release the pre-allocated process before
> taking measurements:
> https://searchfox.org/mozilla-central/rev/
> d6f2d4bb73446534a9231faefae05934fcd05911/dom/chrome-webidl/MessageManager.
> webidl#547-551

That sounds good.

For a patch here then I'll trying calling that function to drop the preallocated process, and adjusting test_base_memory_usage.py to just take the number of pages from process_count.

Comment 3

[Cameron McCormack (:heycam)]

Actually I don't think it's the retained content processes that's the problem here (since at the time we take the measurement we haven't closed any tabs), but the process managed by PreallocatedProcessManager.  Easiest thing to do seems to be to flip the pref to false near the end of the test, which should cause it to drop that process.

Comment 4

[Cameron McCormack (:heycam)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1295516e3026de8d9e2a35bec83a0cca251fdf7d

Comment 5

[Cameron McCormack (:heycam)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a1fe8be4b62ba7262ac81b88a56b49bc5b334ede

Comment 6

[Kris Maglione [:kmag]]

(In reply to Cameron McCormack (:heycam) (away Jan 1) from comment #3)
> Actually I don't think it's the retained content processes that's the
> problem here (since at the time we take the measurement we haven't closed
> any tabs), but the process managed by PreallocatedProcessManager.  Easiest
> thing to do seems to be to flip the pref to false near the end of the test,
> which should cause it to drop that process.

Hmm. I thought it also released the preallocated process. Oh well. Flipping the pref should be fine.

Comment 7

[Cameron McCormack (:heycam)]

Attachment: Bug 1516493 - Use process count for AWSY page load count and explicitly drop the preallocated process

Comment 8

[Cameron McCormack (:heycam)]

I'll land this before bug 1515201.  The "regressions" that mconley identified in that bug will likely show up as a result of this change.  Then when bug 1515201 lands it should hopefully have the effect of making the numbers less noisy.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a791cbab57a3
Use process count for AWSY page load count and explicitly drop the preallocated process r=kmag

Comment 10

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/a791cbab57a3