1108624 - Preallocated Process runs with OOM_ADJ of 1

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Performance, defect, P3)

Description

[Michael Vines [:m1] [:evilmachines]]

STR:
1) Run |adb shell b2g-ps --oom|

Note the "(Preallocated a" process running with OOM_ADJ of 1.  The foreground application typically runs with OOM_ADJ of 2, so the LMK will likely kill the foreground application before the preallocated process.  This seems backwards as the preallocated process is just an optimization to improve app launch time and should be killed if the foreground app needs the memory.


eg:
root@flame-kk:/ # b2g-ps --oom                                                 
APPLICATION    SEC  OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS     WCHAN    PC         NAME
b2g              0     0        48          0          root      204   1     243396 88152 ffffffff b6eae8ac S /system/b2g/b2g
(Nuwa)           0     -16      1           -941       root      504   204   54820  14308 ffffffff b6eae8ac S /system/b2g/b2g
Built-in Keyboa  2     11       754         734        u0_a986   986   504   72420  22940 ffffffff b6eae8ac S /system/b2g/b2g
Homescreen       2     2        166         134        u0_a1163  1163  204   83436  36504 ffffffff b6f528ac S /system/b2g/plugin-container
Usage            2     11       754         734        u0_a1224  1224  504   65560  23624 ffffffff b6eae8ac S /system/b2g/b2g
Find My Device   2     10       687         667        u0_a1427  1427  504   65092  23012 ffffffff b6eae8ac S /system/b2g/b2g
(Preallocated a  2     1        84          67         u0_a5358  5358  504   62788  19564 ffffffff b6eae8ac S /system/b2g/b2g

Comment 1

[Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me)]

any ideas on it ?

Comment 2

[Cervantes Yu [:cyu] [:cervantes]]

We did it in bug 959966. The rationale behind it is that we try to reduce the memory footprint of the preallocated process and keep it alive. If it is killed by the LMK, app launch time will suffer. Also because currently we fork from the chrome process instead of Nuwa, when there is no preallocated process available, the overall memory consumption will also suffer.

Currently the preallocated process consumes 4-5 MB of memory, but after preloading has landed, we will reduce it to ~3MB (see https://bugzilla.mozilla.org/show_bug.cgi?id=970307#c3 ). The current difference of an empty app w/wo the preallocated process is ~4MB. Here are 2 b2g-info entries the of the Template app launched w/wo the preallocated process. They has 4.4 MB of USS difference:

Template 3138 2490    0.7    1  6.5  9.0 22.0  0.0  72.8       2 u0_a3138
Template 3382 2480    1.1    1 10.9 14.8 28.9  0.0  79.6       2 u0_a3382

Some apps show an even larger difference in USS (5.7 MB for the dialer, 10 MB for the homescreen).

Therefore, I think the reason we prevent the preallocated process from being killed is still valid, and we are not going to change this behavior.

Comment 3

[Michael Vines [:m1] [:evilmachines]]

Thanks for the history Cevantes.  

The particular situation that I'm trying to avoid is when all background apps have been killed, and the foreground app still needs more memory.   At this point the kernel can either kill the foreground app or the pre-allocated process.  Right now it's the foreground app that will be killed, and I'm proposing we try to reverse that.

I understand that the next app launch time may suffer as a result, but at least the user didn't lose their foreground app mysteriously.

> Also because currently we fork from the chrome process instead of Nuwa, when there 
> is no preallocated process available, the overall memory consumption will also suffer.

That's definitely a problem.  In this situation why can't we tell Nuwa to fork a new preallocated process instead of forking from the chrome process?  It seems to me on a low memory device we would never want to fork from the chrome process for this reason (even at the expense of a slower launch time).

Comment 4

[Cervantes Yu [:cyu] [:cervantes]]

(In reply to Michael Vines [:m1] [:evilmachines] from comment #3)
> Thanks for the history Cevantes.  
> 
> The particular situation that I'm trying to avoid is when all background
> apps have been killed, and the foreground app still needs more memory.   At
> this point the kernel can either kill the foreground app or the
> pre-allocated process.  Right now it's the foreground app that will be
> killed, and I'm proposing we try to reverse that.
> 
Is this a real case that the memory taken by the preallocated process makes a difference?
If the foreground app consumes so much memory that we need to sacrifice the preallocated process, how much longer can the app remain alive?

> I understand that the next app launch time may suffer as a result, but at
> least the user didn't lose their foreground app mysteriously.
> 
> > Also because currently we fork from the chrome process instead of Nuwa, when there 
> > is no preallocated process available, the overall memory consumption will also suffer.
> 
> That's definitely a problem.  In this situation why can't we tell Nuwa to
> fork a new preallocated process instead of forking from the chrome process? 
> It seems to me on a low memory device we would never want to fork from the
> chrome process for this reason (even at the expense of a slower launch time).

We have bug 1053634 for this issue.

Comment 5

[Michael Vines [:m1] [:evilmachines]]

(In reply to Cervantes Yu from comment #4)
> Is this a real case that the memory taken by the preallocated process makes
> a difference?
> If the foreground app consumes so much memory that we need to sacrifice the
> preallocated process, how much longer can the app remain alive?

Yes in v2.2 we definitely can't leave ~3MB of potentially usable memory by the foreground app on the table.  Available memory is much tighter here than previously, but with more MIPs running in parallel.

> > That's definitely a problem.  In this situation why can't we tell Nuwa to
> > fork a new preallocated process instead of forking from the chrome process? 
> > It seems to me on a low memory device we would never want to fork from the
> > chrome process for this reason (even at the expense of a slower launch time).
> 
> We have bug 1053634 for this issue.

Great, thanks!

Comment 6

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to Michael Vines [:m1] [:evilmachines] from comment #5)
> Yes in v2.2 we definitely can't leave ~3MB of potentially usable memory by
> the foreground app on the table.  Available memory is much tighter here than
> previously, but with more MIPs running in parallel.

What kind of device do you have in mind? On devices with at least 256MiB worth of memory I'd say that reserving 3-5MiB for greatly improved startup times is usually a win. That being said tweaking this behavior is very easy. Changing the "hal.processPriorityManager.gonk.PREALLOC.OomScoreAdjust" [1] pref to 166 would make the preallocated process die before the foreground app, and raising it to 300 would also make it die before the keyboard does.

[1] http://hg.mozilla.org/mozilla-central/file/f14dcd1c8c0b/b2g/app/b2g.js#l722

Comment 7

[Michael Vines [:m1] [:evilmachines]]

(In reply to Gabriele Svelto [:gsvelto] from comment #6)
>
> Changing the "hal.processPriorityManager.gonk.PREALLOC.OomScoreAdjust" [1]
> pref to 166 would make the preallocated process die before the foreground
> app, and raising it to 300 would also make it die before the keyboard does.

Thanks, this works great.  The other issues discussed in this bug are already tracked elsewhere so WFM.

Comment 8

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

This is important enough for testing reproducibility that I really don't want Mozilla and CAF to have different values here.  We should hammer out an agreement on something.

Comment 9

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #8)
> This is important enough for testing reproducibility that I really don't
> want Mozilla and CAF to have different values here.  We should hammer out an
> agreement on something.

My take on this is that we *desperately* needed this on the Tarako where launching w/o the preallocated process was super-slow and it would happen all the time since apps were dying like flies. On other devices, not so much. Even a 256MiB can easily keep multiple apps open at the same time so killing the preallocated process after everything else but before the foreground app is not much of an issue. It won't buy you much in the way of additional headroom (and even less so after bug 970307 lands) but it's not a big drawback either.

Comment 10

[Bobby Chien]

worksforme per comment 7, and this is low priority for fxos phone development.