820456 - Installing a big packaged app could kill the homescreen and make the phone unresponsive

Status: VERIFIED FIXED

(Core Graveyard :: DOM: Apps, defect)

Description

[Julien Wajsberg [:julienw]]

STR:
* go to https://everlong.org/mozilla/bigpackaged/ (don't go there all at once, this is my personal server)
* click on "install app" (this is a about 100 MB packaged app)
* wait during the download

using |top -m 30 -s rss| you can see that the main process is taking more and more memory.

On my device freshly started, when I start the download, I have 3 plugin-container processes (homescreen, browser, and a preallocated one). At the end, there is only the preallocated one.

I also used |while true ; do adb shell b2g-ps ; echo ---- ; sleep 2 ; done| to know exactly what's happening. My personal test showed that Cost Control is launched at one moment, and then browser and homescreen are killed, and then Cost Control.

This can be fixed with a reboot, and the app seems to be correctly installed.


Tried also with https://everlong.org/mozilla/verybigpackaged/ (~150MB) and it makes the phone unresponsive. The b2g process seems repeatedly killed and relaunched. The download seems to continue a little but the phone just shows a black screen that can be only be cured by |adb reboot| (or, as a user, by removing the battery), which makes the phone usable again.


I can see 2 scenarios here, which share the same problem: we don't always know the size in advance, and even if we know it, it may be wrong.

1. limit the size of apps that we can install for v1. It means that we should stop downloading if we hit a certain progress.

2. download to disk/flash for apps that are bigger than a particular size. It means that if we hit a certain progress, we should transfer everything we downloaded in memory so far to a file, and then append to this file.

I believe we could very easily hit the limit with games, that's why I'd prefer to the 2nd scenario implemented, but it would be so easier to implement scenario 1.


I didn't try for hosted-apps-with-appcache.

Comment 1

[[:fabrice] Fabrice Desré]

Gabriele, can you profile/investigate this issue?

Comment 2

[[:fabrice] Fabrice Desré]

I took a look at the package download code and what's happening is that:
- we use NetUtil.asyncFetch() to get the zip at https://mxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#1739 with a channel we create.
- NetUtil.asyncFetch() sets up a a pipe with an infinite number of segments at https://mxr.mozilla.org/mozilla-central/source/netwerk/base/src/NetUtil.jsm#131

I hacked NetUtil to add a segmentCount argument and set it to a sane value, but the download fails to complete with an NS_BASE_STREAM_WOULD_BLOCK error. I guess we need to rewrite that code without NetUtil.asyncFetch().

Comment 3

[[:fabrice] Fabrice Desré]

Attachment: patch

This patch applies on top of the one in bug 815173 - I was able to  install Julien's 100Mb app without crashing and keeping b2g process's RSS constant.

Comment 4

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to Fabrice Desré [:fabrice] from comment #1)
> Gabriele, can you profile/investigate this issue?

Do you still need my help here? Sorry but I saw your request only this morning.

Comment 5

[Gabriele Svelto [:gsvelto] (PTO)]

Comment on attachment 691711 [details] [diff] [review]
patch

I tested the patch on my Otoro in addition to the one in bug 815173 and managed to install the big packages app successfully. I monitored the main process' memory consumption and it stayed almost constant during the whole process. CPU usage was also low.

Comment 6

[Fernando Jiménez Moreno [:ferjm]]

Comment on attachment 691711 [details] [diff] [review]
patch

Review of attachment 691711 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/apps/src/Webapps.jsm
@@ +778,5 @@
>          this.doInstallPackage(msg, mm);
>          break;
>        case "Webapps:GetBasePath":
> +        if (!this.webapps[msg.id]) {
> +          debug("No webapp for " + msg.id);

We should probably return here to avoid a TypeError trying to access |this.webapps[msg.id].basePath|

Comment 7

[[:fabrice] Fabrice Desré]

https://hg.mozilla.org/integration/mozilla-inbound/rev/f75a9f57c14e

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/f75a9f57c14e

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/c28d377a35af
https://hg.mozilla.org/releases/mozilla-b2g18/rev/5f72fcc9ba3b

Comment 10

[Jason Smith [:jsmith]]

Verified on 1/21 by installing the 150 MB packaged app, using other apps, and noticing that the phone remains relatively responsive.