989294 - Synthetic APK never launches if downloaded outside Fx Marketplace

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Web Apps (PWAs), defect, P1)

Description

[henryfhchan]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36

Steps to reproduce:

I downloaded my app apk directly from https://controller.apk.firefox.com/application.apk.  When the app launches, the app is stuck on the splash screen and manifest.webapp is downloaded to /Download/.  The app never launches.

Comment 1

[Aaron Train [:aaronmt]]

In order to replicate the scenario, can you provide which manifest.webapp you provided to the APK factory service. It would also be helpful to detail which device, Android version and which channel of Firefox you used in this particular case.

Comment 2

[henryfhchan]

Https://igen6agon.com/mobilemaker/webapp/manifest.webapp

I have both Firefox and Nightly installed, while another user had Firefox and beta installed.

My android is 4.1, ditto for other user.

Comment 3

[Aaron Train [:aaronmt]]

Thanks for the information. Unfortunately I was not able to reproduce this. To be clear, these are the steps I performed:

* Essentially, I visited: https://controller.apk.firefox.com/application.apk?manifestUrl=https://igen6agon.com/mobilemaker/webapp/manifest.webapp

This prompts for APK download (in Nightly), I then used the notification to install the application.

On open, I successfully launched the application fine.

Can you capture and attach to this bug a log-cat from your device capturing output from when you attempt to launch the application?

Comment 4

[henryfhchan]

Could I directly send you the log-cat? since it seems to contain some sensitive info

Comment 5

[Aaron Train [:aaronmt]]

Received via email. Thanks

D/GeckoAppShell(25860): GeckoLoader.nativeRun /data/app/org.mozilla.fennec-2.apk -greomni /data/app/org.mozilla.fennec-2.apk  -P default -url https://controller.apk.firefox.com/application.apk?manifestUrl=https://igen6agon.com/mobilemaker/webapp/manifest.webapp -width 1080 -height 1920

I/ActivityManager(  534): Start proc org.mozilla.fennec:org.mozilla.fennec.Webapp1 for activity org.mozilla.fennec/.WebApps$WebApp1: pid=26352 uid=10135 gids={3003, 1015, 1006, 1028}

I/GeckoWebappImpl(26352): Trying to launch: {"url":"https:\/\/igen6agon.com\/mobilemaker\/webapp\/manifest.webapp"}

What's with the escaping slashes here?

and later on after Gecko init I see in the log:

E/GeckoConsole(26352): Error getting pref for application/x-web-app-manifest+json.

Comment 6

[Myk Melez [:myk] [@mykmelez]]

I got the logcat too and will investigate it later today for clues to the problem here.

Comment 7

[Myk Melez [:myk] [@mykmelez]]

The log shows this error:

E/GeckoEventDispatcher(26352): Unable to send response
E/GeckoEventDispatcher(26352): org.json.JSONException: No value for __guid__
E/GeckoEventDispatcher(26352):  at org.json.JSONObject.get(JSONObject.java:354)
E/GeckoEventDispatcher(26352):  at org.json.JSONObject.getString(JSONObject.java:510)
E/GeckoEventDispatcher(26352):  at org.mozilla.gecko.EventDispatcher.sendResponse(EventDispatcher.java:107)
E/GeckoEventDispatcher(26352):  at org.mozilla.gecko.GeckoApp.handleMessage(GeckoApp.java:721)

But that's unrelated (bug 989109) and shouldn't cause this problem.


I also see:

E/GeckoConsole(26352): Error getting pref for application/x-web-app-manifest+json.

Which happens on this line of the Helper App Dialog code:

http://hg.mozilla.org/mozilla-central/annotate/ccd91b78561f/mobile/android/components/HelperAppDialog.js#l212

And later:

D/MediaScannerService(22057): [MMF] MediaScannerService.scanFile: /storage/sdcard0/Download/manifest(2).webapp /mimeType: application/x-web-app-manifest+json/checkNoMedia :false

Which suggests that Fennec is trying to download the manifest URL instead of treating it as the ID of the app to start.


I suspect an error in the webapp registry's install process.  But it's hard to tell on an optimized build.

Henry: can you install this debug build on your phone and then launch the webapp using the "Fennec myk" app?

  https://people.mozilla.org/~myk/synthapk/fennec-31.0a1.en-US.android-arm.apk

If you then capture the logcat output, it should include messages from the Webapps.jsm file, which may provide clues about the problem.  You can email the output to me directly at [e-mail] or filter it on the string "Gecko" (which should remove any personal info, but check to be sure!) and then attach the filtered log to this bug.

Comment 8

[henryfhchan]

It launches reliably in this build.

When I press the app on my home screen, choosing Fennec myk reliably loads the page. Choosing Nightly or Fx Beta still reliably hangs on the splash screen and downloads the manifest (per launch).

Comment 9

[Myk Melez [:myk] [@mykmelez]]

Henry: in Nightly, when you go to Settings > Mozilla > About Nightly, what version is listed in the top-left corner of the page?

Comment 10

[henryfhchan]

2014 03 31

Comment 11

[Myk Melez [:myk] [@mykmelez]]

In bug 968129, comment 6, Mihai Pop describes a similar problem, although I think he installed the app from Marketplace, and that problem happens when only Firefox Beta is installed.

Comment 12

[Myk Melez [:myk] [@mykmelez]]

Bug 968129, comment 5 also describes a problem with apps getting stuck on the launch screen.  And bug 982557 describes a hang on launch, although it's specific to apps that were originally installed as legacy shortcuts, and in any case the fix has already landed on Central, Aurora, and Beta.

Comment 13

[Myk Melez [:myk] [@mykmelez]]

Henry, two more questions!

1. If you go to Settings > Tools > Apps, do you see an icon for the app in question?
2. Is your app available from Marketplace or perhaps some other website that calls mozApps.install() to install it?

Comment 14

[henryfhchan]

I can't reproduce now after I cleared my profile (delete data in android settings app)...

However no, to my memory it doesn't appear there, nor does it appear now if I install outside of Firefox.

App is only available via marketplace now, but I downloaded via the direct link and emailed it to my phone. Downloading via fx or chrome led to same result.

Comment 15

[henryfhchan]

After a few hours of debugging, I have finally got this complicated STR(s):

== Legend ==
APK: application.apk (The app, not nightly).
Clear: Clear Nightly Data + Uninstall the App (Not always in this order)
Silver: The background of the splash is Silver in color
Blue: The background of the splash is Blue in color
Load Success: The login screen of the app is shown
Load Fail: The login screen is never shown and a manifest is downloaded
Swipe Quit: The app is swiped off the recent apps list on Android
Clear Ram: The clear all button is pressed on the recent apps list on Android
Quit: Either RAM was cleared or swiped quit [For Nightly], Hardware menu button --> Quit or swiped quit [For App]

A.
1. Clear
2. Start Nightly + Navigate random Page
3. Visit Gmail
4. Pressed the link to https://controller.apk.firefox.com/application.apk?manifestUrl=https://igen6agon.com/mobilemaker/webapp/manifest.webapp
5. Choose Nightly
6. Pressed Notification
7. Pressed Install
8. Pressed Open
9. LOAD SUCCESS
10. Quit App
11. Uninstall App (Nightly still running)
12. Swap to File Manager + Open apk
13. Nightly crashed!!!! + Restarted
14. Switched Back to Install Prompt, pressed install
15. Pressed Open
16. LOAD SUCCESS
17. Uninstall (w/o quitting the app)
18. Launched Installer via File Manager, Install, Open
19. LOAD SUCCESS
20. Quit App
21. Clear RAM
22. Uninstalled App
23. Start Nightly
24. Install App via Folder
25. LOAD FAIL.

B.
1. Clear
2. Start Nightly
3. Swap to File Manger
4. Open APK
5. Install & Run
6. LOAD SUCCESS
7. Quit App
8. Quit Nightly
9. Uninstall App
10. Swap to File Manager
11. Open APK
12. Install & Run
13. LOAD SUCCESS
14. Start Nightly
15. Swap to App
16. Quit App via hardware menu button
17. Exit Nightly
18. Install & Run
19. LOAD FAIL

C.
1. Clear
2. Start Nightly + Navigate random page
3. Open APK, Install & Run
4. LOAD SUCCESS
5. Quit App
6. Clear Ram
7. Uninstall App
8. Install & Run
9. LOAD SUCCESS
10. Start Nightly
11. Swap to App
12. Quit App
13. Uninstall App
14. Swipe Exit Nightly
15. Install & Run
16. FAIL.

D.
1. Clear
2. Install App
3. Start Nightly + Random Nav
4. Start App
5. Quit App
6. Uninstall App
7. Swipe Quit Nihgtly
8. Install & Run
9. LOAD SUCCESS
10. Start Nightly
11. Swap to App
12. Quit App
13. Uninstall App
14. Swipe Exit Nighty
15. Install & Run
16. LOAD FAIL.

E.
1. Clear
2. Start Nightly + Nav
3. Swipe Quit Nihgtly
4. Install & Run
5. LOAD SUCCESS
6. Start Nightly
7. Swap to App
8. Quit App
9. Uninstall App
10. Swipe Exit Nightly
11. Install & Run
12. LOAD FAIL.

F.
1. Clear
2. Install & Run
3. LOAD SUCCESS
4. Start Nightly
5. Swap to App
6. Hardware Menu --> Quit App
7. Uninstall App
8. Swipe Quit Nightly
9. Install & Run
10. LOAD FAIL.

G.
1. Clear
2. Install & Run
3. LOAD SUCCESS (Silver)
4. Start Nightly
5. Swipe Quit App
6. Uninstall App
7. Swipe Quit Nightly
8. Install & Run
9. LOAD FAIL.

H.
1. Clear
2. Install App
3. Start Nightly
4. Uninstall App
5. Swipe quit Nightly
6. Install & Run
7. LOAD SUCCESS (Silver)
8. Clear All Ram
9. Reinstall & Run
10. LOAD SUCCESS (Blue)

I.
1. Clear
2. Install App & Open
3. LOAD SUCCESS (Silver)
4. Start Nightly
5. Swipe App
6. Reinstall App
7. Swipe Nightly
8. Open App
9. LOAD SUCCESS (Blue)

J.
1. Clear
2. Install App
3. Start Nightly
4. Start App
5. Quit App
6. Uninstall App
7. Swipe Quit Nightly
8. Install & Run
9. LOAD SUCCESS (Blue)
10. Start Nightly
11. Swipe Quit App
12. Uninstall App
13. Swipe Quit Nightly
14. Install & Run
15. LOAD FAIL.

From the above STR(s), two main STRs can be deduced.  What happens before does not seem important.

Excluding STR (A):
1. The installed app is started.
2. Nightly is started.
3. The app is quit then uninstalled immediately.
4. Nightly is swiped quit.
5. Install then run.

After refining STR (A), the resulting STR is:
1. Nightly is started.
2. The app is installed then started.
3. The app is quit.
4. Nightly is quit.
5. The app is uninstalled
6. Start Nightly
7. Install then run.

Another thing I noted:
Uninstalling + Installing or direct Installing on top always causes NIL result

A funny bug I saw is that sometimes the background of splash is silver, sometimes its blue.  Also, if Nightly hasn't started once after installing, and the app is run, the app never loads and is stuck on silver.

Comment 16

[henryfhchan]

Continuing STR Summary (1)
1. I tried clearing Nightly's Data (without uninstalling the app)
2. Run App (LOAD SUCCESS - Silver)
3. Run Nightly
4. Swipe App Quit + uninstalled immediately
5. Clear Ram quit Nightly
6. Install & Run
7. LOAD FAIL - BLUE

This shows that it doesn't really matter how you chain the events or how you quit the app, as long as the exact order is followed.

8. Clear Ram
9. Open Nightly
10. Install & Run
11. LOAD FAIL - Blue

Once it starts failing, any sequence of steps make it continue to fail.

Comment 17

[henryfhchan]

BTW:
> 1. If you go to Settings > Tools > Apps, do you see an icon for the app in question?

No, not even when the app LOAD SUCCESS.

> 2. Is your app available from Marketplace or perhaps some other website that calls mozApps.install() to install it?

I planned to distribute it via email to specified users, but they had trouble with it launching and possibly had a different set of STR. (they couldn't recall the process correctly)

Comment 18

[Myk Melez [:myk] [@mykmelez]]

(In reply to henryfhchan from comment #16)
> Continuing STR Summary (1)
> 1. I tried clearing Nightly's Data (without uninstalling the app)
> 2. Run App (LOAD SUCCESS - Silver)
> 3. Run Nightly
> 4. Swipe App Quit + uninstalled immediately
> 5. Clear Ram quit Nightly
> 6. Install & Run
> 7. LOAD FAIL - BLUE

Thanks for the detailed STRs!

Using these steps, I'm able to reproduce the problem on a custom debug build, which means we should be able to debug and resolve it.  It still isn't clear how significant it is, but I'm concerned that it'll manifest for other, more common use cases, especially given the aforementioned comments in bug 968129.  So I'm going to bump this bug's priority, take it, and investigate it immediately.

Comment 19

[Myk Melez [:myk] [@mykmelez]]

Attachment: patch v1: release allocated index when APK uninstalled

This is a bug in UninstallListener.doUninstall.

When the user uninstalls the APK in comment 16, step 4, Android broadcasts android.intent.action.PACKAGE_REMOVED and Fennec receives it, triggering a call to UninstallListener.doUninstall.  And that method correctly uninstalls the app from DOMApplicationRegistry.  But it incorrectly neglects to release its index from Allocator.

Then, when the APK is reinstalled and started, Dispatcher thinks the app is installed in DOMApplicationRegistry, because Allocator knows about it, so WebappImpl tries to launch it, which eventually reaches WebappRT, which can't find the app in DOMApplicationRegistry and so tries to "open it [the manifest URL] like a web page":

http://hg.mozilla.org/mozilla-central/annotate/7bacc9e903b0/mobile/android/chrome/content/WebappRT.js#l110

The proximate fix is a simple one-liner: make UninstallListener.doUninstall release the index when observing APK removal.  Here's a patch that does that.

But this investigation reveals several more issues that we should address in followup bugs:

1. If an app is allocated but isn't installed in DOMApplicationRegistry, WebappRT tries to load the manifest like a web page, which isn't useful.  It should instead install the app!

2. UninstallListener.doUninstall and EventListener.uninstallWebapp mostly duplicate code, as they both kill the app's process, remove its profile, and (with this fix) release its index from the allocator.  We should refactor those methods to eliminate that duplication.

3. On firstrun of an app that isn't yet allocated/installed, WebappImpl.onCreate calls InstallHelper.startInstall to install the app—which includes calculating its splash screen background color—after it displays the splash screen.  The color calculation should happen before the splash screen is displayed.

I'll file all three issues as followups.

Comment 20

[Myk Melez [:myk] [@mykmelez]]

Henry: if you want to verify that my fix resolves the problem, here's a build with the fix:

  https://people.mozilla.org/~myk/synthapk/bug-989294.apk

Note that this build doesn't un-break apps that have already been broken, it just prevents them from getting broken in the first place.  (Unbreaking those apps would require us to address followup issue #1 in comment 19.)

Comment 21

[henryfhchan]

Oops now it just shows a blank page after the splash，using first consolidated STR

Comment 22

[henryfhchan]

Pressing back to the desktop, then relaunching shows about:home at top and blank page. Persistent.

Comment 23

[Myk Melez [:myk] [@mykmelez]]

(In reply to henryfhchan from comment #21)
> Oops now it just shows a blank page after the splash，using first
> consolidated STR

Henry, can you clarify which STR you mean by "first consolidated STR"?  I want to be sure I understand exactly what you're doing when you see that problem!

Comment 24

[Myk Melez [:myk] [@mykmelez]]

In the meantime, I'm going to land the existing fix, which is obviously correct, even if there's another problem here.

Comment 25

[Myk Melez [:myk] [@mykmelez]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/488caa16c617

Comment 26

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/488caa16c617

Comment 27

[Myk Melez [:myk] [@mykmelez]]

Comment on attachment 8400951 [details] [diff] [review]
patch v1: release allocated index when APK uninstalled

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 
  It isn't clear, but perhaps bug 933979 or bug 934756.

User impact if declined: 
  Under certain circumstances, apps will fail to run.

Testing completed (on m-c, etc.): 
  I've tested this locally, and it landed on Central several days ago.

Risk to taking this patch (and alternatives if risky): 
  This is low-risk.

String or IDL/UUID changes made by this patch:
  None.

Comment 28

[Myk Melez [:myk] [@mykmelez]]

(In reply to Myk Melez [:myk] [@mykmelez] from comment #19)

> 1. If an app is allocated but isn't installed in DOMApplicationRegistry,
> WebappRT tries to load the manifest like a web page, which isn't useful.  It
> should instead install the app!

Bug 994982.


> 2. UninstallListener.doUninstall and EventListener.uninstallWebapp mostly
> duplicate code, as they both kill the app's process, remove its profile, and
> (with this fix) release its index from the allocator.  We should refactor
> those methods to eliminate that duplication.

Bug 994983.


> 3. On firstrun of an app that isn't yet allocated/installed,
> WebappImpl.onCreate calls InstallHelper.startInstall to install the
> app—which includes calculating its splash screen background color—after it
> displays the splash screen.  The color calculation should happen before the
> splash screen is displayed.

Bug 994984.

Comment 29

[Ryan VanderMeulen [:RyanVM]]

Depends on bug 933979.

Comment 30

[Myk Melez [:myk] [@mykmelez]]

Attachment: Aurora and Beta branch patch

Here's the equivalent for the Aurora and Beta branches.  Alternately, we could uplift bug 933979, but that would be a larger, riskier change.

Comment 31

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/682f6a88ebec
https://hg.mozilla.org/releases/mozilla-beta/rev/7872e02410a7