1203945 - After "make install-gaia" we do not see the changes in the pushed application, unless we reboot

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia, defect, P1)

Description

[Julien Wajsberg [:julienw]]

STR:
0. Prerequisite: start the application on the device.
1. Change an application code (except System).
2. Push the application with this command:
  APP=<app> make install-gaia
=> The application should be killed on the device. This is OK.
3. Start the application on the phone.

Expected:
* We see the changes made to the application.

Actual:
* We don't see the changes made to the application.
After a reboot we see the changes, which makes it clear that the application was clearly pushed.


To be tested: according to an early investigation it's likely that we have the issue happening with normal updates too. But I'm not sure.

IMO this should be a blocker because this blocks developers badly.

Hey Ting-Yu, we think it's happening because of your changes in bug 1119692, could you please have a look?

Comment 1

[vnicolas]

My understanding is that the JAR cache needs to be cleaned up when an app is pushed on the device. Otherwise we end up re-using the wrong fd. I wonder if we can use inotify to look at what happens to the opened FD and update the cache accordingly.

I confirmed this is the source of the bug as commenting:
 gJarHandler->JarCache()->GetFd(packageFile, &cachedFd);

in dom/ipc/TabParent.cpp fix the issue for me (but likely makes certified app load slower).

This bug is really a big deal for days to days changes in Gaia (except for the system front-end team I believe :)).

Comment 2

[Ting-Yu Chou [:ting] (away)]

(In reply to Julien Wajsberg [:julienw] (PTO Sept 2nd -> Sept 10th) from comment #0)
> To be tested: according to an early investigation it's likely that we have
> the issue happening with normal updates too. But I'm not sure.

There's a flush-cache-entry event to clear the cache [1] once applyDownload() is called.

[1] https://dxr.mozilla.org/mozilla-central/source/dom/apps/Webapps.jsm#1868

Comment 3

[Ting-Yu Chou [:ting] (away)]

Won't there be security concern to clear the cached JAR whenever the file is overwritten?

Comment 4

[Ting-Yu Chou [:ting] (away)]

Will do flush-cache-entry when an app is pushed, just like devtool.

Comment 5

[Ting-Yu Chou [:ting] (away)]

(In reply to Ting-Yu Chou [:ting] from comment #4)
> Will do flush-cache-entry when an app is pushed, just like devtool.

Hmm... can't do this from make install-gaia as the script runs in another runtime.

Comment 6

[Julien Wajsberg [:julienw]]

Maybe, when Gecko detects that an app process is stopped (killed or normally stopped), check that the jar is still the same ?

Comment 7

[Ting-Yu Chou [:ting] (away)]

I prefer not to create a new path to flush jar cache, I'll see if I can fix this through similar mechanism of devtool.

Comment 8

[Ting-Yu Chou [:ting] (away)]

Note press "Install and Run" on WebIDE will trigger flush-cache-entry and load updated content.

Comment 9

[Ting-Yu Chou [:ting] (away)]

Whenever I connect to /data/local/debugger-socket, the usb debugging prompt will pop up to ask connection allowence. This can be disabled by pref "devtools.debugger.prompt-connection", which is default enabled.

Comment 10

[Ting-Yu Chou [:ting] (away)]

(In reply to Ting-Yu Chou [:ting] from comment #9)
> Whenever I connect to /data/local/debugger-socket, the usb debugging prompt
> will pop up to ask connection allowence. This can be disabled by pref
> "devtools.debugger.prompt-connection", which is default enabled.

Julien, how does this sound to you (or other gaia developer) if you need to go through usb debugging prompt for "APP=x make install-gaia"?

Comment 11

[vnicolas]

IMHO it seems OK as long as there is a pref to disable it (which exists already).

Comment 12

[Julien Wajsberg [:julienw]]

I usually disable this prompt on my own build.
We could also disable the prompt by default on eng builds?
BTW you need to check if your change works for certified apps; or if we need to enable devtools.debugger.forbid-certified-apps as well.

BTW I think make install-gaia should eventually use the same higher-level interface than WebIDE. Of course this is out of scope for this bug :)

Comment 13

[Ting-Yu Chou [:ting] (away)]

I also found "adb push" will delete the original file, though the fd still works. Using 'cat x > y' will not delete the file and reading from the fd will get updated content.

But push-to-device.js push to /system/b2g/webapps, and jar channel read from /data/local/webapps...

Comment 14

[Ting-Yu Chou [:ting] (away)]

(In reply to Ting-Yu Chou [:ting] from comment #13)
> But push-to-device.js push to /system/b2g/webapps, and jar channel read from
> /data/local/webapps...

Somehow the app also exists in /data/local/webapps, but after flash they are only in /system/b2g/webapps.

Comment 15

[GitHub Autolander Bot]

Attachment: [gaia] janus926:bug-1203945 > mozilla-b2g:master

Comment 16

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8661150 [details] [review]
[gaia] janus926:bug-1203945 > mozilla-b2g:master

Julien, could you please help test this patch your side? My local test does work but I don't really use "make install-gaia", would like to have a double check. Thanks.

Comment 17

[Julien Wajsberg [:julienw]]

(In reply to Ting-Yu Chou [:ting] from comment #13)
> I also found "adb push" will delete the original file, though the fd still
> works. Using 'cat x > y' will not delete the file and reading from the fd
> will get updated content.
> 
> But push-to-device.js push to /system/b2g/webapps, and jar channel read from
> /data/local/webapps...

Actually it pushes to the place you ask it for using the environment variable "GAIA_INSTALL_PARENT".

Comment 18

[Tim Guan-tin Chien [:timdream] (please needinfo)]

I agree this is annoying too but as this can be turned off, we should not prioritize this as blockers.

(In reply to vnicolas from comment #11)
> IMHO it seems OK as long as there is a pref to disable it (which exists
> already).

Comment 19

[Julien Wajsberg [:julienw]]

No, the bug can't be turned off.

Comment 20

[Tim Guan-tin Chien [:timdream] (please needinfo)]

(In reply to Julien Wajsberg [:julienw] from comment #19)
> No, the bug can't be turned off.

Someone should correct Vivien on that ...

Comment 21

[Julien Wajsberg [:julienw]]

What Vivien says is there is a pref to disable the prompt that will appear with Ting-Yu's patch (comment 9).

Comment 22

[Ting-Yu Chou [:ting] (away)]

If any gaia developer can help double confirm attachment 8661150 [details] [review] does work ...

Comment 23

[Justin D'Arcangelo [:justindarc]]

(In reply to Ting-Yu Chou [:ting] from comment #22)
> If any gaia developer can help double confirm attachment 8661150 [details] [review]
> does work ...

It does NOT work:

```
failed to copy '/Users/Justin/Development/Mozilla/gaia/profile/webapps/music-nga.gaiamobile.org/manifest.webapp' to '/tmp/b2g/webapps/music-nga.gaiamobile.org/manifest.webapp': No such file or directory
failed to copy '/Users/Justin/Development/Mozilla/gaia/profile/webapps/music-nga.gaiamobile.org/application.zip' to '/tmp/b2g/webapps/music-nga.gaiamobile.org/application.zip': No such file or directory
/system/bin/sh: cat: /tmp/b2g/webapps/music-nga.gaiamobile.org/manifest.webapp: No such file or directory
/system/bin/sh: cat: /tmp/b2g/webapps/music-nga.gaiamobile.org/application.zip: No such file or directory
```

Comment 24

[Hema Koka [:hema]]

justin is saying that it is taking him 5 times longer to flash and update his phone to try his changes. This is hurting developers productivity. making it a blocker. 

Ting-Yu, please help out.

Thanks much!

Hema

Comment 25

[Ting-Yu Chou [:ting] (away)]

(In reply to Justin D'Arcangelo [:justindarc] from comment #23)
> failed to copy
> '/Users/Justin/Development/Mozilla/gaia/profile/webapps/music-nga.gaiamobile.
> org/manifest.webapp' to
> '/tmp/b2g/webapps/music-nga.gaiamobile.org/manifest.webapp': No such file or
> directory

As the error message says there's no such file or directory, could you please check is the path correct? And how to reproduce it also your working environment. It works for me on Ubuntu.

Comment 26

[Justin D'Arcangelo [:justindarc]]

(In reply to Ting-Yu Chou [:ting] from comment #25)
> (In reply to Justin D'Arcangelo [:justindarc] from comment #23)
> > failed to copy
> > '/Users/Justin/Development/Mozilla/gaia/profile/webapps/music-nga.gaiamobile.
> > org/manifest.webapp' to
> > '/tmp/b2g/webapps/music-nga.gaiamobile.org/manifest.webapp': No such file or
> > directory
> 
> As the error message says there's no such file or directory, could you
> please check is the path correct? And how to reproduce it also your working
> environment. It works for me on Ubuntu.

I did check my local path and it definitely did exist. I believe the error was in reference to the path being missing on the device. I'm on OS X.

Comment 27

[Ting-Yu Chou [:ting] (away)]

Will get a mac to test when in the office.

Comment 28

[Ting-Yu Chou [:ting] (away)]

I can't reproduce comment 23 with my Aries on Ricky's Mac, but when I tried his Aries, there's the error, which comes with permission denied:

  Test SDK directory: /Users/Ricky/Documents/gaia/b2g_sdk/39.0a1-2015-03-05-16-02-02
  test -f /Users/Ricky/Documents/gaia/b2g_sdk/39.0a1-2015-03-05-16-02-02/B2G.app/Contents/MacOS/xpcshell
  [svoperapps.js] PROFILE_DIR, GAIA_DISTRIBUTION_DIR, VARIANT_PATH are all required
  [app] building sms app...
  [DEBUG_LOG]: LD_LIBRARY_PATH=
  [push] Waiting for device ...
  rm failed for //cache/*, Permission denied
  [push] GAIA_INSTALL_PARENT unset; probing previous installation location...
  Invalid JSON file : /Users/Ricky/Library/Caches/TemporaryItems/pushGaiaoo5v67y6/webapps.json
  Content of JSON file:
  /system/bin/sh: cat: /data/local/webapps/webapps.json: Permission denied


  [push] Install webapp(s) to /system/b2g...
  remount failed: Permission denied
  failed to copy '/Users/Ricky/Documents/gaia/profile/webapps/sms.gaiamobile.org/manifest.webapp' to '/tmp/b2g/webapps/sms.gaiamobile.org/manifest.webapp': No such file or directory
  failed to copy '/Users/Ricky/Documents/gaia/profile/webapps/sms.gaiamobile.org/application.zip' to '/tmp/b2g/webapps/sms.gaiamobile.org/application.zip': No such file or directory
  /system/bin/sh: can't create /system/b2g/webapps/sms.gaiamobile.org/manifest.webapp: Read-only file system
  /system/bin/sh: can't create /system/b2g/webapps/sms.gaiamobile.org/application.zip: Read-only file system
  [push] Restarting Messages...

Is this what you saw?

Comment 29

[Ting-Yu Chou [:ting] (away)]

(In reply to Ting-Yu Chou [:ting] from comment #28)
> I can't reproduce comment 23 with my Aries on Ricky's Mac, but when I tried
> his Aries, there's the error, which comes with permission denied:

Permission denied is because his device is with foxfooding image, but mine is not.

I've updated the patch a bit to get back double quote and splash for push command. Not sure will it make any differences for you.

I'll see if I can get some more gaia developers to test.

Comment 30

[Justin D'Arcangelo [:justindarc]]

No, that wasn't the error I got. I was using a Flame. Can you try running `make install APP=music-nga`? It seemed like it deleted music-nga off the device and was unable to copy it back because the dir was missing.

Comment 31

[Ting-Yu Chou [:ting] (away)]

I just figured out why, Flame does not have /tmp...

Comment 32

[Ting-Yu Chou [:ting] (away)]

Comment on attachment 8661150 [details] [review]
[gaia] janus926:bug-1203945 > mozilla-b2g:master

Push to /data/local/tmp instead of /tmp. Tested on Flame and Aries.

Comment 33

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Comment on attachment 8661150 [details] [review]
[gaia] janus926:bug-1203945 > mozilla-b2g:master

Thanks!

Comment 34

[Tim Guan-tin Chien [:timdream] (please needinfo)]

master: https://github.com/mozilla-b2g/gaia/commit/6b36df77d4d47cbaecafe4e3a1d3ea7400f17fc7