Closed Bug 1170836 Opened 4 years ago Closed 4 years ago

Importing certified themes in dev mode doesn't work

Categories

(Firefox OS Graveyard :: Gaia::Theme Editor, defect, P1, blocker)

ARM
Gonk (Firefox OS)
defect

Tracking

(blocking-b2g:2.5+, firefox39 wontfix, firefox40 wontfix, firefox41 fixed, b2g-master fixed)

VERIFIED FIXED
2.2 S14 (12june)
blocking-b2g 2.5+
Tracking Status
firefox39 --- wontfix
firefox40 --- wontfix
firefox41 --- fixed
b2g-master --- fixed

People

(Reporter: callahad, Assigned: fabrice)

References

Details

(Keywords: dogfood)

Attachments

(2 files)

Steps to Reproduce:

(This is on an Aries device running the dogfood update channel)

1. Download and flash aries.zip from https://tools.taskcluster.net/task-inspector/#xRVy3KtOQdKKP1gu0quJIw/1

2. Set up the device and apply the OTA (build identifier 20150602123959, build number eng.worker.20150528.204812)

3. Reboot (should work normally)

4. Open the Studio app and attempt to enable the Solarized Light theme, then the Solarized Dark theme.

(note: tapping "enable" results in an app.zip download notification, no other action.)

5. Open WebIDE, Runtime Info, root device then request elevated privileges.

6. Reboot (should work normally)

7. Open the Studio app and attempt to enable the Solarized Light theme, then the Solarized Dark theme.

(note: this time you should see success notifications)

8. Attempt to power down or reboot the phone.

What should happen:

- The phone restarts

What actually happens:

- The screen goes black and unresponsive, but the backlight remains on and the phone does not power down or restart.

When in this state, the phone is completely unusable. I haven't managed to fix it either, short of re-flashing. It's also impossible to connect the devtools over USB, since you can't see and tap on the "accept debugging" prompt, though wifi debugging still works if you had previously connected over wifi with the "scan and remember" option.

The phone responds to adb normally, and an `adb reboot` will get it back to a semi-functional state, but attempting to power down or reboot will re-manifest the issue.

In addition to the black screen, other features are broken after step 7:

- Messages in SMS threads aren't visible.
- Rearranging icons and scrolling in the home screen's edit mode doesn't work.

If I connect with the WebIDE and inspect the system app, I can see that the black screen is the normal div#poweroff-splash, however, the animation never plays and shutdown never actually happens. Using the devtools to set its visibility to hidden allows me to see and interact with the phone as if nothing had ever happened.

[Blocking Requested - why for this release]: Renders phone unusable
Attached file adb logcat output
Attaching adb logcat output; looks there's like an issue with invalid css variables from the theme, especially around transitions.

This might make sense as a mechanism for failure, since things like actually powering down are set as event listeners for transitionend:

https://github.com/mozilla-b2g/gaia/blob/26650467a184a357186b41db6c9c922a073257e0/apps/system/js/sleep_menu.js#L398-L400

ni? :etienne per :drs's request.
Flags: needinfo?(etienne)
Thanks so much for the really detailed investigation and report, Dan.

I'm also bringing in Hubert to take a look.
Flags: needinfo?(hub)
blocking-b2g: spark? → spark+
This was fixed in bug 1161481, how old is the Studio build shipped in those images?
Flags: needinfo?(hub)
Flags: needinfo?(etienne)
Flags: needinfo?(drs)
We're always building directly from the repo. The image that Dan is using is only a few days old. Thus, the Studio app in it is also only a few days old.

Dan, could you try installing the Studio app directly from the repo [1] and see if the issue still repros? There are no build steps, so it should be fairly easy.

[1] https://github.com/fxos/studio
Flags: needinfo?(drs) → needinfo?(dan.callahan)
The master branch still breaks my phone.

As of writing, master was at 0faa606f92f90011d4890bf97ff61bbe5e1741cd.

Any suggestions for unbreaking the phone? Or backing up / restoring my settings? Having to reflash to get back into a working state is a major bummer in terms of setting the whole damn phone up again. and again. and again. :)
Flags: needinfo?(dan.callahan)
OK, going into Settings -> Themes -> Default gets me back to a working state.
Etienne, since this is still busted according to comment 5, could you please take a look?
Flags: needinfo?(etienne)
The reboot doesn't reboot correctly it seems.  I ran adb reboot and was able to reboot following.
Sorry, I prematurely pointed Daniel here.

(In reply to Daniel Holbert [:dholbert] from comment #9)
> (I don't recall what I was doing on the phone just before this, but I
> definitely hadn't used the theme editor recently.)

Based on that, this sounds like a different issue. Daniel's going to file a bug for this.
(Yup, I spun off bug 1171827 for my issue. Marking comment 9 as offtopic to autohide it.)
(In reply to Doug Sherk (:drs) (use needinfo?) from comment #7)
> Etienne, since this is still busted according to comment 5, could you please
> take a look?

Don't have anything actionable.
Flags: needinfo?(etienne)
(In reply to Dan Callahan [:callahad] from comment #5)
> The master branch still breaks my phone.
> 
> As of writing, master was at 0faa606f92f90011d4890bf97ff61bbe5e1741cd.
> 

The fix was definitely in at this point.
And the --transition-duration value is hard coded in the JS so a older indexeddb database would not explain the issue.

So my best guess is that there's an issue (maybe a race, since I can't reproduce it) with the theme loading in the system app.
Selecting the "Default theme" fixes it since we're reloading the stylesheet when the selected theme changes.
Duplicate of this bug: 1171943
I also reproduced the issue (see bug 1171943). After talking to Etienne, the lastest version of Studio works on the latest Cypress build[1] (from May 15th):
* The theme is applied
* You can reboot
* There is no freezes in lieu of transitions.

From Etienne's POV, it seems like a patch is missing in Gecko. Fabrice, do you have an idea of what it could be?

[1] https://tools.taskcluster.net/index/artifacts/#gecko.v1.cypress.latest.linux.aries-eng/gecko.v1.cypress.latest.linux.aries-eng.opt
Flags: needinfo?(fabrice)
According to Fabrice in bug 1167156 comment 18, this might just work with Developer Mode enabled.
No, I can't think of anything.
Depends on: 1172051
Johan, can you repro this with Developer Mode enabled?

To enable DM:
1. Open WebIDE, connect to your Aries.
2. Open "Runtime Info" in the device menu.
3. Enable ADB root if it's not enabled already.
4. Reconnect to the device after you get automatically disconnected, go back to "Runtime Info".
5. Request higher privs.
Flags: needinfo?(jlorenzo)
Flags: needinfo?(fabrice)
Per bug 1171943 comment 0, Developper Mode was enabled with the instructions found at: https://github.com/fxos/lightsaber#enable-developer-mode . Are the instructions in comment 18 equivalent? If so, yes I repro'd with developer mode enabled.
Flags: needinfo?(jlorenzo) → needinfo?(drs)
Johan, the instructions there are very out of date and incorrect. I wrote up some instructions on how to enable Developer Mode here:
https://developer.mozilla.org/en-US/Firefox_OS/Developer_Mode#Enabling_and_disabling
Flags: needinfo?(drs)
I enabled the Developer Mode with WebIDE and I still reproduce the problem: No theme applied, no transitions when opening apps, and the device gets sluggish.

By the way, when dev mode is not enabled, the customizer app shows this link at start up: https://github.com/fxos/lightsaber#enable-developer-mode . I'd recommend to change the README of this project to point to the one you gave in comment 20.

Finally, I'm sorry to highlight bad news but, as:
* the happy path is currently not working,
* the bug has a huge negative impact on the first time experience,
* the flash of the 400 devices will begin next Monday,
* and as we don't have a clue about the root cause,
=> we might start to consider to ship spark without this app. What do you think, Doug?
Flags: needinfo?(drs)
(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #21)
> I enabled the Developer Mode with WebIDE and I still reproduce the problem:
> No theme applied, no transitions when opening apps, and the device gets
> sluggish.

I'm just speculating now, but maybe something in mozilla-central landed and broke this after cypress was abandoned.
 
> By the way, when dev mode is not enabled, the customizer app shows this link
> at start up: https://github.com/fxos/lightsaber#enable-developer-mode . I'd
> recommend to change the README of this project to point to the one you gave
> in comment 20.

Thanks, filed bug 1172425.

> Finally, I'm sorry to highlight bad news but, as:
> * the happy path is currently not working,
> * the bug has a huge negative impact on the first time experience,
> * the flash of the 400 devices will begin next Monday,
> * and as we don't have a clue about the root cause,
> => we might start to consider to ship spark without this app. What do you
> think, Doug?

I'm inclined to agree. Fabrice, Etienne, I'd prefer not to cut this from the build. Please investigate this jointly so that we don't have to.
Flags: needinfo?(fabrice)
Flags: needinfo?(etienne)
Flags: needinfo?(drs)
QA: Does this work correctly on a build of mozilla-central from ~2 months ago?
Naoki, where can we get a build that is older than 2 months ago to check this for Doug?
Flags: needinfo?(nhirata.bugzilla)
I'm not convinced that using such an old build is useful. It will lack too many features that are needed anyway.
Flags: needinfo?(fabrice)
I think that's the earliest aries build we have:
https://tools.taskcluster.net/index/artifacts/#gecko.v1.mozilla-central.revision.linux.ff2e0722804149cc85a6f4a26ae9983a10e4800c.aries-eng/gecko.v1.mozilla-central.revision.linux.ff2e0722804149cc85a6f4a26ae9983a10e4800c.aries-eng.opt

You would need to make a custom aries build with an earlier gecko/gaia.
Please try with this build.

I also agree with Fabrice.  If it doesn't work there, I think we need to investigate in other ways.
Flags: needinfo?(nhirata.bugzilla) → needinfo?(ktucker)
I looked a bit at what is happening. Here's the metadata for one of the themes:
"{f3e07910-b887-4786-8d3b-b877c15354d7}": {
    "installOrigin": "http://gaiamobile.org",
    "manifestURL": "app://themeapp84535138.gaiamobile.org/update.webapp",
    "version": 1,
    "localId": 1051,
    "id": "{f3e07910-b887-4786-8d3b-b877c15354d7}",
    "basePath": "/data/local/webapps",
    "name": "Solarized Dark",
    "installTime": 1433805120672,
    "removable": true,
    "progress": 0,
    "installState": "installed",
    "downloadAvailable": false,
    "downloading": false,
    "readyToApplyDownload": false,
    "downloadSize": 0,
    "lastUpdateCheck": 1433805120672,
    "updateTime": 1433805120672,
    "manifestHash": "3528175afcf8acca640204cb807a07b1",
    "installerAppId": 0,
    "installerIsBrowser": false,
    "role": "theme",
    "origin": "app://{f3e07910-b887-4786-8d3b-b877c15354d7}",
    "appStatus": 3,
    "kind": "packaged",
    "redirects": null,
    "widgetPages": [],
    "storeId": "",
    "storeVersion": 0,
    "enabled": true,
    "csp": "",
    "additionalLanguages": {}
  }

Clearly the origin is wrong, since the manifest asks for it to be {"name":"Solarized Dark","role":"theme","type":"certified","origin":"app://themeapp84535138.gaiamobile.org"}

Looks like something is broken in app.import() which is one the non-trivial backports that happened when moving back from cypress to m-c
QA's help is no longer necessary based on comment 28, thanks anyways.
Flags: needinfo?(ktucker)
Keywords: qaurgent, qawanted
Summary: [spark] Theme editor (Studio) renders phone unusable (black screen when rebooting) → Importing certified themes in dev mode doesn't work
Attached patch themes.patchSplinter Review
We were not checking dev mode to allow custom origins, and also only allowed custom origins for privileged apps - but not the certified ones.
Assignee: nobody → fabrice
Attachment #8617029 - Flags: review?(ferjmoreno)
Flags: needinfo?(etienne)
Comment on attachment 8617029 [details] [diff] [review]
themes.patch

Review of attachment 8617029 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/apps/ImportExport.jsm
@@ +420,2 @@
>              manifest.origin) {
> +

nit: remove extra line
Attachment #8617029 - Flags: review?(ferjmoreno) → review+
https://hg.mozilla.org/mozilla-central/rev/13f8dcb5b7a5
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → 2.2 S14 (12june)
Still seeing the same issue with this patch...

  "{df238e59-16c8-41b7-a13f-55243b057319}": {
    "installOrigin": "http://gaiamobile.org",
    "manifestURL": "app://themeapp18068847.gaiamobile.org/update.webapp",
    "version": 1,
    "localId": 1072,
    "id": "{df238e59-16c8-41b7-a13f-55243b057319}",
    "basePath": "/data/local/webapps",
    "name": "Solarized Dark",
    "installTime": 1433940852782,
    "removable": true,
    "progress": 0,
    "installState": "installed",
    "downloadAvailable": false,
    "downloading": false,
    "readyToApplyDownload": false,
    "downloadSize": 0,
    "lastUpdateCheck": 1433940852782,
    "updateTime": 1433940852782,
    "manifestHash": "89b365148931a670835015a46de34191",
    "installerAppId": 0,
    "installerIsBrowser": false,
    "role": "theme",
    "origin": "app://{df238e59-16c8-41b7-a13f-55243b057319}",
    "appStatus": 3,
    "kind": "packaged",
    "redirects": null,
    "widgetPages": []
  }
Flags: needinfo?(fabrice)
Priority: -- → P1
Are you 300% sure you have dev mode enabled?
Flags: needinfo?(fabrice)
(In reply to Fabrice Desré [:fabrice] from comment #35)
> Are you 300% sure you have dev mode enabled?

Yep, at least if it's still named
user_pref("dom.apps.developer_mode", true);


And I actually went as far as to pull my omni.ja to make sure ImportExport.jsm was properly patched.
QA: Please verify that this works now. You will have to flash the Theme Editor yourself as the one on yesterday's nightly is slightly out of date. Just follow the regular build instructions for Gaia:
|GAIA_DISTRIBUTION_DIR=distros/spark make reset-gaia|

Time is of the essence here. We have to make the final call tomorrow on whether or not to include Theme Editor in our release, and it hinges on this bug.
Flags: needinfo?(pbylenga)
Flags: needinfo?(ktucker)
Keywords: qaurgent, qawanted
Flags: needinfo?(ktucker) → needinfo?(bzumwalt)
(In reply to Doug Sherk (:drs) (use needinfo?) from comment #37)
> QA: Please verify that this works now. You will have to flash the Theme
> Editor yourself as the one on yesterday's nightly is slightly out of date.
> Just follow the regular build instructions for Gaia:
> |GAIA_DISTRIBUTION_DIR=distros/spark make reset-gaia|
> 
> Time is of the essence here. We have to make the final call tomorrow on
> whether or not to include Theme Editor in our release, and it hinges on this
> bug.

Also note that you must enable Developer Mode using these instructions before following the STR, even if it doesn't prompt you to: https://developer.mozilla.org/en-US/Firefox_OS/Developer_Mode
Was unable to reproduce issue, cannot complete STR from comment 0 past step 7. After initially enabling Solarized Light Theme and Solarized Dark theme in step 4 (both gave bottom banner message about "Theme Installed", never saw a message about "app.zip"), enable buttons for both themes are greyed out on step 7 and are not able to be pressed again.

No issues with phone rebooting incorrectly observed. Is observed behavior part of the fix Doug?

Device: Aries 3.0
Build ID: 20150611131651
Gaia: c3b15607bfdc53fd4fb7ff1d26b39ba38b3f1b41
Gecko: 9ebd530c5843
Gonk: 75c7e6ca80d0c7a53f346ecfcde2343be95808c9
Version: 41.0a1 (3.0)
Firmware Version: D5803_23.1.A.1.28_NCB.ftf
User Agent: Mozilla/5.0 (Mobile; rv:41.0) Gecko/41.0 Firefox/41.0
Flags: needinfo?(bzumwalt) → needinfo?(drs)
That all sounds right. I'm not sure what your process is, but I think at this point we should switch to VERIFIED FIX. Leaving it up to you.

I'm still a little bit uncomfortable that Etienne was having issues with this, but I think it's safe to say that it's because he didn't set the other prefs that are listed here:
https://developer.mozilla.org/en-US/Firefox_OS/Developer_Mode#Developer_prefs_and_settings_enabled
Flags: needinfo?(drs) → needinfo?(bzumwalt)
Double checked that all developer prefs and settings were correctly set on my end. I feel pretty good saying this is verified since I am seeing the same thing on two different devices.
Status: RESOLVED → VERIFIED
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(bzumwalt)
Keywords: qaurgent, qawanted
Flags: needinfo?(pbylenga) → needinfo?(ktucker)
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(ktucker)
(In reply to Brogan Zumwalt [:BroganZ] from comment #41)
> Double checked that all developer prefs and settings were correctly set on
> my end. I feel pretty good saying this is verified since I am seeing the
> same thing on two different devices.

But did you see the theme change?
The colors, of the theme app itself, did they changed when enabling the theme?
Flags: needinfo?(bzumwalt)
(In reply to Etienne Segonzac (:etienne) from comment #42)
> But did you see the theme change?
> The colors, of the theme app itself, did they changed when enabling the
> theme?

Verified on [1] with studio at 635f9db7481e6411ac2e1cb19fbd680565f9bb94, and the theme does change. Testing done.
* Activate a theme => The theme is immediately applied
* Duplicate another theme and change to it => The correct theme is applied
* Change an applied theme on the fly => The theme gets re-installed and you can see your modification immediately
* In settings, change the theme to another one => Applied directly
* Rollback to the default theme => No problem except that the homescreen you set before playing with Studio is forgotten. I filed bug 1174148 for that.

I haven't deeply tested the application, but the happy paths are passing to me. I think this app is currently dogfood-able.

[1] Build ID               20150611132111
Gaia Revision          68269e7b6510930eb2f644f69d27d456c1bdec75
Gaia Date              2015-06-10 23:36:01
Gecko Revision         https://hg.mozilla.org/mozilla-central/rev/9ebd530c5843
Gecko Version          41.0a1
Device Name            aries
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.worker.20150605.140045
Firmware Date          Fri Jun  5 14:00:54 UTC 2015
Bootloader             s1
Flags: needinfo?(bzumwalt)
See Also: → 1174148
blocking-b2g: spark+ → 2.5+
You need to log in before you can comment on or make changes to this bug.