1089710 - Upgrade to B2G Desktop 39

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Build, defect)

Description

[Staś Małolepszy :stas]

I suggest we use a newer version of B2G Desktop in Gaia's build system.  Currently it's:

B2G_SDK_VERSION := 34.0a1                                                                 
B2G_SDK_DATE := 2014/08/2014-08-12-04-02-01

Comment 1

[Staś Małolepszy :stas]

Attachment: B2G_SDK_VERSION := 36.0a1

https://github.com/mozilla-b2g/gaia/pull/25548
https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=58978f5ce334

Comment 2

[Staś Małolepszy :stas]

Kevin, Julien -- do you have any ideas why Gij in https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=936ea60fbb91 might be failing?

Comment 3

[Julien Wajsberg [:julienw]]

I don't know Gij enough :/ But I also think b2g_sdk is not what is used to run the integration tests... I think it's used only to run the build scripts.

but I don't really know actually :/

Comment 4

[Staś Małolepszy :stas]

Thanks for chiming in.  I too think b2g_sdk is only used for build scripts, which is why I need it for bug 994357 (which uses <template>'s Document Fragments which recently got a fix in mozilla-central).

I'll keep investigating.

Comment 5

[Kevin Grandon :kgrandon]

Comment on attachment 8512093 [details] [diff] [review]
B2G_SDK_VERSION := 36.0a1

Review of attachment 8512093 [details] [diff] [review]:
-----------------------------------------------------------------

Going to re-assign this since Vivien is out for a bit. Not sure who the best person to review this is, but I think Ricky should perhaps do it as a new peer of the build system?

Comment 6

[Ricky Chien [:rickychien] (inactive)]

After applying this patch, it works fine on Ubuntu.
However, I saw such error on my mac with |make|:

Test SDK directory: /Users/Ricky/Documents/gaia/b2g_sdk/36.0a1-2014-10-27-04-02-37
test -f /Users/Ricky/Documents/gaia/b2g_sdk/36.0a1-2014-10-27-04-02-37/B2G.app/Contents/MacOS/xpcshell
JavaScript error: file:///Users/Ricky/Documents/gaia/b2g_sdk/36.0a1-2014-10-27-04-02-37/B2G.app/Contents/MacOS/components/DirectoryProvider.js, line 1: NS_ERROR_FILE_NOT_FOUND: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIXPCComponents_Utils.import]
JavaScript error: /Users/Ricky/Documents/gaia/build/xpcshell-commonjs.js, line 12: NS_ERROR_FILE_NOT_FOUND: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIXPCComponents_Utils.import]
JavaScript error: /Users/Ricky/Documents/gaia/build/xpcshell-commonjs.js, line 123: TypeError: CommonjsRunner is not a constructor

Do you try it on mac before?

Comment 7

[Staś Małolepszy :stas]

Hi Ricky. No, I hadn't tried it before on a Mac, but I did minutes ago and I could reproduce the same problem as you had.  Unfortunately, I don't really know what is going on.  Do you have any ideas?  I'm hoping that you can help me as I'm not really familiar with our build system;  I just need this bug to move forward in bug 994357 :)

On my Ubuntu, I am also getting the following error:

JavaScript error: file:///srv/moz/gaia/b2g_sdk/36.0a1-2014-11-06-04-02-07/b2g/components/nsHandlerService.js, line 120: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]

Comment 8

[Ricky Chien [:rickychien] (inactive)]

I think that it may relate to loader while import 'resource://gre/modules/commonjs/toolkit/loader.js'.

However, I have no idea why it breaks on mac but work fine on windows. :(

Alex, do you have any idea?

Comment 9

[Alexandre Poirot [:ochameau]]

I don't know. 
Looking at the error logs, it looks like the root issue is in DirectoryProvider.js.
But it mentions Line 1, so that we don't really know which line is really failing :(

I do not have any Mac next to me to give it a try.
Could you try editing DirectoryProvider.js locally so figure out which line fails?
Otherwise, you may also try to use an even more recent b2g build, to see if that is something that may already have been fix.

(In reply to Staś Małolepszy :stas from comment #7)
> On my Ubuntu, I am also getting the following error:
> 
> JavaScript error:
> file:///srv/moz/gaia/b2g_sdk/36.0a1-2014-11-06-04-02-07/b2g/components/
> nsHandlerService.js, line 120: NS_ERROR_FAILURE: Component returned failure
> code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]

This exception is harmless and already happens on master.

Comment 10

[Ricky Chien [:rickychien] (inactive)]

Error comes from Cu.import("resource://gre/modules/XPCOMUtils.jsm"); at line 5 in DirectoryProvider.js which is beautified by http://jsbeautifier.org/.
I even try it on latest b2g build but there is same problem here.

Comment 11

[Zibi Braniecki [:zbraniecki][:gandalf]]

What's the next step here? Who can help with this? It's becoming important as it blocks us from landing DOM Overlays and we'd prefer to land it early in 2.2 cycle.

Comment 12

[Ricky Chien [:rickychien] (inactive)]

I'm going to investigate this upgrade problem on OS X.

Comment 13

[Ricky Chien [:rickychien] (inactive)]

Problem comes from:

const Cu = Components.utils;
Cu.import("resource://gre/modules/XPCOMUtils.jsm");
Cu.import("resource://gre/modules/Services.jsm");

NS_ERROR_FILE_NOT_FOUND occurred in Cu.import(...) whichever modules in resource://gre/modules/* on Mac.
I'm sure both XPCOMUtils.jsm and Services.jsm between b2g 34.0a1 and b2g 36.0a1 are same. Still cannot find the root cause.

Alex, do you know who is good at this part can help us?

Comment 14

[Ricky Chien [:rickychien] (inactive)]

(In reply to Ricky Chien [:rickychien] from comment #13)
> Problem comes from:
> 
> const Cu = Components.utils;
> Cu.import("resource://gre/modules/XPCOMUtils.jsm");
> Cu.import("resource://gre/modules/Services.jsm");

This code is in DirectoryProvider.js

Comment 15

[Ricky Chien [:rickychien] (inactive)]

B2G broke on Mac since 2014/09/2014-09-30-16-02-07 build [1].


[1] http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014/09/2014-09-30-16-02-07-mozilla-central/

Comment 16

[Ricky Chien [:rickychien] (inactive)]

Hi Stephen,

I saw there some commits relate to Mac so that caused failures when running |make| on Mac (see also comment 13). This regression occur on 2014/09/2014-09-30-16-02-07 b2g_sdk 35.0a1 and I find out these changesets in [1] could be the protential risk to this issue.

Do you have any idea about that? Or maybe someone can help with us? Thanks!

[1] http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=7c24470b6b3a&tochange=2ae57957e4bb

Comment 17

[Stephen A Pohl [:spohl]]

The meaning of the GreD (aka gre) has changed, i.e. it's now pointing at Contents/Resources. I suspect that B2G.app still has all its resources files under Contents/MacOS, which is why the imports can't be resolved. Personally, I'd recommend updating the way B2G.app is packaged, as this will be required to sign with Apple's v2 signatures (I'm assuming that's something we'd want to do eventually). See bug 1046906 and dependent bugs to see what was done for Firefox Desktop.

A temporary workaround might be to make imports read from GreBinD (which points to Contents/MacOS on Mac), but this will prevent B2G.app from being signed with Apple v2 signatures.

Comment 18

[Ricky Chien [:rickychien] (inactive)]

How to specify GreBinD? I tried Cu.import("resource://gre/bin/modules/XPCOMUtils.jsm"); but in vain.

I move Contents/MacOS/modules to Contents/Resources/modules and it works, even though b2g crash in the end. I think that it's possible that I miss something, but in my experiment has proved your right.

There are so many utils.import() which point to GreD currently. Are there any bug dealing with updating B2G.app package now? I think we should set dependency for it.

Comment 19

[Ricky Chien [:rickychien] (inactive)]

(In reply to Ricky Chien [:rickychien] from comment #18)
> I move Contents/MacOS/modules to Contents/Resources/modules and it works,
> even though b2g crash in the end. I think that it's possible that I miss
> something, but in my experiment has proved your right.

After moving modules/, I also move Contents/MacOS/res to Contents/Resources/res can work properly.

Comment 20

[Stephen A Pohl [:spohl]]

(In reply to Ricky Chien [:rickychien] from comment #18)
> How to specify GreBinD? I tried
> Cu.import("resource://gre/bin/modules/XPCOMUtils.jsm"); but in vain.

Right, this won't work since conceptually, you're actually looking for a path equivalent to "resource://gre/../MacOS/modules/XPCOMUtils.jsm". I'm pretty sure this wouldn't work though, due to the relative path. Ideally, something like |Cu.import("resource://greBinD/modules/XPCOMUtils.jsm");| would work, but greBinD may not be defined for the lookup in Components.utils.import.

> There are so many utils.import() which point to GreD currently.

Since Components.utils.import is really used to load *resources*, this is actually correct. Like I mentioned earlier, the proper fix would be to move the resources to Contents/Resources, but a workaround may be to find a way to load resources out of Contents/MacOS.

> Are there any bug dealing with updating B2G.app package now? I think we
> should set dependency for it.

I'm not aware of such a bug. If you do file one, please CC me on it. I have virtually no experience with B2G, but I'm happy to help out or provide feedback wherever I can.

Comment 21

[Zibi Braniecki [:zbraniecki][:gandalf]]

Just FYI, 8 days ago bug 1097450 landed and it depends on Gecko 36 (mozSettings.get API change). It's currently impossible to launch integration tests on desktop until this bug is fixed.

Comment 22

[Julien Wajsberg [:julienw]]

On Desktop on MacOS X, you mean?

Comment 23

[Ricky Chien [:rickychien] (inactive)]

Not really understand you question. :)

Problem arose from B2G desktop 36 only on Mac OS X.

Comment 24

[Julien Wajsberg [:julienw]]

FTR because we use a slightly older gecko to build, it seems we can't use some of the shorthand notations as explained on https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer

I got this issue when using a shorthand notation for method names:

Exception: SyntaxError: missing : after property id
jsComparator@resource://gre/modules/commonjs/toolkit/loader.js -> file:///home/julien/travail/git/gaia/build/utils.js:195:35
HTMLOptimizer.prototype.aggregateJsResources/scripts<@resource://gre/modules/commonjs/toolkit/loader.js -> file:///home/julien/travail/git/gaia/build/webapp-optimize.js:365:10

Comment 25

[Staś Małolepszy :stas]

Has there been any progress on this lately?  We're still using B2G SDK 34 from August 2014 on master :(  I don't have a Mac to test on; can the error from comment 6 be reproduced also with Gecko 38?

Comment 26

[Stephen A Pohl [:spohl]]

This is mainly blocked on bug 1101331.

Comment 27

[Ricky Chien [:rickychien] (inactive)]

I just tested it and unfortunately the problem still exist with Gecko 38.0a1.

Comment 28

[GitHub Autolander Bot]

Attachment: [gaia] RickyChien:upgrade-b2g-39 > mozilla-b2g:master

Comment 29

[Ricky Chien [:rickychien] (inactive)]

In case of it breaks testing again, push to try for verification

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d84baee8213d

Cross fingers and wait for try green.

Comment 30

[Ricky Chien [:rickychien] (inactive)]

In case of it breaks testing again, push to try for verification

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c2d2a7205946

Cross fingers and wait for try green.

Comment 31

[Ricky Chien [:rickychien] (inactive)]

Landed in master:

https://github.com/mozilla-b2g/gaia/commit/c278cec4edaa6dd7a01c703c7109d51e0fda9257

Comment 32

[Ryan VanderMeulen [:RyanVM]]

https://treeherder.mozilla.org/logviewer.html#?job_id=1462099&repo=b2g-inbound
06:17:47 INFO - You rock! 1 errors has been removed from apps/system/style/devtools/devtools_auth.css. Please update build/csslint/xfail.list with the updated number of errors (0) or remove the file from build/csslint/xfail.list 

Follow-up Linter fix:
Master: https://github.com/mozilla-b2g/gaia/commit/4c38360375ae2692ffc4e644bd3f56a2bf58c002

Comment 33

[Julien Wajsberg [:julienw]]

Ricky, I think the error didn't show up in the try because you need to remove the git_revision property when you specify a branch.

Comment 35

[Staś Małolepszy :stas]

OMG, you guys fixed this. I go on vacation for a week and when I come back this morning, you totally make my day.  Thanks!