908698 - Gaia unit tests: /bin/bash: ./xulrunner-sdk-26/bin/run-mozilla.sh: No such file or directory

Status: RESOLVED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

Gaia unit tests are currently red on TBPL with the following error:

08:43:52     INFO - Copy/paste: make
08:43:52     INFO - Using env: {'DEBUG': '1', 'DESKTOP': '0', 'NOFTU': '1', 'USE_LOCAL_XULRUNNER_SDK': '1'}
08:43:53     INFO -  test -d profile-debug || mkdir -p profile-debug
08:43:53     INFO -  run-js-command  applications-data
08:43:53     INFO -  /bin/bash: ./xulrunner-sdk-26/bin/run-mozilla.sh: No such file or directory
08:43:53     INFO -  make: *** [applications-data] Error 127

This is happening because gaia now wants xulrunner-sdk-26 for make, but the slaves are downloading an earlier version.

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: Use xulrunner-sdk-26 folder in gaia-unit script,

Gaia now expects the xulrunner-sdk to be in a xulrunner-sdk-26 folder.  I might also need to upload a new xulrunner-sdk to tooltool, but we can see if this is enough to fix the tests, first.

Comment 2

[Julien Wajsberg [:julienw]]

So, this is due to bug 906316, sorry about that.

Jonathan, if you merely run "make" the correct xulrunner should be downloaded and extracted in the correct directory. Or maybe the TBPL VM don't have access to Internet ?

Comment 3

[Julien Wajsberg [:julienw]]

Ah I get it, it's because of "USE_LOCAL_XULRUNNER_SDK". Maybe we should go on and use the old "xulrunner-sdk" directory if USE_LOCAL_XULRUNNER_SDK is set. Or maybe even use a path set by USE_LOCAL_XULRUNNER_SDK ? What do you think ?

Comment 4

[Jonathan Griffin (:jgriffin)]

(In reply to Julien Wajsberg [:julienw] from comment #2)
> So, this is due to bug 906316, sorry about that.
> 
> Jonathan, if you merely run "make" the correct xulrunner should be
> downloaded and extracted in the correct directory. Or maybe the TBPL VM
> don't have access to Internet ?

Yes, it isn't that easy in buildbot; the build slaves cannot download the xulrunner-sdk from the internet, so we set USE_LOCAL_XULRUNNER_SDK=1 and pre-download it before invoking 'make'.  Currently, we're downloading an old version, and putting it in the wrong location, so this pre-seeding isn't working.

Comment 5

[Jonathan Griffin (:jgriffin)]

> Or maybe even use a path set by USE_LOCAL_XULRUNNER_SDK ? What do you think ?

This option would definitely make the automation more robust in the future.

Comment 6

[Julien Wajsberg [:julienw]]

Do you know if there are other users of USE_LOCAL_XULRUNNER_SDK ?

Comment 7

[Jonathan Griffin (:jgriffin)]

No idea!  If we're concerned about breaking someone else, we could introduce a different env variable.

Comment 8

[Andrew Halberstadt [:ahal]]

Comment on attachment 794716 [details] [diff] [review]
Use xulrunner-sdk-26 folder in gaia-unit script,

Review of attachment 794716 [details] [diff] [review]:
-----------------------------------------------------------------

Any idea why they changed this?

Comment 9

[Julien Wajsberg [:julienw]]

I have a very simple patch to change the Makefile, using the existing XULRUNNER_DIRECTORY that you'll be able to set from the commandline.

You'll keep the existing USE_LOCAL_XULRUNNER_SDK but just specify XULRUNNER_DIRECTORY too.

Will upload in 5 minutes.

Comment 10

[Julien Wajsberg [:julienw]]

Attachment: patch v1

Now we can define XULRUNNER_DIRECTORY from the command-line. If not defined, it
will use the default xulrunner-sdk-26.
---
 Makefile |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comment 11

[Julien Wajsberg [:julienw]]

(In reply to Andrew Halberstadt [:ahal] from comment #8)

> Any idea why they changed this?

Because on different branches we might need different xulrunner runtimes (and we do these days), and the current way of doing it was redownloading Xulrunner when changing branch. Not good. :)

Comment 12

[Julien Wajsberg [:julienw]]

So with this patch you'll need to define:

* XULRUNNER_DIRECTORY=<relative path>
* USE_LOCAL_XULRUNNER_SDK=1

Would you need to use an absolute path ? If yes the change is bigger (we need to remove all "./" at the start of all paths) and would need to be well tested on all platforms, which I can't do right now.

Comment 13

[Jonathan Griffin (:jgriffin)]

No, we can use a relative path (which I assume is relative to the gaia repo?)

Comment 14

[Jonathan Griffin (:jgriffin)]

Attachment: Use xulrunner-sdk-26 folder in gaia-unit script,

New version of the patch which goes along with julienw's change to gaia in comment #12

Comment 15

[Andrew Halberstadt [:ahal]]

Comment on attachment 794748 [details] [diff] [review]
Use xulrunner-sdk-26 folder in gaia-unit script,

Review of attachment 794748 [details] [diff] [review]:
-----------------------------------------------------------------

Lgtm.

Comment 16

[Julien Wajsberg [:julienw]]

I made a travis run using the PR at https://github.com/mozilla-b2g/gaia/pull/11736.

No evidence of problems related to this patch so landing the gaia part.

master: cb1ef14cfd641ed2d6ba99402af57f28ae35dbc2

Comment 17

[Julien Wajsberg [:julienw]]

We'll likely need bug 908984 too, otherwise the email sub-build won't work with another xulrunner directory.

Comment 18

[Diego Wilson [:diego]]

(In reply to Julien Wajsberg [:julienw] from comment #6)
> Do you know if there are other users of USE_LOCAL_XULRUNNER_SDK ?

FWIW My team does use this too. Your patch helped. Thanks Julien!

Comment 19

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/build/mozharness/rev/7b73036ccfb1

Will see if this works ok on cedar.

Comment 20

[Julien Wajsberg [:julienw]]

As I said, we'll need bug 908984 too, otherwise email build won't work, but I want to test it thoroughly on various environments before merging. Probably tomorrow.

Comment 21

[Julien Wajsberg [:julienw]]

master: 2a74e6be62895a020659c44b5c7c8eb68b3695f5, reverted cb1ef14cfd641ed2d6ba99402af57f28ae35dbc2.

Comment 22

[Julien Wajsberg [:julienw]]

The reverted patch will be integrated in the new patch for bug 906316, but we'll still need the patch for TBPL, so I'm not closing this bug.

Comment 23

[Jonathan Griffin (:jgriffin)]

Use xulrunner_sdk v26 in gaia_unit:  https://hg.mozilla.org/build/mozharness/rev/9c8e61a7e7aa

Comment 24

[Jonathan Griffin (:jgriffin)]

Gaia unit tests are running again (but orange...that's for another bug) on TBPL.