935576 - [marionette-js-runner] js marionette tests are failing on tbpl

Status: RESOLVED FIXED

(Testing Graveyard :: JSMarionette, defect)

Description

[Gareth Aye [:gaye] (back from PTO)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=30210224&tree=Cedar

At a glance, it seems like this might have to do with sockit-to-me building... will flag aus and lightsofapollo.

Comment 1

[James Lal [:lightsofapollo]]

Aus is working on it

Comment 2

[Gareth Aye [:gaye] (back from PTO)]

Hey Aus - Apparently we don't allow compilers on buildbot. I don't know why this is the case, but it's definitely what's going wrong. I'd like to dig into why we don't allow compilers on test machines, but in the meantime would it be possible to pre-build sockit-to-me?

Comment 3

[Gareth Aye [:gaye] (back from PTO)]

https://github.com/mozilla-b2g/sockit-to-me/commit/360a359fca28f23d5dca753719978059412d73f8

Comment 4

[Gareth Aye [:gaye] (back from PTO)]

https://github.com/mozilla-b2g/sockit-to-me/commit/9783fe4410dc23c92612345c9f366cd4abb5fa59

Comment 5

[Aus Lacroix [:aus]]

Nice hack, Gareth! :D

Comment 6

[Gareth Aye [:gaye] (back from PTO)]

Attachment: bug_935576.patch

Hi Aki,

The gaia integration tests look in $GAIA/b2g and if there's nothing there they download the latest nightly b2g desktop. As part of the b2g desktop build, we compile a fresh b2g desktop. We should definitely make sure to use the freshly compiled one instead of downloading a nightly. This patch copies the newly built b2g into $GAIA so that the gaia integration tests will pick it up.

Comment 7

[Aki Sasaki (not active)]

Comment on attachment 830912 [details] [diff] [review]
bug_935576.patch

>+        # Copy the b2g desktop we built to the gaia directory so that it
>+        # gets used by the marionette-js-runner.
>+        copy = ' '.join([
>+            'cp',
>+            '-r',
>+            os.path.join(dirs['abs_work_dir'], 'b2g'),
>+            os.path.join(dirs['abs_gaia_dir'], 'b2g')
>+        ])
>+        self.run_command(copy, cwd=dirs['abs_work_dir'])

1) Running a run_command() with a string works.  I usually reserve it for when I want the shell to interpret something (e.g. a shell $ENV var, or a wildcard).
Otherwise, I tend to prefer using a list (basically get rid of the |' '.join()| ) since then I know any special characters won't get munged by the shell.  When the command is in the form of a list, it's like each argument is wrapped in single quotes on the command line.

2) I think this could be self.copytree().
http://hg.mozilla.org/build/mozharness/file/3edc26b689b0/mozharness/base/script.py#l292

Comment 8

[Gareth Aye [:gaye] (back from PTO)]

Attachment: bug_935576.patch

Thanks for the review Aki! Here's an updated one... would you mind making sure the inheritance from the mixin looks okay?

Comment 9

[Aki Sasaki (not active)]

Comment on attachment 830931 [details] [diff] [review]
bug_935576.patch

I'll land the gaia_integration.py part.  MercurialScript already inherits BaseScript, which inherits ScriptMixin, so the gaia_test.py change isn't needed.

Comment 10

[Aki Sasaki (not active)]

Comment on attachment 830931 [details] [diff] [review]
bug_935576.patch

https://hg.mozilla.org/build/mozharness/rev/86d6c8356fd9
Transplanted to production.

Comment 11

[John Hopkins (:jhopkins)]

in production

Comment 12

[Gareth Aye [:gaye] (back from PTO)]

18:46:28     INFO - Copy/paste: make test-integration
18:46:28     INFO - Using env: {'NPM_REGISTRY': 'http://npm-mirror.pub.build.mozilla.org',
18:46:28     INFO -  'REPORTER': 'mocha-tbpl-reporter'}
18:46:28     INFO -  /bin/bash: adb: command not found
18:46:28     INFO -  npm install --registry http://npm-mirror.pub.build.mozilla.org
18:46:45     INFO -  > sockit-to-me@0.1.8 install /builds/slave/test/gaia/node_modules/marionette-client/node_modules/sockit-to-me
18:46:45     INFO -  > ./tools/copy.js || node-gyp configure build
18:46:45     INFO -  execvp(): No such file or directory
18:46:45     INFO -  npm ERR! Error: spawn ENOENT
18:46:45     INFO -  npm ERR!     at errnoException (child_process.js:980:11)
18:46:45     INFO -  npm ERR!     at Process.ChildProcess._handle.onexit (child_process.js:771:34)
18:46:45     INFO -  npm ERR! If you need help, you may report this log at:
18:46:45     INFO -  npm ERR!     <http://github.com/isaacs/npm/issues>
18:46:45     INFO -  npm ERR! or email it to:
18:46:45     INFO -  npm ERR!     <[e-mail]>
18:46:45     INFO -  npm ERR! System Linux 3.2.0-23-generic-pae
18:46:45     INFO -  npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install" "--registry" "http://npm-mirror.pub.build.mozilla.org"
18:46:45     INFO -  npm ERR! cwd /builds/slave/test/gaia
18:46:45     INFO -  npm ERR! node -v v0.10.21
18:46:45     INFO -  npm ERR! npm -v 1.3.11
18:46:45     INFO -  npm ERR! syscall spawn
18:46:45     INFO -  npm ERR! code ENOENT
18:46:45     INFO -  npm ERR! errno ENOENT
18:46:59     INFO -  npm ERR!
18:46:59     INFO -  npm ERR! Additional logging details can be found in:
18:46:59     INFO -  npm ERR!     /builds/slave/test/gaia/npm-debug.log
18:46:59     INFO -  npm ERR! not ok code 0
18:46:59     INFO -  make: *** [node_modules/.bin/mozilla-download] Error 1
18:46:59    ERROR - Return code: 2
18:46:59     INFO - TinderboxPrint: gaia-integration-tests: <em class="testfail">T-FAIL</em>
18:46:59    ERROR - Tests exited with return code 2: harness failures
18:46:59    ERROR - # TBPL FAILURE #

^ Latest issues. Adding some more debug logs to the mozharness script.

Comment 13

[Gareth Aye [:gaye] (back from PTO)]

Attachment: gaia-integration-environment-logging.patch

Comment 14

[Gareth Aye [:gaye] (back from PTO)]

Hey Aki,

I'd like to add some logging to give me more visibility into what's happening in the gaia integration tests. Please take a look and push my commit if it looks good to you. If you could also cherry-pick/merge/encourage my patch into production so I can reap all of the debugging benefits later today, I would appreciate it!

Comment 15

[Aki Sasaki (not active)]

Comment on attachment 8348856 [details] [diff] [review]
gaia-integration-environment-logging.patch

http://hg.mozilla.org/build/mozharness/rev/11198ff7b73c
transplanted to production

Comment 16

[Gareth Aye [:gaye] (back from PTO)]

Attachment: fix-make-test-integration.patch

I just ran the script with these modifications on a loaned test slave and it fixes the issues we were having. Aki - will you push this for me? This patch fixes the script.

Comment 17

[Aki Sasaki (not active)]

Comment on attachment 8349656 [details] [diff] [review]
fix-make-test-integration.patch

http://hg.mozilla.org/build/mozharness/rev/d61cda6c25c2