1129937 - VERBOSE=1 is broken

Status: RESOLVED FIXED

(Testing Graveyard :: JSMarionette, defect)

Description

[Dale Harvey (:daleharvey)]

Broken possibly by the retry patch, only broken when REPORTER=mocha-tbpl-reporter is defined (so usually works locally)

Comment 1

[Dale Harvey (:daleharvey)]

So its breaking in https://github.com/mozilla-b2g/gaia/blob/master/bin/gaia_marionette_retry.js#L85, not entirely sure what the semantics of runTest needs to be to fix this properly

Comment 2

[Dale Harvey (:daleharvey)]

Tried

  .then(function(result) {
    // Now we have to output our result and package it
    // so that it can be aggregated for final results.
    var stdout = result.stdout;

    if (stdout.indexOf('*~*~*') === -1) {
      console.log('[stdout] =>', stdout);
      return Promise.resolve({pass: 0, fail: 0, pending: 0});
    }

But failing to complete

Comment 3

[Dale Harvey (:daleharvey)]

Gareth, any suggestions here?

Comment 4

[Gareth Aye [:gaye] (back from PTO)]

So to add a bit more info, we added a wrapper around the integration tests on TBPL which will call marionette-mocha on individual test files (retrying on failure to mask intermittents) and parse the harness output. The wrapper (./bin/gaia_marionette_retry.js) also aggregates the output from calling marionette-mocha on all of the test files in a run. I reproduced the issue here by running

REPORTER=mocha-tbpl-reporter VERBOSE=1 TEST_FILES=apps/calendar/test/marionette/day_view_test.js make test-integration

and found that the issue was that marionette-mocha (with VERBOSE=1) was exiting with no output and the child process' exit code = 8 (which sometimes indicates an uncaught exception according to http://nodejs.org/api/process.html#process_exit_codes). Still need to dig a bit deeper, but thought I'd update anyone else looking into the issue.

Comment 5

[Gareth Aye [:gaye] (back from PTO)]

Just to clarify, the node child process is what's created by $GAIA/bin/gaia_marionette_retry.js to run $GAIA/node_modules/.bin/marionette-mocha (the process that's failing with exit code 8)

Comment 6

[Gareth Aye [:gaye] (back from PTO)]

After playing around a bit more, I'm pretty sure this is an issue with how gaia_marionette_retry.js parses cli arguments. Will have a fix shortly!

Comment 7

[GitHub Autolander Bot]

Attachment: [gaia] gaye:bug-1129937 > mozilla-b2g:master

Comment 8

[Gareth Aye [:gaye] (back from PTO)]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/28472

Patch!

Comment 9

[Gareth Aye [:gaye] (back from PTO)]

Fixes for me and :mwargers in #fxos-automation so going to land since green on ci.

Comment 10

[Gareth Aye [:gaye] (back from PTO)]

https://github.com/mozilla-b2g/gaia/commit/40ba3b4a6dc8f139978c606bcff95b915ed66b0e landed on gaia master