Closed Bug 1129937 Opened 7 years ago Closed 7 years ago

VERBOSE=1 is broken

Categories

(Testing Graveyard :: JSMarionette, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: daleharvey, Assigned: gaye)

Details

Attachments

(2 files)

Broken possibly by the retry patch, only broken when REPORTER=mocha-tbpl-reporter is defined (so usually works locally)
So its breaking in https://github.com/mozilla-b2g/gaia/blob/master/bin/gaia_marionette_retry.js#L85, not entirely sure what the semantics of runTest needs to be to fix this properly
Tried

  .then(function(result) {
    // Now we have to output our result and package it
    // so that it can be aggregated for final results.
    var stdout = result.stdout;

    if (stdout.indexOf('*~*~*') === -1) {
      console.log('[stdout] =>', stdout);
      return Promise.resolve({pass: 0, fail: 0, pending: 0});
    }

But failing to complete
Gareth, any suggestions here?
Flags: needinfo?(gaye)
So to add a bit more info, we added a wrapper around the integration tests on TBPL which will call marionette-mocha on individual test files (retrying on failure to mask intermittents) and parse the harness output. The wrapper (./bin/gaia_marionette_retry.js) also aggregates the output from calling marionette-mocha on all of the test files in a run. I reproduced the issue here by running

REPORTER=mocha-tbpl-reporter VERBOSE=1 TEST_FILES=apps/calendar/test/marionette/day_view_test.js make test-integration

and found that the issue was that marionette-mocha (with VERBOSE=1) was exiting with no output and the child process' exit code = 8 (which sometimes indicates an uncaught exception according to http://nodejs.org/api/process.html#process_exit_codes). Still need to dig a bit deeper, but thought I'd update anyone else looking into the issue.
Flags: needinfo?(gaye)
Just to clarify, the node child process is what's created by $GAIA/bin/gaia_marionette_retry.js to run $GAIA/node_modules/.bin/marionette-mocha (the process that's failing with exit code 8)
After playing around a bit more, I'm pretty sure this is an issue with how gaia_marionette_retry.js parses cli arguments. Will have a fix shortly!
Assignee: nobody → gaye
Fixes for me and :mwargers in #fxos-automation so going to land since green on ci.
https://github.com/mozilla-b2g/gaia/commit/40ba3b4a6dc8f139978c606bcff95b915ed66b0e landed on gaia master
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.