1307114 - Make sure gecko.log is provided as a log artifact

Status: RESOLVED INVALID

(Testing :: Mochitest, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

In bug 1274255 we lacked the logging to help determine what was wrong, because gecko.log isn't provided as an artifact (or isn't written at all, I'm not sure).

That logging should be made visible in the task artifacts.

Comment 1

[David Burns :automatedtester]

The marionette harness does this already. When you say "not provided as an artifact" do we need to do a mozharness change?

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Looking at
  https://tools.taskcluster.net/task-inspector/#UM_BpNHkTBS0kb_xwcgbDQ/0
I don't see a gecko.log artifact, and searching the main (stdout) log for that task doesn't even show any mention of `gecko.log`.  Grepping for `gecko.log` in the source shows mozharness code to write it:

testing/mozharness/scripts/marionette.py:                                                       'gecko.log'))
testing/mozharness/scripts/marionette.py:            # .. or gecko.log if it exists
testing/mozharness/scripts/marionette.py:            gecko_log = os.path.join(self.config['base_work_dir'], 'gecko.log')
testing/mozharness/scripts/marionette.py:                self.info('dumping gecko.log')
testing/mozharness/scripts/marionette.py:                self.info('gecko.log not found')

and it looks like it defaults to writing it in the current directory:

testing/marionette/harness/marionette/runner/base.py:                               " to write to stdout. Default: './gecko.log'")
testing/marionette/harness/marionette/runner/base.py:        # If no workspace is set, default location for gecko.log is .
testing/marionette/harness/marionette/runner/base.py:            self.gecko_log = os.path.join(self.workspace_path or '', 'gecko.log')

so I'm guessing that file should, instead, be written to the artifacts directory (~/artifacts), whether by a mozharness change or a change to the task definition I don't know.

Comment 3

[David Burns :automatedtester]

Maja, can you have a quick look into this?

Comment 4

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

The mochitest harness starts a firefox binary and then connects Marionette client to that binary, so no gecko.log is produced. Since the Marionette class doesn't manage the firefox binary, it doesn't control logging. The mochitest harness would have to set prefs on the firefox binary that it's starting.

In any case, I think the best way to go is to have any marionette logging dumped to stdout (thus including it in log_info, etc) since marionette is started many times during a mochitest run. (That is, it might be easier to read that way that to have all the marionette sessions collected in a separate log file, or have one log file per session.)

Comment 5

[Andrew Halberstadt [:ahal]]

I'm not aware of mochitests ever having saved a 'gecko.log' file before, I think that's specific to the Mn jobs via mozharness (not something inherent in marionette itself). For mochitests, gecko output should just be interleaved in the normal log and that should include marionette initialization (e.g ctrl-F a mochitest log for "Listening on port 2828" and you should find it).

Dustin, is that what you were looking for? Assuming this was filed for mochitest/reftest, I think this bug is invalid, but I'll wait for clarification before closing.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

I was looking for data to debug bug 1274255, and :AutomatedTester indicated I'd find it in gecko.log.  So if either that data is already available somewhere and we just didn't find it, or if there is no such data, then yes this is invalid.  Otherwise, if there's a logfile somewhere in the task run that is not being made available for analysis after the run completes, then there's something to do here.

Comment 7

[Andrew Halberstadt [:ahal]]

Ok, then this is invalid. To clarify:

* Mn jobs upload gecko.log because the main log doesn't contain gecko output
* Mochitest/reftest jobs do not upload gecko.log because gecko output is interleaved in the main log

The confusion seemed to be around whether we were talking about Mn or mochitest/reftest.