Closed Bug 1093045 Opened 5 years ago Closed 3 years ago

Generate and upload HTML logs for Marionette based tests

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1275243

People

(Reporter: davehunt, Unassigned, Mentored)

Details

We now have a HTML log option available through mozlog, which is very similar to GaiaTest's own HTML report. We should enable this for all Marionette based tests to assist with diagnosing failures - the report contains screenshots and page source immediately following any failed tests.

To enable this we need the py [1] dependency installed and to add the --log-html command line argument. We should also upload this file so that it's available to view via Treeherder.

This will create a duplicate report for Gaia UI tests, however once bug 1065410 is resolved we can remove the --html-output command line argument and switch over to the new report.
Jonathan: I'm still not entirely familiar with mozharness. Could you point me in the right direction for this? Alternatively, can you suggest someone who might be able to assist? I think it would be very useful for helping us to diagnose intermittent failures.
Looks like I missed the needinfo here.
Flags: needinfo?(jgriffin)
Hey Dave,

There are three parts to this.

1) you need to file a bug to get the py package installed on our internal pypi.  You can use bug 1028386 as a template.

2) you need to modify https://dxr.mozilla.org/build:mozharness/source/scripts/marionette.py#262 to install it; you can just call self.register_virtualenv_module(module_name) and it will be downloaded from our internal pypi

3) you need to modify https://dxr.mozilla.org/mozilla-central/source/testing/config/mozharness/marionette.py to replace --html-output with --log-html, or add the latter if you need both args.  You also need to modify https://dxr.mozilla.org/build:mozharness/source/scripts/marionette.py#348 in the same way

If you use the filename https://dxr.mozilla.org/build:mozharness/source/scripts/marionette.py#348 for this, it should automatically be uploaded by the script.  If you use a different filename, you just need to make sure it gets written to dirs['abs_blob_upload_dir']; everything in that directory is uploaded at the end of the run.
Flags: needinfo?(jgriffin)
We no longer need to depend on py since bug 1016929 was fixed. I'm happy to mentor the mozharness changes to get HTML reports using mozlog uploaded for all Marionette based tests.
Mentor: dave.hunt
Henrik: I recall seeing these HTML reports in Treeherder. Can this be closed?
Flags: needinfo?(hskupin)
Correct. This has actually been done in bug 1275243.
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(hskupin)
Resolution: --- → DUPLICATE
Duplicate of bug: 1275243
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.