Closed Bug 1531963 Opened 5 years ago Closed 5 years ago

[wpt-sync] Sync PR 15570 - [wptrunner] Add screenshot hashes to wptreport JSON along with a new formatter for screenshot data URIs

Categories

(Testing :: web-platform-tests, enhancement, P4)

Product:
Testing
Component:
web-platform-tests
Type:
enhancement

Tracking

(firefox68 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla68
Tracking Flags:
Tracking Status
firefox68 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 15570 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/15570
Details from upstream follow.

Robert Ma <robertma@chromium.org> wrote:

[wptrunner] Add screenshot hashes to wptreport JSON along with a new formatter for screenshot data URIs

Data flow:

  1. In executors, if external reftest runner is used, we directly add hashes to the reftest_screenshots field of the extra log data in RefTestImplementation.
  2. Marionette internal reftests does not use RefTestImplementation and produces log data from within Marionette, so we use reftest_result_converter to check & compute the hash if it does not exist. This introduces some additional computation inadvertently.
  3. When log data is sent to the formatter, WptreportFormatter extracts test paths and hashes from reftest_screenshots into wptreport.json in the format expected by wpt.fyi; WptscreenshotFormatter, a new formatter, extracts only the screenshots and dumps them into another file as data URIs.

The implementation should be compatible with all existing log formatters and other consumers of logs, since we only add a new field without modifying existing fields. I've checked --log-mach and --log-tbpl formatters, as well as reftest-analyzer-structured (which takes raw logs produced by --log-raw).

See individual commit messages for more detailed explanation.

Working towards web-platform-tests/wpt.fyi#57

Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7ccdd0077950
[wpt PR 15570] - [wptrunner] Add screenshot hashes to wptreport JSON along with a new formatter for screenshot data URIs, a=testonly

https://hg.mozilla.org/mozilla-central/rev/7ccdd0077950

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox68: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.