Reftest analyzer doesn't work with local runs of WPT

RESOLVED FIXED in Firefox 67

Status

enhancement
RESOLVED FIXED
4 months ago
4 months ago

People

(Reporter: gsnedders, Assigned: gsnedders)

Tracking

Trunk
mozilla67
Points:
---

Firefox Tracking Flags

(firefox67 fixed)

Details

Attachments

(1 attachment)

Assignee

Description

4 months ago

It seems the reftest analyzer still doesn't work with output from WPT.

An example test for reftest-analyzer.xhtml's testsuite:

    { "name": "wpt reftests (Linux local)",
      "log": `SUITE-START | Running 1 tests
TEST-START | /css/css-backgrounds/border-image-6.html
TEST-UNEXPECTED-FAIL | /css/css-backgrounds/border-image-6.html | Testing http://web-platform.test:8000/css/css-backgrounds/border-image-6.html == http://web-platform.test:8000/css/css-backgrounds/border-image-6-ref.html
REFTEST   IMAGE 1 (TEST): data:image/png;base64,
REFTEST   IMAGE 2 (REFERENCE): data:image/png;base64,
TEST-INFO took 425ms
SUITE-END | took 2s`,
      "expected": { "pass": 0, "unexpected": 1, "random": 0, "skip": 0 },
    },

Bug 1362811 changed enough for WPT reftests to work with reftest-analyzer.xhtml in automation, due to the mozharness prefix; the conclusion of bug 1265582 was to add reftest-analyzer-structured.xhtml but that can't be used without a local copy due to Bug 1530002.

It's not clear what we should fork on to support output from mozlog; should we just support TEST-UNEXPECTED-FAIL etc. without any prefix whatsoever?

I'd be fine with support it with no prefix.

Assignee

Updated

4 months ago
Assignee: nobody → geoffers+mozilla
Status: NEW → ASSIGNED

Comment 3

4 months ago
Pushed by btara@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a351f6e4fb9c
make reftest-analyzer.xhtml work with local WPT output r=dbaron

Comment 4

4 months ago
bugherder
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
You need to log in before you can comment on or make changes to this bug.