Closed Bug 1501619 Opened 6 years ago Closed 6 years ago

[wpt-sync] Sync PR 13698 - [run_web_tests] Check for extra baselines

Categories

(Core :: Web Audio, enhancement, P4)

Product:
Core
Component:
Web Audio
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla65
Tracking Flags:
Tracking Status
firefox65 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream]) 

Sync web-platform-tests PR 13698 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/13698
Details from upstream follow.

Xianzhu Wang <wangxianzhu@chromium.org> wrote:
>  [run_web_tests] Check for extra baselines
>  
>  After https://chromium-review.googlesource.com/c/chromium/src/+/1277654,
>  we no longer generate text result of many tests, thus these tests no
>  longer need the text baselines.
>  
>  This CL checks for extra baselines in the following cases:
>  1. Extra -expected.png (existing logic) for ref tests;
>  2. Extra -expected.txt, -expected.png, -expected.wav if the test doesn't
>     generate the result;
>  3. Extra -expected.txt for all-pass testharness tests.
>  
>  For 2, the extra baseline will be treated as a mismatch failure (which
>  can be interpreted as that the test doesn't generate the result, but
>  the baseline expects the result).
>  
>  For 1 and 3, there are just warning messages because for now we don't
>  have a good way to present them in the layout test result page.
>  
>  We have already had a similar logic in pre-submit for 3 and the new
>  check in run_web_tests is more accurate. For other types of extra
>  baselines, it's hard to implement the check in presubmit because it
>  needs more knowledge than static pre-submit knows.
>  
>  Bug: 703899
>  Cq-Include-Trybots: luci.chromium.try:linux-blink-gen-property-trees;luci.chromium.try:linux_layout_tests_layout_ng;luci.chromium.try:linux_layout_tests_slimming_paint_v2;master.tryserver.blink:linux_trusty_blink_rel
>  Change-Id: I9016d5583791f179f1d51561245602af615d21de
>  Reviewed-on: https://chromium-review.googlesource.com/1286894
>  WPT-Export-Revision: 23f955341070ed3f7f92c588242674cc8ad2d713
PR 13698 applied with additional changes from upstream: 4e2ecda993fcfd636c09f61421c8b0a4244e8113
Component: web-platform-tests → Web Audio
Product: Testing → Core
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Ran 1 tests and 14 subtests
OK     : 1
PASS   : 14
Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8e42fe25838c
[wpt PR 13698] - [run_web_tests] Check for extra baselines, a=testonly
https://hg.mozilla.org/mozilla-central/rev/8e42fe25838c
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox65: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
You need to log in before you can comment on or make changes to this bug.