1620642 - reftest-analyzer improvements for analyzing test failures

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, enhancement, P3)

Description

[Bert Peers [:bpeers]]

Follow-up for Bug 1619393 for RefTest-Analyzer improvements. Some ideas:

1. better heatmap visualization -- color code the differences;
2. show a histogram of differences
3. other options to simplify finding good fuzzy ranges/values
4. quality of life improvements: print fuzzy values in copy-pasteable format; URL link to the actual tests; other ideas from comments on 1619393.

Comment 1

[Bert Peers [:bpeers]]

Attachment: overview.png

Adding an annotated screenshot to help the code review.

Comment 2

[Bert Peers [:bpeers]]

Attachment: Bug 1620642 - reftest-analyzer improvements for analyzing test failures

Turn the difference checkbox into a radio that adds "heatmap"; it uses
WebGL to show both images, their absolute difference, and a color-coded
max difference. The quadrants split following the mouse.
This helps to separate large variations (red) from small variations
(green) and helps to compare the images without losing track of where
they are.

Comment 3

[Bert Peers [:bpeers]]

Attachment: 2020-03-06 17-20-52.mkv

Adding a demo video to help the code review.
The shader/perf is not really that choppy, just encoding at 4K at high bitrate :)

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/263ce25c220a
reftest-analyzer improvements for analyzing test failures r=jgilbert

Comment 5

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/263ce25c220a