Closed Bug 1599619 Opened 5 years ago Closed 4 years ago

ubuntu1804 - svg/smil/style/anim-css-font-1.svg == svg/smil/style/anim-css-font-1-ref.svg | image comparison, max difference: 255, number of differing pixels: 1520

Categories

(Core :: SVG, defect, P5)

Product:
Core
Component:
SVG
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla72
Tracking Flags:
Tracking Status
firefox72 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: dholbert)

References

Details

(Keywords: leave-open)

Attachments

(5 files)

Screen Shot 2019-11-26 at 14.25.29.png
48.80 KB, image/png
Details
possible fix v1
684 bytes, text/plain
Details
Bug 1599619: Wait for a paint before making dynamic change in SMIL reftest "anim-css-font-1.svg". r?boris
47 bytes, text/x-phabricator-request
Details | Review
0001-Bug-1599619-Wait-for-one-more-frame-before-making-dy.patch
1.80 KB, patch
Details | Diff | Splinter Review
Bug 1599619 - annotate anim-css-font-1.svg with permissive fuzzy-if for ubuntu1804
47 bytes, text/x-phabricator-request
Details | Review

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=277707835&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U8PqUNlgQYOJsYm0gCU91A/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U8PqUNlgQYOJsYm0gCU91A/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

suite: reftest
chunk: 1

context:
[task 2019-11-22T19:42:40.944Z] 19:42:40 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1.svg == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1-ref.svg
[task 2019-11-22T19:42:40.945Z] 19:42:40 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1.svg | 39 / 104 (37%)
[task 2019-11-22T19:42:41.016Z] 19:42:41 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1-ref.svg | 39 / 104 (37%)
[task 2019-11-22T19:42:41.181Z] 19:42:41 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1.svg == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1-ref.svg | image comparison, max difference: 255, number of differing pixels: 1520

In case the reftest analyzer link expires.

:dholbert - on searchfox, I see that this test has not changed since 2009(!) so chances are you may not remember much about this test, but now that I've abandoned the previous debian10 work and instead decided to focus on ubuntu1804 (for better continuity), I am seeing this test fail with quite a large difference in reftest.

The only annotation I see for this test is random-if(/^Windows\x20NT\x206\.1/.test(http.oscpu)).

Do you have any ideas as to why some of the fonts would render differently? As far as I can tell, the reference image is expecting everything to be rendered in the same way.

Flags: needinfo?(dholbert)

I think the existing "random-if" annotation is probably unrelated -- that's effectively random-if(this-is-windows-7) and was part of a mass change to mitigate all sorts of weird font/character bugs that our reftests hit specifically on Win7 (in bug 1392106).

In this failure here on Ubuntu 18.04, it looks like the testcase is rendering with all of the text chunks showing the same font, when really there's supposed to be a variety of fonts across the line, as shown in the reference case. [1]

I can actually reproduce the failure locally, in my Nightly on Ubuntu 18.10, if I load the testcase while devtools are open. There must be a race condition of some sort, which the reftest happens to also trigger on these Ubuntu 18.04 testrunner boxes. I've got a simple patch which seems to fix it locally (in my load-the-test-with-devtools-open scenario), by waiting for a paint until we make our dynamic change -- could you test the patch and see if it helps, Edwin? (I'll post it shortly.)

(BTW, I also retriggered the test run a few times on your Try run, and it failed the same way every time -- so this wasn't just a one-off issue.)

[1] Rough explanation of test: each row of the testcase is testing a particular animation, at a particular point in time, by virtue of cloning the animation and choosing a different start time for it. The whole first column is testing the state before the animation starts, so that first column should all be identical. Then, on each row, the second and third abc instances are expected to be the same (they're the font that's applied in the first half of the animation), and the fourth and fifth abc instances are also expected to be the same. (These groups may look the same if the OS uses the same font under the hood, e.g. for the system fonts named caption and icon which seem to be the same underlying font.)

Flags: needinfo?(dholbert)
Attached file possible fix v1 —

Here's a possible fix -- does this help?

Flags: needinfo?(egao)

Thanks, I've pushed to try with the ubuntu1804 image, set to run 5 times:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=05e7a7abf739697461e205bcaa3ca593d81f1574

Flags: needinfo?(egao)

Great -- looks like that does indeed address the issue. I'll post the patch for review.

This small change seems to help us come out on the good side of a race
condition that this test is failing on our Ubuntu 18.04 test runner platform.

Assignee: nobody → dholbert
Pushed by dholbert@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/da8ad3dc5fca
Wait for a paint before making dynamic change in SMIL reftest "anim-css-font-1.svg". r=boris

https://hg.mozilla.org/mozilla-central/rev/da8ad3dc5fca

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox72: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla72

I'm reopening because it turns out the fix didn't seem to have fixed the issue on ubuntu1804, as seen in this try push:
try push: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=f5136a4bdd3d759b1631b02862c51fec1c766160
reftest analyzer: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/En5gOT_8RG-s-MkBt0tChA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-12-10T07:08:55.756Z] 07:08:55     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1.svg == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1-ref.svg
[task 2019-12-10T07:08:55.756Z] 07:08:55     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1.svg | 39 / 104 (37%)
[task 2019-12-10T07:08:55.960Z] 07:08:55     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1-ref.svg | 39 / 104 (37%)
[task 2019-12-10T07:08:56.207Z] 07:08:56     INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1.svg == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/svg/smil/style/anim-css-font-1-ref.svg | image comparison, max difference: 255, number of differing pixels: 1520

Boris - I see that Daniel is on parental leave until 1/6/2019. I am partially to blame here, since I only ran the R1 push and didn't open the log to check, so it's possible the issue was never fixed with the patch. Nonetheless, could I ask you to take a look or pass the ni to someone who may know about this test?

Status: RESOLVED → REOPENED
Flags: needinfo?(boris.chiou)
Resolution: FIXED → ---

Not sure the reason, but perhaps the race condition is still there. I am trying to reproduce it locally (on my Linux 18.04.3 LTS desktop).

(In reply to Boris Chiou [:boris] from comment #12)

Created attachment 9115310 [details] [diff] [review]
0001-Bug-1599619-Wait-for-one-more-frame-before-making-dy.patch

Try to wait for one more frame.

Hi, Edwin, could you please try this patch and push to try again? I cannot reproduce locally now, so just try some tiny update.

Flags: needinfo?(boris.chiou) → needinfo?(egao)

Try push finished, issue still looks to be reproducible. Not sure why this is the case.

I changed the retry value to something a bit ridiculous (~25) and made a push to see if that helps:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&selectedJob=280631389&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Crunning%2Cpending%2Crunnable&tier=1%2C2%2C3&revision=a3dcd40db5901060f9e243273289971a0361d930

Thanks for the verification. Looking for other possible solutions.

Would it be acceptable if I temporarily marked this test with annotations that would enable both ubuntu1604 and ubuntu1804 to pass for the time being? Once the migration to ubuntu1804 is complete, I can tighten up the annotation to the exact diff/pixelcount values.

I'm ok to add the condition for this test case. And mark this bug leave-open, so we will remember to fix it in the future.

Will post a patch soon. Thanks!

Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4edd49dbcae3
annotate anim-css-font-1.svg with permissive fuzzy-if for ubuntu1804 r=jmaher
Status: REOPENED → RESOLVED
Closed: 5 years ago → 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: