1558849 - Intermittent /css/vendor-imports/mozilla/mozilla-central-reftests/ruby/ruby-inlinize-blocks-002.html

Status: RESOLVED FIXED

(Core :: Layout: Block and Inline, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mjzffr [at] gmail.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=250931074&repo=try
Full log: https://queue.taskcluster.net/v1/task/NQlYxiO_QEqwk2IWKvgSrg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/NQlYxiO_QEqwk2IWKvgSrg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

Observed on Windows 7 after Bug 1553584 was fixed.

Comment 2

[Daniel Holbert [:dholbert]]

This is a 1px difference, at the bottom-right antialiased fringe of an "a" character (which is on top of a red border in the testcase but occluded by that border in the reference case).

The "pure reftest" version of this test does have an annotation for 1px being fuzzy on Windows:
https://searchfox.org/mozilla-central/rev/1133b6716d9a8131c09754f3f29288484896b8b6/layout/reftests/w3c-css/submitted/ruby/reftest.list#3
...so this seems like a known issue. We can either copy over that fuzzy annotation into a wpt .ini file for this testcase, or (better) someone could investigate fixing this by tweaking the stacking order and/or specifying a sans-serif font perhaps.

Comment 3

[Intermittent Failures Robot]

24 failures in 951 pushes (0.025 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 16
• mozilla-central: 4
• try: 1
• mozilla-inbound: 3

Platform breakdown:

• windows10-64: 15
• windows10-64-shippable: 8
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-13&endday=2019-06-13&tree=all

Comment 4

[Intermittent Failures Robot]

40 failures in 859 pushes (0.047 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 20
• try: 8
• mozilla-central: 8
• mozilla-inbound: 4

Platform breakdown:

• windows10-64-shippable: 18
• windows10-64: 17
• windows10-aarch64: 1
• windows10-64-devedition: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-14&endday=2019-06-14&tree=all

Comment 5

[Intermittent Failures Robot]

18 failures in 228 pushes (0.079 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 4
• mozilla-inbound: 3
• try: 3

Platform breakdown:

• windows10-64: 13
• windows10-64-shippable: 2
• windows10-64-ccov: 1
• windows10-64-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-15&endday=2019-06-15&tree=all

Comment 6

[Andreea Pavel [:apavel]]

Retriggers range: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C10%2Cx64%2Cdebug%2Cweb%2Cplatform%2Ctests%2Ctest-windows10-64%2Fdebug-web-platform-tests-reftests-e10s-5%2Cw%28wr5%29&tochange=362cff551a019dc78cfc03047ee93d9fa72f1148&fromchange=962a2dc1dc620eec4f89f235d467aaafd6f08872&group_state=expanded&selectedJob=252145885

Maja this seems to be from bug 1553584, can you please take a look?

There are 79 total failures in the last 5 days on windows10-64 opt and debug, windows10-64-shippable opt.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=252134788&repo=mozilla-inbound&lineNumber=50314

Comment 7

[Intermittent Failures Robot]

95 failures in 4753 pushes (0.02 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• try: 16
• autoland: 46
• mozilla-central: 20
• mozilla-inbound: 13

Platform breakdown:

• windows7-32: 1
• windows10-64: 50
• windows10-64-shippable: 35
• windows10-64-devedition: 7
• windows10-aarch64: 1
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 8

[Intermittent Failures Robot]

15 failures in 200 pushes (0.075 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• mozilla-central: 4
• try: 2

Platform breakdown:

• windows10-64: 8
• windows10-64-ccov: 2
• windows10-64-shippable: 4
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-17&endday=2019-06-17&tree=all

Comment 9

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Based on Comment 2, I think the right thing is for someone from dholbert's team to decide how to fix this intermittent.

Comment 10

[Daniel Holbert [:dholbert]]

Attachment: Bug 1558849: Use sans-serif font in reftest ruby-inlinize-blocks-002.html to reduce likelihood of fuzziness. r?jfkthame

We're seeing occasional fuzzy-mismatch failures with a single pixel differing,
where an antialiased "a" serif overlaps an adjacent table border. Hopefully a
sans-serif font will reduce the likelihood of this fuzziness.

Comment 11

[Daniel Holbert [:dholbert]]

The attached patch should hopefully fix the copy of this test that lives in layout/reftests. (And I'm removing the fuzzy-if-windows annotation there, to see if it actually helps.)

Assuming it fixes the fuzziness in that copy of the test, then this bug here (with fuzzy mismatches in the WPT copy) will be addressed after this patch is landed, and its changes are synchronized to upstream wpt/css/vendor-imports, and then synchronized back into our WPT import.

Here's a Try run to test these changes to the layout/reftests copy of this test:
[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=3ef70a5d0c838c554218299c0bc1c7bdeef7e015

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/56f5a5873ad9
Use sans-serif font in reftest ruby-inlinize-blocks-002.html to reduce likelihood of fuzziness. r=jfkthame

Comment 13

[Daniel Holbert [:dholbert]]

Triggered autoland.

I'm also adding "leave-open" because it'll probably be a week or so until this is actually fully fixed in the WPT copy of this test (via the synchronization upstream & back referred to in comment 11).

Comment 14

[Intermittent Failures Robot]

24 failures in 437 pushes (0.055 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 13
• mozilla-central: 6
• try: 3
• mozilla-inbound: 2

Platform breakdown:

• windows10-64-shippable: 8
• windows10-64: 13
• windows10-64-ccov: 2
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-18&endday=2019-06-18&tree=all

Comment 15

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/56f5a5873ad9

Comment 16

[Intermittent Failures Robot]

43 failures in 614 pushes (0.07 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 27
• mozilla-central: 7
• mozilla-inbound: 5
• try: 4

Platform breakdown:

• windows10-64-shippable: 24
• windows10-64: 17
• windows10-64-ccov: 1
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-19&endday=2019-06-19&tree=all

Comment 17

[Intermittent Failures Robot]

21 failures in 562 pushes (0.037 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 8
• mozilla-central: 4
• mozilla-inbound: 8
• try: 1

Platform breakdown:

• windows10-64-shippable: 12
• windows10-64: 8
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-20&endday=2019-06-20&tree=all

Comment 18

[Intermittent Failures Robot]

30 failures in 515 pushes (0.058 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-inbound: 3
• try: 2
• autoland: 13
• mozilla-central: 12

Platform breakdown:

• windows10-64: 17
• windows10-64-shippable: 8
• windows10-64-ccov: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-21&endday=2019-06-21&tree=all

Comment 19

[Daniel Holbert [:dholbert]]

We might as well annotate this test to stop the intermittent failures until the synchronization described in comment 11 has completed.

Actually, it looks like we did already annotate this test as "disabled", here:
https://hg.mozilla.org/mozilla-central/rev/c271ae1b60b9043ee090a2f6c46f3003093ce6b2#l4.3
...but the annotation is extremely specific: only on win32 / x86, with a particular version 6.1.7601 [of windows?].

I think this fuzzy failure really affected all windows builders (regardless of version/architecture). So the failure reports that have been getting through are for the jobs that aren't covered by that extremely-specific annotation -- namely, 64-bit windows testruns.

Let's just generify the annotation to make this test disabled on windows entirely, and then we can reenable it (removing the .ini file) in a week or so once the comment 11 synchronization process has completed.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/da96f37538d8
Temporarily disable web-platform-test ruby-inlinize-blocks-002.html on more categories of Windows testers. (no review, test-annotation-only)

Comment 21

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/da96f37538d8

Comment 22

[Intermittent Failures Robot]

147 failures in 2581 pushes (0.057 failures/push) were associated with this bug in the last 7 days.

This is the #3 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 74
• mozilla-central: 39
• try: 14
• mozilla-inbound: 20

Platform breakdown:

• windows10-64: 68
• windows10-64-ccov: 12
• windows10-64-shippable: 62
• windows10-64-devedition: 4
• windows10-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558849&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 23

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Import into WPT in https://github.com/web-platform-tests/wpt/pull/17541

Comment 24

[Daniel Holbert [:dholbert]]

Just an update: the synchronization process hasn't quite completed yet, so the .ini file can't be removed yet.

We're waiting on the next WPT-->mozilla-central sync, in order for https://searchfox.org/mozilla-central/source/testing/web-platform/tests/css/vendor-imports/mozilla/mozilla-central-reftests/ruby/ruby-inlinize-blocks-002.html to get a font: 16px sans-serif; declaration. After that happens, we can remove the .ini file and close out this intermittent-failure bug as fixed.

Comment 25

[Daniel Holbert [:dholbert]]

Another update, since it's been a few weeks: we're still waiting for a sufficiently-new WPT update, before we can remove the .ini file that skips this test.

We did have a recent WPT update in bug 1561193, but that one didn't update us far enough to have picked up these changes, apparently.

Just as a sanity-check -- the version of this test on github is indeed up to date, so the changes did make it that far (via comment 23's pull):
https://github.com/web-platform-tests/wpt/blob/master/css/vendor-imports/mozilla/mozilla-central-reftests/ruby/ruby-inlinize-blocks-002.html#L11

So I imagine the next WPT update will get us the fixed version of this test, and then we can reenable it by removing the .ini file.

Comment 26

[Daniel Holbert [:dholbert]]

Looks like bug 1564632 has the WPT sync that will bring in the changes that will mean we can revert comment 20 and close this out. I'll look into doing that later on today, probably.

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c8ebaec59407
part 2: re-enable web-platform-test ruby-inlinize-blocks-002.html now that it's received a fix to hopefully prevent its fuzzy mismatch. (no review, test-annotation-only)

Comment 28

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/c8ebaec59407