1616373 - Intermittent text-shadow/text-shadow-selected-1.html == text-shadow/text-shadow-selected-1-ref.html | image comparison, max difference: 79, number of differing pixels: 3567

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: cbrindusan [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=289390631&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bXLOtNI5R2S8s4cRXWUFeQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bXLOtNI5R2S8s4cRXWUFeQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2020-02-18T19:53:43.857Z] 19:53:43 INFO - REFTEST TEST-START | layout/reftests/text-shadow/text-shadow-selected-1.html == layout/reftests/text-shadow/text-shadow-selected-1-ref.html
[task 2020-02-18T19:53:43.857Z] 19:53:43 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/text-shadow/text-shadow-selected-1-ref.html | 25 / 32 (78%)
[task 2020-02-18T19:53:43.877Z] 19:53:43 INFO - [Child 14747, Main Thread] WARNING: Trying to request nsIHttpChannel from DocumentChannel, this is likely broken: file /builds/worker/workspace/build/src/netwerk/ipc/DocumentChannel.cpp, line 64
[task 2020-02-18T19:53:43.960Z] 19:53:43 INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (79, 3567) <= (1, 1000)
[task 2020-02-18T19:53:44.089Z] 19:53:44 INFO - REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/text-shadow/text-shadow-selected-1.html == layout/reftests/text-shadow/text-shadow-selected-1-ref.html | image comparison, max difference: 79, number of differing pixels: 3567

Range of retriggers:
https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=9b9642fe35393d217a9bf2d7d702b7f22e2502d3&tochange=e3aadf80c0997bbaf491f74b31089c40adec7f0c&fromchange=fcdecbc2aeae10123529fef4717a39ca7c5f729a&selectedJob=289390631

This is almost permafailing, :hsivonenen, can you please take a look if this starts from your push?
https://hg.mozilla.org/integration/autoland/rev/a80f94ffe7260e09532e37e79671a49cc1acb77b

Comment 1

[Henri Sivonen (:hsivonen)]

This test expects window.focus() to paint immediately. This probably isn't true anymore, and the test should account for an IPC round trip by removing reftest-wait via a timeout.

Comment 2

[Henri Sivonen (:hsivonen)]

Attachment: Bug 1616373 - Use reftest-wait for text-shadow-selected-*.

Comment 3

[Henri Sivonen (:hsivonen)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=23f05e6e9a03f2b67af493d389deb7e6f21b7d4c

Comment 4

[Henri Sivonen (:hsivonen)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2ed40f49ffa6dacc4976972db93cd2da3d857638

Comment 5

[Chris Peterson [:cpeterson]]

Tracking reftest failures for Fission Nightly (M6)

Comment 6

[Timothy Nikkel (:tnikkel)]

The patch in bug 1615504 force ipc messages to be processed before taking the reftest snapshot by sending an empty message and waiting for a reply, so might fix this.

Comment 7

[Intermittent Failures Robot]

38 failures in 4045 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #25 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 28
• mozilla-central: 8
• try: 2

Platform breakdown:

• linux1804-64-qr: 38

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1616373&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 8

[Henri Sivonen (:hsivonen)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ca083205b66ceb4f2211f32f6b2c96625e73e6e6

Comment 9

[Henri Sivonen (:hsivonen)]

Ooh. layout/reftests/text-shadow/text-shadow-on-selection-1.html and layout/reftests/text-shadow/text-shadow-on-selection-2.html apparently fail to fire onfocus?

Comment 10

[Henri Sivonen (:hsivonen)]

(In reply to Henri Sivonen (:hsivonen) from comment #9)

Ooh. layout/reftests/text-shadow/text-shadow-on-selection-1.html and layout/reftests/text-shadow/text-shadow-on-selection-2.html apparently fail to fire onfocus?

Does not appear to fail for me locally when running the test individually.

Comment 11

[Intermittent Failures Robot]

38 failures in 1083 pushes (0.035 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 36
• mozilla-central: 2

Platform breakdown:

• linux1804-64-qr: 38

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1616373&startday=2020-02-27&endday=2020-02-27&tree=all

Comment 12

[Narcis Beleuzu [:NarcisB]]

There are 88 failures in the last 2 days.
All of them on linux1804-64-qr

Comment 13

[Oana Pop-Rus]

R/B : https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2C18.04%2Cx64%2Cquantumrender%2Cdebug%2Creftests%2Cwith%2Cfission%2Cenabled%2Ctest-linux1804-64-qr%2Fdebug-reftest-fis-e10s-5%2Cr-fis%28r5%29&tochange=a80f94ffe7260e09532e37e79671a49cc1acb77b&fromchange=fcdecbc2aeae10123529fef4717a39ca7c5f729a&selectedJob=289390631

Comment 14

[Intermittent Failures Robot]

53 failures in 1002 pushes (0.053 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 53

Platform breakdown:

• linux1804-64-qr: 53

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1616373&startday=2020-02-28&endday=2020-02-28&tree=all

Comment 15

[Cosmin Sabou [:CosminS]]

Henri, looks like the patch in comment 8 doesn't work.
This is now in disable recommended bug list with 156 total failures since it was filed 11 days ago: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2020-02-12&endday=2020-02-29&tree=trunk&bug=1616373
There are 4 test failing here, so disabling them would a bit extreme:

• /text-shadow-selected-1.html
• /text-shadow-selected-2.html
• /text-shadow-on-selection-1.html
• /text-shadow-on-selection-2.html

Could you have another go at this? Thank you.

Comment 16

[Andreea Pavel [:apavel]]

Attachment: Bug 1616373 - disable text-shadow-*selected-1.html/selected-2.html/on-selection-1.html/on-selection-2.html on linux debug fission. r=jmaher

Comment 17

[Timothy Nikkel (:tnikkel)]

I think the reftest harness might be getting confused somehow. When this fails we only load the ref, and not the test.

Comment 18

[Timothy Nikkel (:tnikkel)]

(In reply to Timothy Nikkel (:tnikkel) from comment #17)

I think the reftest harness might be getting confused somehow. When this fails we only load the ref, and not the test.

The reftest harness keeps some canvas's around indexed by the url and can re-use them if a url is re-used. This appears to be what is happening. So the harness isn't getting confused.

Comment 19

[Intermittent Failures Robot]

127 failures in 5233 pushes (0.024 failures/push) were associated with this bug in the last 7 days.

This is the #5 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 119
• try: 1
• mozilla-central: 7

Platform breakdown:

• linux1804-64-qr: 127

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1616373&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 20

[Cosmin Sabou [:CosminS]]

Kats, could you take a look over this since it only fails on linux webrender? Will update the disable patch in the meanwhile cause it fails really often.

Comment 21

[Timothy Nikkel (:tnikkel)]

(In reply to Cosmin Sabou [:CosminS] from comment #20)

Kats, could you take a look over this since it only fails on linux webrender? Will update the disable patch in the meanwhile cause it fails really often.

It only fails with fission, and I think we only run fission reftests on linux with webrender. My try pushes have plenty of failures without webrender (but with fission).

Comment 22

[Kartikaya Gupta (email:[e-mail])]

Per Timothy's comment it seems specific to fission, not webrender. Redirecting needinfo to :farre as this appears to be a harness problem (as opposed to a test-specific problem) with fission enabled.

Comment 23

[Timothy Nikkel (:tnikkel)]

I'm not sure if this is a (reftest) harness problem because the test doesn't use anything that would be different with fission enabled: there are no iframes in the test so AFAICT the test should be the same in fission as it is in e10s mode. And bug 1556627 caused this failure, which did not touch the reftest harness.

Comment 24

[Kartikaya Gupta (email:[e-mail])]

(In reply to Timothy Nikkel (:tnikkel) from comment #23)

I'm not sure if this is a (reftest) harness problem because the test doesn't use anything that would be different with fission enabled: there are no iframes in the test so AFAICT the test should be the same in fission as it is in e10s mode. And bug 1556627 caused this failure, which did not touch the reftest harness.

Ah, I see. At any rate, it looks like Henri anticipated fallout based on https://bugzilla.mozilla.org/show_bug.cgi?id=1556627#c208 (I didn't read the whole bug) and is aware of this bug. It's not WR specific so I don't think I have much to add here, and I'll clear the needinfo on :farre.

Comment 25

[Timothy Nikkel (:tnikkel)]

I put printfs at all of the functions added in bug 1556627 in ContentParent/Child. In a passing run we get several calls to ContentParent::RecvSetFocusedBrowsingContext, in a failing run we do not get that (none of the aforementioned printfs are hit during these handful of reftests failing here). So seems like the problem lies with the focus handling somehow.

Comment 26

[Timothy Nikkel (:tnikkel)]

Attachment: Bug 1616373. Handle the browsing context housing the active window moving to a different content process. r=hsivonen

This is a straw man patch, it seems to fix the bug based on my debugging the problem, but I'm not sure if it's a good way to fix the problem.

My understanding of the problem is as follows. We create the main content document, first it is about:blank in content process A. When the bug happens The about:blank document has WindowRaised called on it and this makes it the active window, and then it tells the parent process that it is the active browsing context. When the bug doesn't happen this doesn't happen, perhaps because we navigate away from about:blank before it can happen.

The browsing context navigates to the new document and it gets moved (?) to a different content process. BrowsingContextGroup::EnsureSubscribed is called, if the parent process got told of the active document above then we call SendSetupFocusedAndActive. When the child recieves this it calls SetActiveBrowsingContextFromOtherProcess on the focus manager.

nsFocusManager::WindowRaised is called for the new document, we bail early because of this line

https://searchfox.org/mozilla-central/rev/d5b34cc8872177d3ee071e06f787c2a14268595b/dom/base/nsFocusManager.cpp#661

because we are the browsing context that was set from another process. And so we fail to make the Focus call at the end of that function. So mFocusedWindow remains null in the child process.

The nsFocusManager::WindowShown call that happens for every new document load thus does not do anything because this line

https://searchfox.org/mozilla-central/rev/d5b34cc8872177d3ee071e06f787c2a14268595b/dom/base/nsFocusManager.cpp#906

always fails because mFocusedWindow is null and it never changes.

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2b21fa5970ca
Handle the browsing context housing the active window moving to a different content process. r=hsivonen

Comment 28

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/2b21fa5970ca

Comment 29

[Cosmin Sabou [:CosminS]]

Just to be on the safe side I've retriggered here and indeed the patch is working: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=linux%2C18.04%2Cx64%2Cquantumrender%2Cdebug%2Creftests%2Cwith%2Cfission%2Cenabled%2Ctest-linux1804-64-qr%2Fdebug-reftest-fis-e10s-2%2Cr-fis%28r2%29&revision=2b21fa5970ca6903b443e8e1f3cd761c9c07b529&selectedJob=291773540
Thanks Timothy for solving this.

Comment 30

[Timothy Nikkel (:tnikkel)]

Ah, sorry, I should have mentioned that I re-triggered many more times than that on try to verify before I even posted the patch here.

Comment 31

[Intermittent Failures Robot]

35 failures in 5483 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

This is the #32 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 32
• mozilla-central: 2
• try: 1

Platform breakdown:

• linux1804-64-qr: 35

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1616373&startday=2020-03-02&endday=2020-03-08&tree=all