1703735 - Intermittent Fission layout/svg/tests/test_context_properties_allowed_domains.html | reftest comparison: == http://example.com/tests/layout/svg/tests/file_context_fill_fallback_red.html red

Status: RESOLVED FIXED

(Core :: Layout, defect)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=335794794&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Tb6jvcBhT1GGSnMspjrFJQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Tb6jvcBhT1GGSnMspjrFJQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2021-04-08T03:39:00.624Z] 03:39:00     INFO - TEST-START | layout/svg/tests/test_context_properties_allowed_domains.html
[task 2021-04-08T03:39:01.084Z] 03:39:01     INFO - TEST-INFO | started process screentopng
[task 2021-04-08T03:39:01.343Z] 03:39:01     INFO - TEST-INFO | screentopng: exit 0
[task 2021-04-08T03:39:01.344Z] 03:39:01     INFO - Buffered messages logged at 03:39:00
[task 2021-04-08T03:39:01.344Z] 03:39:01     INFO - add_task | Entering test 
[task 2021-04-08T03:39:01.345Z] 03:39:01     INFO - Buffered messages logged at 03:39:01
[task 2021-04-08T03:39:01.345Z] 03:39:01     INFO - TEST-PASS | layout/svg/tests/test_context_properties_allowed_domains.html | First, ensure that red and green do not match. 
[task 2021-04-08T03:39:01.345Z] 03:39:01     INFO - TEST-PASS | layout/svg/tests/test_context_properties_allowed_domains.html | The allowed domain should show green. 
[task 2021-04-08T03:39:01.346Z] 03:39:01     INFO - Buffered messages finished
[task 2021-04-08T03:39:01.346Z] 03:39:01     INFO - TEST-UNEXPECTED-FAIL | layout/svg/tests/test_context_properties_allowed_domains.html | The disallowed domain should show red. 
[task 2021-04-08T03:39:01.347Z] 03:39:01     INFO -     SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
[task 2021-04-08T03:39:01.347Z] 03:39:01     INFO -     @layout/svg/tests/test_context_properties_allowed_domains.html:79:9
[task 2021-04-08T03:39:01.347Z] 03:39:01     INFO - add_task | Leaving test 
[task 2021-04-08T03:39:01.348Z] 03:39:01     INFO - GECKO(1539) | MEMORY STAT | vsize 2508MB | residentFast 116MB | heapAllocated 9MB
[task 2021-04-08T03:39:01.348Z] 03:39:01     INFO - TEST-OK | layout/svg/tests/test_context_properties_allowed_domains.html | took 494ms
[task 2021-04-08T03:39:01.349Z] 03:39:01     INFO - TEST-START | Shutdown```

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1699892

Comment 2

[Natalia Csoregi [:nataliaCs]]

Update:

There have been 33 failures since the bug has been created:

• 2 failures on Linux 18.04 x64 asan opt
• 14 failures on Linux 18.04 x64 WebRender Shippable opt
• 1 failure on Linux 18.04 x64 shippable opt
• 16 failures on Linux 18.04 x64 asan opt/debug

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=336089772&repo=autoland&lineNumber=9362

[task 2021-04-10T22:51:51.733Z] 22:51:51 INFO - TEST-PASS | layout/svg/tests/test_context_properties_allowed_domains.html | The allowed domain should show green.
[task 2021-04-10T22:51:51.733Z] 22:51:51 INFO - Buffered messages finished
[task 2021-04-10T22:51:51.734Z] 22:51:51 INFO - TEST-UNEXPECTED-FAIL | layout/svg/tests/test_context_properties_allowed_domains.html | The disallowed domain should show red.
[task 2021-04-10T22:51:51.735Z] 22:51:51 INFO - SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
[task 2021-04-10T22:51:51.735Z] 22:51:51 INFO - @layout/svg/tests/test_context_properties_allowed_domains.html:79:9
[task 2021-04-10T22:51:51.736Z] 22:51:51 INFO - add_task | Leaving test
[task 2021-04-10T22:51:51.737Z] 22:51:51 INFO - GECKO(12315) | MEMORY STAT | vsize 2508MB | residentFast 115MB | heapAllocated 10MB
[task 2021-04-10T22:51:51.739Z] 22:51:51 INFO - TEST-OK | layout/svg/tests/test_context_properties_allowed_domains.html | took 519ms

Comment 3

[Intermittent Failures Robot]

42 failures in 3854 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #33 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 33
• mozilla-central: 9

Platform and build breakdown:

• linux1804-64-qr: 18
  • debug: 2
  • opt: 16
• linux1804-64-shippable-qr: 20
  • opt: 20
• linux1804-64-asan: 3
  • opt: 3
• linux1804-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1703735&startday=2021-04-05&endday=2021-04-11&tree=all

Comment 4

[:Gijs (he/him)]

Our team is pretty swamped, Daniel, are you able to take a look here?

Comment 5

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1703735 - Use assertSnapshots to get the screenshots when the test fails. r=dholbert

This should allow us to diagnose why is it failing.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d90a911773fa
Use assertSnapshots to get the screenshots when the test fails. r=Gijs

Comment 7

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/d90a911773fa

Comment 8

[Daniel Holbert [:dholbert]]

Thanks, emilio!

Comment 9

[Daniel Holbert [:dholbert]]

Here's the treeherder intermittent-failure tracker page for this bug:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1703735&startday=2021-04-06&endday=2021-12-31&tree=all

The latest report there is from 9abc6b8301bf which is before the assertSnapshots diagnostic patch, so we're still waiting on a failure from after that change in order to know what's going on here.

One observation from looking at the failures so far: as hinted by the bug summary here, all of the failures are in this subtest: The disallowed domain should show red. There, the test is loading a document that uses context-fill in an image (which is loaded via a domain that's not on the allow-list), and we expect the image's context-fill styling to be ignored. And it's not looking how we expect. (Once we have screenshots from the patch that emilio landed, we'll hopefully know more about what that means -- i.e. whether context-fill is improperly being honored, or whether the image is looking weird for some other reason, e.g. just a fuzzy mismatch or something)

Comment 11

[Daniel Holbert [:dholbert]]

Per bug 1704589, it looks like the "disallowed domain" snapshot is actually ending up blank (i.e. just a white iframe) -- it looks like the image hasn't loaded/rendered yet there, for some reason.

So that's kind of good news? Still need to mitigate the test failure, though.

Comment 12

[Daniel Holbert [:dholbert]]

Attachment: Bug 1703735: Adjust test_context_properties_allowed_domains.html to wait for two rAFs (i.e. at least one paint) before taking screenshot of iframe. r?emilio

Previously this test was waiting for the iframe's load event; now it will wait
for one additional requestAnimationFrame tick.

This probably shouldn't be necessary, but it feels like it might help -- we'll
see after it lands. (If this does fix the issue, then we should probably file a
separate bug on what seems to be a race condition between cross-origins' load events and them actually being ready to be screenshotted/painted.)

Comment 13

[Daniel Holbert [:dholbert]]

Two notes here:
(1) As noted in the extended commit message: I'm not 100% sure if my patch will help, but if it does, then that may indicate that we have a race condition, and I'll spin off a separate bug for that.
(2) I cribbed the await new Promise(resolve => requestAnimationFrame(resolve)); pattern from this existing test:
https://searchfox.org/mozilla-central/rev/d8194cbbeaec11962ed67f83aea9984bf38f7c63/testing/web-platform/tests/css/cssom-view/resize-event-on-initial-layout.html#13
(That other test actually waits for two paints, though I don't think we need that here.)

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b85dba9383a1
Adjust test_context_properties_allowed_domains.html to wait for two rAFs (i.e. at least one paint) before taking screenshot of iframe. r=emilio

Comment 15

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b85dba9383a1

Comment 16

[Intermittent Failures Robot]

18 failures in 1076 pushes (0.017 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 15
• mozilla-central: 2
• try: 1

Platform and build breakdown:

• linux1804-64-qr: 12
  • debug: 3
  • opt: 9
• linux1804-64-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1703735&startday=2021-04-14&endday=2021-04-14&tree=all

Comment 17

[Daniel Holbert [:dholbert]]

Looks like that didn't fix it; we're still getting failure reports, in builds that include the comment 12-comment 14 patch, e.g. this one:
https://treeherder.mozilla.org/logviewer?job_id=336569959&repo=autoland&lineNumber=9368

We probably need to adjust the test to wait for two rAFs in the iframe's own inner-doc.

Comment 18

[Daniel Holbert [:dholbert]]

(and I'm mostly AFK today, but I'll try to get to that tomorrow, if nobody else gets to it first.)

Comment 19

[Daniel Holbert [:dholbert]]

Attachment: Bug 1703735: Adjust test_context_properties_allowed_domains.html to wait for a paint inside its iframe document before taking a snapshot. r?Gijs,emilio

I'm hoping this will address the intermittent failures (which seem to suggest
that we were taking the snapshot too early).

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/111a26fbef24
Adjust test_context_properties_allowed_domains.html to wait for a paint inside its iframe document before taking a snapshot. r=emilio

Comment 21

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/111a26fbef24

Comment 22

[Daniel Holbert [:dholbert]]

OK, bug 1705843 (and some additional logging to diagnose that issue) helped me figure out what's actually going wrong here: the navigation between the two URLs (from example.org to example.com) is occasionally failing.

This results in the iframe being blank (as shown the failure screenshot, noted in comment 11). And if we're relying on a script inside the iframe to tell us when we're OK to proceed, then that signal never comes and we time out (as in bug 1705843).

Happily, we can avoid this issue by simply loading each of the example.* URLs in its own dedicated iframe; I'll post a patch to do that shortly. There does seem to be a real navigation/networking issue that's involved in the test failure here, and I'll spin off a separate bug for that, with a patch that includes a test (a version of this bug's original test) that demonstrates this issue.

Comment 23

[Daniel Holbert [:dholbert]]

Attachment: Bug 1703735: Use a different iframe for each tested scenario in test_context_properties_allowed_domains.html. r?emilio

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/75b1dd5732cd
Use a different iframe for each tested scenario in test_context_properties_allowed_domains.html. r=emilio

Comment 25

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/75b1dd5732cd

Comment 26

[Intermittent Failures Robot]

51 failures in 4255 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #36 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 44
• mozilla-central: 6
• try: 1

Platform and build breakdown:

• linux1804-64-qr: 28
  • opt: 24
  • debug: 4
• linux1804-64-shippable-qr: 23
  • opt: 23

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1703735&startday=2021-04-12&endday=2021-04-18&tree=all

Comment 27

[Daniel Holbert [:dholbert]]

Looks like no reports since 4/16. Pretty sure the latest patch (landed in comment 24) fixed this. Hooray!