1857798 - Perma [tier 2] /html/user-activation/navigation-state-reset-crossorigin.sub.html | Post-navigation state reset. - Test timed out

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=431744391&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/P0qqNHGeS0OxSzwQI7AOJw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-10-08T12:18:43.369Z] 12:18:43     INFO - TEST-START | /html/user-activation/navigation-state-reset-crossorigin.sub.html
[task 2023-10-08T12:18:43.375Z] 12:18:43     INFO - Closing window cf9652a3-389a-4395-9dc4-965f537e1824
[task 2023-10-08T12:18:46.263Z] 12:18:46     INFO - 
[task 2023-10-08T12:18:46.263Z] 12:18:46     INFO - TEST-UNEXPECTED-TIMEOUT | /html/user-activation/navigation-state-reset-crossorigin.sub.html | Post-navigation state reset. - Test timed out
[task 2023-10-08T12:18:46.263Z] 12:18:46     INFO - TEST-TIMEOUT | /html/user-activation/navigation-state-reset-crossorigin.sub.html | took 2896ms
[task 2023-10-08T12:18:46.267Z] 12:18:46     INFO - PID 26663 | 1696767526266	Marionette	INFO	Stopped listening on port 43156
[task 2023-10-08T12:18:46.716Z] 12:18:46     INFO - PID 26663 | console.error: ({})
[task 2023-10-08T12:18:46.898Z] 12:18:46     INFO - Browser exited with return code 0

Comment 1

[Natalia Csoregi [:nataliaCs]]

Seems to be caused by Bug 1791079.
Oliver, can you take a look please?
Thank you.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1791079

Comment 3

[Oliver Medhurst [:canadahonk]]

I know Chromium has this flaky as well so I am pretty sure the test itself just is. I'll mark it as so.

Comment 4

[Intermittent Failures Robot]

6 failures in 4018 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 6

Platform and build breakdown:

• linux1804-64-shippable-qr: 4
  • opt: 4
• windows11-64-2009-shippable-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1857798&startday=2023-10-02&endday=2023-10-08&tree=all

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Please note that there are three other tests as well that seem to be flaky / perma failing for this revision:

https://treeherder.mozilla.org/jobs?repo=mozilla-central&revision=fba49989dde6aa32244c622fd43abe19a32128ae&searchStr=wpt&selectedTaskRun=P0qqNHGeS0OxSzwQI7AOJw.0

Comment 6

[Oliver Medhurst [:canadahonk]]

Attachment: Bug 1857798 - Mark user activation WPTs as flaky

These tests can generally be flaky (Chromium experiences the same issues)

Marked as [..., TIMEOUT, NOTRUN]

Comment 7

[Oliver Medhurst [:canadahonk]]

FYI Patch also updates the tests Henrik posted, not just this bug's singular.

Comment 8

[Peter Van der Beken [:peterv]]

Is there an upstream bug filed on making the tests less flaky?

Comment 9

[Oliver Medhurst [:canadahonk]]

Not afaik. Chromium bug 1, Chromium bug 2 for reference.

Comment 10

[Intermittent Failures Robot]

36 failures in 3764 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #44 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 36

Platform and build breakdown:

• linux1804-64-shippable-qr: 16
  • opt: 16
• windows11-64-2009-shippable-qr: 9
  • opt: 9
• macosx1015-64-shippable-qr: 11
  • opt: 11

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1857798&startday=2023-10-09&endday=2023-10-15&tree=all

Comment 11

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Hi Oliver, following Peter's comment 8, I think it's worthwhile doing a bit further investigation to see what's up in the test cases themselves, or at minimum, we should file an upstream bug for this. Also, while interop-2024 selection process isn't done, https://github.com/web-platform-tests/interop/issues/428 receives lots of interests and Safari folks are fixing some testing failures. So having an upstream issue, may help the selection and the scoping.

Comment 12

[Oliver Medhurst [:canadahonk]]

Found this WPT GH issue on it generally, but it was closed as fixed last year but still flaky in both Gecko and Chromium. I'll open a new one referencing it.

Comment 13

[Oliver Medhurst [:canadahonk]]

Filed upstream: https://github.com/web-platform-tests/wpt/issues/42629

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8145c0b27aa4
Mark user activation WPTs as flaky r=dom-core,edgar

Comment 15

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/8145c0b27aa4

Comment 16

[Norisz Fay [:noriszfay]]

This is still happening as seen here

Comment 17

[Intermittent Failures Robot]

50 failures in 3484 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

This is the #21 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 50

Platform and build breakdown:

• macosx1015-64-shippable-qr: 22
  • opt: 22
• linux1804-64-shippable-qr: 18
  • opt: 18
• windows11-64-2009-shippable-qr: 10
  • opt: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1857798&startday=2023-10-16&endday=2023-10-22&tree=all

Comment 18

[Oliver Medhurst [:canadahonk]]

Attachment: Bug 1857798 - Mark user activation WPTs as passing for fission off

There were still intermittent fails in opt/release.

Comment 19

[Oliver Medhurst [:canadahonk]]

Submitted new patch to mark as flaky for non-debug too.

Comment 20

[Cosmin Sabou [:CosminS]]

(In reply to Oliver Medhurst [:canadahonk] from comment #19)

Submitted new patch to mark as flaky for non-debug too.

They don't fail on debug, the problem from the beginning was that there are some subtests that don't have specific expectations.
These should be it: https://hg.mozilla.org/try/rev/57b39b848b270b5701551bdf260dfd234db465db
Latest patch won't get rid of those failures.

Unfortunately when testing that try rev I've encountered a bug in the wpt harness: https://treeherder.mozilla.org/logviewer?job_id=433339386&repo=try, filed Bug 1860642.

Comment 21

[Edgar Chen [:edgar]]

So I wonder if this is caused by the format error, could you try if

 [navigation-state-reset-crossorigin.sub.html]
   # There is a webdriver bug (Bug 1856989) which breaks cross-process iframe clicks
   bug: https://github.com/web-platform-tests/wpt/issues/42629
   expected:
     if fission: TIMEOUT
     if not debug: [OK, TIMEOUT, NOTRUN] # Flaky (Bug 1857798)

works?

Comment 22

[Cosmin Sabou [:CosminS]]

I have these on try for now: https://hg.mozilla.org/try/file/f6594c1fa840831f03f505e796a492a34ad44e55/testing/web-platform/meta/html/user-activation/navigation-state-reset-crossorigin.sub.html.ini
I'm expecting these to work, if they don't will give it another go. Try push: https://treeherder.mozilla.org/jobs?repo=try&group_state=expanded&revision=e60e1aa0e75db1b90f57c0ef81132da874a609bb&searchStr=backlog

Comment 23

[Cosmin Sabou [:CosminS]]

(In reply to Edgar Chen [:edgar] (OOO ~ 10/29) from comment #21)

So I wonder if this is caused by the format error, could you try if

 [navigation-state-reset-crossorigin.sub.html]
   # There is a webdriver bug (Bug 1856989) which breaks cross-process iframe clicks
   bug: https://github.com/web-platform-tests/wpt/issues/42629
   expected:
     if fission: TIMEOUT
     if not debug: [OK, TIMEOUT, NOTRUN] # Flaky (Bug 1857798)

works?

Looks like neither updates do not work.
https://hg.mozilla.org/try/rev/1639314b2237c4b772fd5598f35e37f6a93eb918 still has these failures https://treeherder.mozilla.org/logviewer?job_id=433503496&repo=try - try push

https://hg.mozilla.org/try/rev/f6594c1fa840831f03f505e796a492a34ad44e55 still has timeouts in navigation-state-reset-crossorigin.sub.html | Post-navigation state reset. and propagation-same-and-cross-origin.sub.html | Clicking on the cross-origin navigable activates parent navigable even though their expectation is set to TIMEOUT. https://treeherder.mozilla.org/logviewer?job_id=433488674&repo=try - try push
at least in this latter case the one set to NOTRUN (propagation-crossorigin.sub.html | Propagation test ) is gone.
The only thing I can think of for the other two that timeout is disabling because any other expectation does not seem to work.

Comment 24

[Edgar Chen [:edgar]]

Okay, so it seems we still need to mark some sub-test properly, I think the patch in comment#22 doesn't work because of the lack of . at the end of sub-test name, https://searchfox.org/mozilla-central/rev/e4afef5d3ff67781dc1377912344694f3cf3a226/testing/web-platform/tests/html/user-activation/navigation-state-reset-crossorigin.sub.html#50?

Comment 25

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Yes, the dot is necessary. Otherwise the sub test name doesn't match and the entry is ignored.

Comment 26

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1791079

Comment 27

[Oliver Medhurst [:canadahonk]]

Changed patch to just generally mark them as flaky, as it seems more reasonable then trying to take into account various scenarios. We can always restrict later, better to just fix now imo.

Comment 28

[Cosmin Sabou [:CosminS]]

The failures here should go away after the latest wpt-sync: https://hg.mozilla.org/integration/autoland/rev/8923dca93b808a23856287229fc8f1c0c1d1e20e

Comment 29

[Natalia Csoregi [:nataliaCs]]

Failure changed to TEST-UNEXPECTED-PASS | /html/user-activation/navigation-state-reset-crossorigin.sub.html | Post-navigation state reset. - expected TIMEOUT

Comment 30

[Oliver Medhurst [:canadahonk]]

Patch for this bug is needed to fix unexpected pass.

Comment 31

[Intermittent Failures Robot]

39 failures in 3657 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 39

Platform and build breakdown:

• linux1804-64-shippable-qr: 10
  • opt: 10
• macosx1015-64-shippable-qr: 10
  • opt: 10
• windows11-64-2009-shippable-qr: 5
  • opt: 5
• android-em-7-0-x86_64-shippable-qr: 6
  • opt: 6
• android-em-7-0-x86_64-shippable-lite-qr: 8
  • opt: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1857798&startday=2023-10-23&endday=2023-10-29&tree=all

Comment 32

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/db619dbaec0a
Mark user activation WPTs as passing for fission off r=dom-core,edgar

Comment 33

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/db619dbaec0a

Comment 34

[Intermittent Failures Robot]

6 failures in 3356 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 6

Platform and build breakdown:

• android-em-7-0-x86_64-shippable-lite-qr: 3
  • opt: 3
• android-em-7-0-x86_64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1857798&startday=2023-10-30&endday=2023-11-05&tree=all