1875674 - Intermittent /fetch/api/cors/cors-keepalive.any.html | <multiple subtests here are failing with TEST-UNEXPECTED-PASS> | single tracking bug

Status: RESOLVED FIXED

(Core :: DOM: Networking, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=444045616&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/J93AoaCUSKq9y8_whvhEyQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2024-01-21T11:08:41.220Z] 11:08:41     INFO - TEST-PASS | /fetch/api/cors/cors-keepalive.any.html | [keepalive] Cross domain basic usage GET request in unload [cors mode] 
[task 2024-01-21T11:08:41.220Z] 11:08:41     INFO - TEST-FAIL | /fetch/api/cors/cors-keepalive.any.html | [keepalive] Cross domain different port GET request in unload [no-cors mode, server forbid CORS] - assert_equals: token should be on (stashed). expected (string) "on" but got (object) null
[task 2024-01-21T11:08:41.220Z] 11:08:41     INFO - assertStashedTokenAsync/</<@http://web-platform.test:8000/fetch/api/resources/keepalive-helper.js:96:26
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2604:25
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - Test.prototype.step_func/<@http://web-platform.test:8000/resources/testharness.js:2651:35
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - TEST-FAIL | /fetch/api/cors/cors-keepalive.any.html | [keepalive] Cross domain different port GET request in unload [no-cors mode] - assert_equals: token should be on (stashed). expected (string) "on" but got (object) null
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - assertStashedTokenAsync/</<@http://web-platform.test:8000/fetch/api/resources/keepalive-helper.js:96:26
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2604:25
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - Test.prototype.step_func/<@http://web-platform.test:8000/resources/testharness.js:2651:35
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - TEST-UNEXPECTED-PASS | /fetch/api/cors/cors-keepalive.any.html | [keepalive] Cross domain different port GET request in unload [cors mode, server forbid CORS] - expected FAIL
[task 2024-01-21T11:08:41.221Z] 11:08:41     INFO - TEST-INFO | expected FAIL
[task 2024-01-21T11:08:41.234Z] 11:08:41     INFO - .........................
[task 2024-01-21T11:08:41.236Z] 11:08:41     INFO - TEST-OK | /fetch/api/cors/cors-keepalive.any.html | took 12807ms
[task 2024-01-21T11:08:41.238Z] 11:08:41     INFO - PID 9304 | 1705835321200	Marionette	INFO	Stopped listening on port 39023

Comment 1

[Iulian Moraru]

Attachment: Bug 1875674 - mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r=jgraham,aryx,manuel

Comment 2

[Iulian Moraru]

To check the other failing subtests I will provide the following links from treeherder:

• for failures on Linux 18.04 x64 WebRender debug no-fis
• for failures on Linux 18.04 x64 WebRender asan opt simple and no-fis
• for failures on Linux 18.04 x64 CCov WebRender opt

Comment 3

[Iulian Moraru]

Other failing subtests that appeared later:

• for failures on Linux 18.04 x64 WebRender opt nofis
• for failures on Linux 18.04 x64 WebRender debug swr-nofis

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/afef685a0f5e
mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r=aryx

Comment 5

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/afef685a0f5e

Comment 6

[Sandor Molnar[:smolnar]]

Still failing on these subtest:

wpt

Comment 7

[Intermittent Failures Robot]

27 failures in 3599 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #41 most frequent failure this week.

Repository breakdown:

• mozilla-central: 21
• autoland: 6

Platform and build breakdown:

• linux1804-64-qr: 13
  • debug: 11
  • opt: 2
• linux1804-64-asan-qr: 8
  • opt: 8
• linux1804-64-ccov-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-15&endday=2024-01-21&tree=all

Comment 8

[Narcis Beleuzu [:NarcisB]]

Attachment: Bug 1875674 - Mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r?aryx

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Malte, could you take a look at these frequent test failures or redirect the request? Thank you.

Comment 10

[Malte Jürgens [:maltejur]]

I'll take a look, but I have no experience with WPT. Manuel, do you have any ideas here?

Comment 11

[Malte Jürgens [:maltejur]]

From what I can tell, this seems to be caused by https://github.com/web-platform-tests/wpt/pull/43984

Comment 12

[Intermittent Failures Robot]

21 failures in 744 pushes (0.028 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 11
• mozilla-central: 7
• mozilla-beta: 3

Platform and build breakdown:

• linux1804-64-qr: 9
  • debug: 6
  • opt: 3
• linux1804-64-asan-qr: 7
  • opt: 7
• android-em-7-0-x86_64-qr: 1
  • debug: 1
• linux1804-64-ccov-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 1
  • opt: 1
• linux1804-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-22&endday=2024-01-22&tree=all

Comment 13

[Intermittent Failures Robot]

31 failures in 789 pushes (0.039 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 3
• mozilla-central: 10
• autoland: 14
• mozilla-beta: 4

Platform and build breakdown:

• linux1804-64-qr: 16
  • debug: 12
  • opt: 4
• linux1804-64-asan-qr: 3
  • opt: 3
• linux1804-64-ccov-qr: 3
  • opt: 3
• linux1804-64-shippable-qr: 6
  • opt: 6
• linux1804-32-shippable-qr: 2
  • opt: 2
• linux1804-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-23&endday=2024-01-23&tree=all

Comment 14

[Manuel Bucher [:manuel]]

I don't have any ideas for the reason of failure yet. Maybe some timing changed. Thanks for linking the likely cause. If you are at loss with this bug and want to pass it on, necko can also take over this bug.

Comment 15

[Intermittent Failures Robot]

29 failures in 845 pushes (0.034 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• try: 3
• mozilla-beta: 8
• mozilla-central: 6

Platform and build breakdown:

• linux1804-64-qr: 10
  • debug: 8
  • opt: 2
• linux1804-64-shippable-qr: 6
  • opt: 6
• linux1804-64-asan-qr: 7
  • opt: 7
• linux1804-32-shippable-qr: 3
  • opt: 3
• android-em-7-0-x86_64-qr: 2
  • debug: 2
• linux1804-64-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-24&endday=2024-01-24&tree=all

Comment 16

[Malte Jürgens [:maltejur]]

That would be appreciated, thanks. I haven't been able to figure out much more, but a good next lead would be this quote from [e-mail] in https://crbug.com/1484680:

The fetch/api/*/keepalive WPTs are now consistently flakily timing out upstream [0], even with // META: timeout=long. PTAL and see if there's a way to deflake the tests for firefox.

Prior to https://crrev.com/c/5195267, these tests were written such that they only worked in content_shell (crbug.com/1518022), which masked the timeouts we're seeing now.

Comment 17

[Intermittent Failures Robot]

16 failures in 724 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 1
• autoland: 9
• mozilla-beta: 3
• mozilla-central: 3

Platform and build breakdown:

• linux1804-64-qr: 11
  • debug: 10
  • opt: 1
• linux1804-64-asan-qr: 3
  • opt: 3
• linux1804-64-ccov-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-25&endday=2024-01-25&tree=all

Comment 18

[Frederik Braun [:freddy]]

Yes. It seems to be a test change rather than a gekco change. If Chrome isn't looking into fixing it, we should either schedule that for ourselves or make sure there's better tests so that we can remain confident in coverage.

Comment 19

[Intermittent Failures Robot]

24 failures in 668 pushes (0.036 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-central: 6
• mozilla-beta: 4
• try: 2

Platform and build breakdown:

• android-em-7-0-x86_64-qr: 2
  • debug: 2
• linux1804-64-qr: 15
  • debug: 11
  • opt: 4
• linux1804-64-asan-qr: 3
  • opt: 3
• linux1804-32-shippable-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-26&endday=2024-01-26&tree=all

Comment 20

[Narcis Beleuzu [:NarcisB]]

Attachment: Bug 1875674 - Mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r?aryx

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7886f6e6cbd7
Mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r=aryx

Comment 22

[Intermittent Failures Robot]

23 failures in 165 pushes (0.139 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 13
• mozilla-beta: 4
• mozilla-central: 4
• try: 2

Platform and build breakdown:

• linux1804-64-asan-qr: 7
  • opt: 7
• linux1804-64-shippable-qr: 4
  • opt: 4
• android-em-7-0-x86_64-qr: 2
  • opt: 2
• linux1804-64-qr: 6
  • opt: 1
  • debug: 5
• linux1804-32-shippable-qr: 2
  • opt: 2
• linux1804-64-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-27&endday=2024-01-27&tree=all

Comment 23

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/7886f6e6cbd7

Comment 24

[Intermittent Failures Robot]

161 failures in 4034 pushes (0.04 failures/push) were associated with this bug in the last 7 days.

This is the #4 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 76
• mozilla-central: 38
• mozilla-beta: 35
• try: 12

Platform and build breakdown:

• linux1804-64-qr: 76
  • debug: 60
  • opt: 16
• linux1804-64-asan-qr: 33
  • opt: 33
• android-em-7-0-x86_64-qr: 7
  • debug: 5
  • opt: 2
• linux1804-64-ccov-qr: 8
  • opt: 8
• linux1804-64-shippable-qr: 24
  • opt: 24
• linux1804-64-devedition-qr: 4
  • opt: 4
• linux1804-32-shippable-qr: 9
  • opt: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-22&endday=2024-01-28&tree=all

Comment 25

[Narcis Beleuzu [:NarcisB]]

Attachment: Bug 1875674 - Mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r?aryx

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b41f15cc1e5f
Mark as intermittent multiple subtests in cors-keepalive.any.html on linux. r=aryx

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c50571808ab9
Fix lint error

Comment 28

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/b41f15cc1e5f
https://hg.mozilla.org/mozilla-central/rev/c50571808ab9

Comment 29

[Intermittent Failures Robot]

24 failures in 698 pushes (0.034 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 13
• mozilla-beta: 5
• mozilla-central: 6

Platform and build breakdown:

• linux1804-64-qr: 10
  • debug: 6
  • opt: 4
• linux1804-64-asan-qr: 5
  • opt: 5
• linux1804-64-shippable-qr: 3
  • opt: 3
• android-em-7-0-x86_64-shippable-qr: 2
  • opt: 2
• linux1804-64-ccov-qr: 3
  • opt: 3
• android-em-7-0-x86_64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-29&endday=2024-01-29&tree=all

Comment 30

[Intermittent Failures Robot]

18 failures in 764 pushes (0.024 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 7
• pine: 1
• try: 3
• mozilla-central: 4
• mozilla-beta: 3

Platform and build breakdown:

• linux1804-64-qr: 7
  • debug: 6
  • opt: 1
• android-em-7-0-x86_64-qr: 3
  • debug: 3
• linux1804-64-ccov-qr: 2
  • opt: 2
• linux1804-64-asan-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 3
  • opt: 3
• linux1804-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-30&endday=2024-01-30&tree=all

Comment 31

[Intermittent Failures Robot]

15 failures in 700 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 4
• autoland: 7
• mozilla-beta: 4

Platform and build breakdown:

• linux1804-64-asan-qr: 3
  • opt: 3
• linux1804-64-ccov-qr: 1
  • opt: 1
• linux1804-64-qr: 7
  • debug: 5
  • opt: 2
• linux1804-64-shippable-qr: 3
  • opt: 3
• android-em-7-0-x86_64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-31&endday=2024-01-31&tree=all

Comment 32

[Intermittent Failures Robot]

37 failures in 822 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 4
• autoland: 10
• mozilla-beta: 22
• try: 1

Platform and build breakdown:

• linux1804-64-ccov-qr: 1
  • opt: 1
• linux1804-64-qr: 17
  • opt: 5
  • debug: 12
• linux1804-64-asan-qr: 10
  • opt: 10
• linux1804-64-shippable-qr: 4
  • opt: 4
• android-em-7-0-x86_64-qr: 2
  • debug: 2
• linux1804-32-shippable-qr: 1
  • opt: 1
• linux1804-64-devedition-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-01&endday=2024-02-01&tree=all

Comment 33

[Intermittent Failures Robot]

117 failures in 3952 pushes (0.03 failures/push) were associated with this bug in the last 7 days.

This is the #8 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 47
• mozilla-beta: 40
• mozilla-central: 22
• pine: 1
• try: 7

Platform and build breakdown:

• linux1804-64-qr: 53
  • debug: 37
  • opt: 16
• linux1804-64-asan-qr: 22
  • opt: 22
• linux1804-64-shippable-qr: 16
  • opt: 16
• android-em-7-0-x86_64-shippable-qr: 2
  • opt: 2
• linux1804-64-ccov-qr: 8
  • opt: 8
• android-em-7-0-x86_64-qr: 8
  • debug: 8
• linux1804-64-devedition-qr: 5
  • opt: 5
• linux1804-32-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-01-29&endday=2024-02-04&tree=all

Comment 34

[Intermittent Failures Robot]

33 failures in 757 pushes (0.044 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 6
• mozilla-beta: 25
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-qr: 15
  • opt: 1
  • debug: 14
• linux1804-64-asan-qr: 8
  • opt: 8
• linux1804-64-shippable-qr: 4
  • opt: 4
• linux1804-64-ccov-qr: 1
  • opt: 1
• linux1804-32-shippable-qr: 2
  • opt: 2
• linux1804-64-devedition-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-06&endday=2024-02-06&tree=all

Comment 35

[Sunil Mayya]

We should probably disable the keepalive wpts for fetch until we implement it.

Comment 36

[Intermittent Failures Robot]

24 failures in 788 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-beta: 13
• mozilla-central: 3

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1
• linux1804-64-qr: 13
  • debug: 11
  • opt: 2
• linux1804-64-asan-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 4
  • opt: 4
• linux1804-64-devedition-qr: 2
  • opt: 2
• windows11-64-2009-qr: 1
  • debug: 1
• linux1804-32-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-07&endday=2024-02-07&tree=all

Comment 37

[Intermittent Failures Robot]

23 failures in 706 pushes (0.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 3
• mozilla-central: 2
• mozilla-beta: 17
• try: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 3
  • opt: 3
• linux1804-64-qr: 12
  • debug: 12
• linux1804-64-asan-qr: 4
  • opt: 4
• linux1804-32-shippable-qr: 1
  • opt: 1
• linux1804-64-devedition-qr: 2
  • opt: 2
• linux1804-64-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-08&endday=2024-02-08&tree=all

Comment 38

[Sunil Mayya]

These test has been disabled by the patch for Bug 1875702.

Comment 39

[Intermittent Failures Robot]

105 failures in 4009 pushes (0.026 failures/push) were associated with this bug in the last 7 days.

This is the #11 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 27
• mozilla-beta: 69
• mozilla-central: 8
• try: 1

Platform and build breakdown:

• linux1804-64-qr: 49
  • opt: 6
  • debug: 43
• android-em-7-0-x86_64-lite-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 13
  • opt: 13
• linux1804-64-asan-qr: 23
  • opt: 23
• linux1804-64-ccov-qr: 2
  • opt: 2
• linux1804-32-shippable-qr: 6
  • opt: 6
• linux1804-64-devedition-qr: 8
  • opt: 8
• linux1804-64-tsan-qr: 1
  • opt: 1
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-05&endday=2024-02-11&tree=all

Comment 40

[Intermittent Failures Robot]

1 failures in 3125 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-12&endday=2024-02-18&tree=all

Comment 41

[Intermittent Failures Robot]

2 failures in 4052 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1
• linux1804-32-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-02-26&endday=2024-03-03&tree=all

Comment 42

[Intermittent Failures Robot]

1 failures in 4198 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1875674&startday=2024-03-04&endday=2024-03-10&tree=all