1916221 - Frequent failure in testing/web-platform/tests/webdriver/tests/bidi/network/continue_response/cookies.py

Status: RESOLVED FIXED

(Remote Protocol :: WebDriver BiDi, defect, P2)

Description

[Andrea Marchesini [:baku]]

See https://treeherder.mozilla.org/logviewer?job_id=472569821&repo=try&lineNumber=48409

Comment 1

[Andrea Marchesini [:baku]]

Attachment: Bug 1916221 - Fix Intermittent failure in testing/web-platform/tests/webdriver/tests/bidi/network/continue_response/cookies.py,

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a35755338025
Fix Intermittent failure in testing/web-platform/tests/webdriver/tests/bidi/network/continue_response/cookies.py, r=webdriver-reviewers,Sasha

Comment 3

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Failed to create upstream wpt PR due to merge conflicts. This requires fixup from a wpt sync admin.

Comment 5

[tszentpeteri]

https://hg.mozilla.org/mozilla-central/rev/a35755338025

Comment 6

[Serban Stanca [:SerbanS]]

The problem seems to persist, as it can be seen here. It looks that is quite frequent now.

Hi Andrea! Could you please take another look?

Thank you!

Comment 7

[Intermittent Failures Robot]

33 failures in 493 pushes (0.067 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 32
• mozilla-central: 1

Platform and build breakdown:

• android-em-7-0-x86_64-lite-qr: 9
  • opt: 9
• android-em-7-0-x86_64-qr: 14
  • opt: 6
  • debug-isolated-process: 4
  • debug: 4
• linux1804-64-qr: 5
  • opt: 4
  • debug: 1
• linux1804-64-tsan-qr: 2
  • opt: 2
• macosx1015-64-qr: 1
  • opt: 1
• windows11-64-2009-qr: 1
  • debug: 1
• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1916221&startday=2024-09-02&endday=2024-09-02&tree=all

Comment 8

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/47933 for changes under testing/web-platform/tests

Comment 9

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 10

[Intermittent Failures Robot]

58 failures in 674 pushes (0.086 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 36
• mozilla-central: 18
• try: 4

Platform and build breakdown:

• android-em-7-0-x86_64-lite-qr: 7
  • opt: 7
• linux1804-64-qr: 5
  • debug: 2
  • opt: 3
• macosx1015-64-qr: 8
  • debug: 6
  • opt: 2
• linux1804-64-tsan-qr: 3
  • opt: 3
• android-em-7-0-x86_64-qr: 21
  • debug-isolated-process: 10
  • opt: 4
  • debug: 7
• windows11-32-2009-qr: 3
  • debug: 1
  • opt: 2
• windows11-32-2009-shippable-qr: 2
  • opt: 2
• windows11-64-2009-shippable-qr: 1
  • opt: 1
• windows11-64-2009-qr: 4
  • opt: 3
  • debug: 1
• linux1804-64-asan-qr: 2
  • opt: 2
• android-em-7-0-x86_64-shippable-lite-qr: 1
  • opt: 1
• android-em-7-0-x86_64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1916221&startday=2024-09-03&endday=2024-09-03&tree=all

Comment 11

[Intermittent Failures Robot]

61 failures in 818 pushes (0.075 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-central: 16
• autoland: 41
• try: 4

Platform and build breakdown:

• android-em-7-0-x86_64-qr: 21
  • debug: 12
  • debug-isolated-process: 5
  • opt: 4
• macosx1015-64-qr: 7
  • debug: 6
  • opt: 1
• linux1804-64-ccov-qr: 1
  • opt: 1
• android-em-7-0-x86_64-lite-qr: 7
  • opt: 7
• linux1804-64-shippable-qr: 2
  • opt: 2
• windows11-64-2009-shippable-qr: 1
  • opt: 1
• windows11-32-2009-qr: 4
  • debug: 3
  • opt: 1
• android-em-7-0-x86_64-shippable-lite-qr: 2
  • opt: 2
• android-em-7-0-x86_64-shippable-qr: 2
  • opt: 2
• macosx1015-64-shippable-qr: 1
  • opt: 1
• windows11-64-2009-qr: 6
  • opt: 3
  • debug: 3
• linux1804-64-tsan-qr: 3
  • opt: 3
• linux1804-64-qr: 3
  • opt: 2
  • debug: 1
• linux1804-64-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1916221&startday=2024-09-04&endday=2024-09-04&tree=all

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Do we actually know what regressed this test? It's actually a new test that Julian added recently on bug 1853887.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It's related to the expire date: expected = 1725633380, actual = 1725633379 which corresponds to Friday, 6. September 2024 14:36:20 and Friday, 6. September 2024 14:36:19.

Might there be a rounding error somewhere given that the actual timestamp differs by just 1 from the expected one.

Comment 14

[Julian Descottes [:jdescottes]]

Right, I added this test recently, but the failures started after Bug 1913161 landed 3 days ago I believe.

Comment 15

[Julian Descottes [:jdescottes]]

The regressed-by info was correctly set on the duplicate bug 1916238, but not here.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1913161

Comment 17

[Julian Descottes [:jdescottes]]

Oh I think the regression info is wrong actually. This is most likely regressed by Bug 1909980.
So I initially suspected a rounding issue with datetime apis, since timestamps are in microseconds, int() would round the value while strftime would truncate the extra data.
But even when hardcoding the values (timestamp and formatted string), the intermittent would persist. Sometimes the computed expiry timestamp is still off by 1.

Looking at the patch from Bug 1909980, it makes more sense. We add a delta to the provided expiry at https://searchfox.org/mozilla-central/rev/f09e3f9603a08b5b51bf504846091579bc2ff531/netwerk/cookie/CookieParser.cpp#753-755

if (StaticPrefs::network_cookie_useServerTime()) {
  expires += delta;
} else if (expires <= aCurrentTime &&

And turning off the preference network.cookie.useServerTime "fixes" the intermittent.
So either we do that, or we only assert that actual and expected expiry are close enough. But I imagine there's no guarantee that the gap will always be at most 1ms?

Comment 18

[Julian Descottes [:jdescottes]]

Attachment: Bug 1916221 - [wdspec] Fix continueResponse cookie test intermittent with flaky expiry date parsing

The parsed expiry for cookies might be slightly different from the provided value, because
Firefox will attempt to compensate the delta between the server time and the current time.
Instead of expecting an exact match, assert that the actual expiry is within 1ms of the expected
value.

Comment 19

[Intermittent Failures Robot]

45 failures in 781 pushes (0.058 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 24
• try: 9
• autoland: 12

Platform and build breakdown:

• macosx1015-64-qr: 5
  • debug: 4
  • opt: 1
• linux1804-64-asan-qr: 1
  • opt: 1
• android-em-7-0-x86_64-qr: 19
  • debug-isolated-process: 7
  • debug: 9
  • opt: 3
• linux1804-64-qr: 3
  • debug: 2
  • opt: 1
• linux1804-64-tsan-qr: 1
  • opt: 1
• android-em-7-0-x86_64-shippable-lite-qr: 1
  • opt: 1
• windows11-64-2009-shippable-qr: 1
  • opt: 1
• android-em-7-0-x86_64-shippable-qr: 4
  • opt: 4
• android-em-7-0-x86_64-lite-qr: 4
  • opt: 4
• windows11-64-2009-qr: 2
  • opt: 1
  • debug: 1
• linux1804-64-ccov-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 1
  • opt: 1
• windows11-32-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1916221&startday=2024-09-05&endday=2024-09-05&tree=all

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b04eab8729b7
[wdspec] Fix continueResponse cookie test intermittent with flaky expiry date parsing r=baku,webdriver-reviewers,Sasha

Comment 21

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/48019 for changes under testing/web-platform/tests

Comment 22

[Intermittent Failures Robot]

15 failures in 673 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 6
• autoland: 2
• try: 7

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 1
  • opt: 1
• linux1804-64-qr: 2
  • debug: 2
• windows11-32-2009-qr: 2
  • debug: 2
• android-em-7-0-x86_64-lite-qr: 3
  • opt: 3
• android-em-7-0-x86_64-qr: 4
  • opt: 3
  • debug: 1
• linux1804-64-asan-qr: 1
  • opt: 1
• windows11-64-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1916221&startday=2024-09-06&endday=2024-09-06&tree=all

Comment 23

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/b04eab8729b7

Comment 24

[Intermittent Failures Robot]

213 failures in 3668 pushes (0.058 failures/push) were associated with this bug in the last 7 days.

This is the #1 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 65
• autoland: 123
• try: 25

Platform and build breakdown:

• macosx1015-64-qr: 21
  • debug: 16
  • opt: 5
• linux1804-64-tsan-qr: 10
  • opt: 10
• android-em-7-0-x86_64-qr: 79
  • debug: 33
  • debug-isolated-process: 26
  • opt: 20
• android-em-7-0-x86_64-lite-qr: 30
  • opt: 30
• linux1804-64-qr: 18
  • debug: 8
  • opt: 10
• linux1804-64-asan-qr: 6
  • opt: 6
• windows11-32-2009-qr: 10
  • debug: 7
  • opt: 3
• linux1804-64-ccov-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 4
  • opt: 4
• android-em-7-0-x86_64-shippable-lite-qr: 4
  • opt: 4
• windows11-64-2009-shippable-qr: 5
  • opt: 5
• android-em-7-0-x86_64-shippable-qr: 7
  • opt: 7
• macosx1015-64-shippable-qr: 1
  • opt: 1
• windows11-64-2009-qr: 13
  • opt: 7
  • debug: 6
• windows11-32-2009-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1916221&startday=2024-09-02&endday=2024-09-08&tree=all

Comment 25

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot