1902326 - Perma devtools/client/netmonitor/<test> | Test timed out when Gecko 129 merges to beta on 2024-07-08

Status: VERIFIED FIXED

(DevTools :: Netmonitor, defect, P2)

Description

[Atila Butkovits]

Central-as-beta-simulation

How to run these simulations

Failure log

Comment 1

[Julian Descottes [:jdescottes]]

Thanks for the ping!

Just curious, are we are sure that Bug 1756770 regressed this or is this just a supposition at this point?

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This did not start with bug 1756770, the dt4 task already failed on Tuesday for the central-as-beta simulations.

Comment 3

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Regression range (successful run on Sunday, no run on Monday, failed on Tuesday)

Comment 4

[Intermittent Failures Robot]

10 failures in 4625 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 10

Platform and build breakdown:

• windows11-64-2009-qr: 10
  • debug: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1902326&startday=2024-06-10&endday=2024-06-16&tree=all

Comment 5

[Julian Descottes [:jdescottes]]

Indeed it's already failing on https://treeherder.mozilla.org/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunning%2Cpending%2Crunnable&revision=99ed39877ba3ca5997b4dc0e0da89273e80d91c7&searchStr=dt&selectedTaskRun=cd8kyyT9TC293QCjzawrrQ.0 where my change did not land (It's in the regression range, but the backout as well).

I'm going to remove bug 1756770 as the regressing bug for now. The failures only seem to occur on windows at the moment.

Comment 6

[Julian Descottes [:jdescottes]]

I did an intermediary push, regression range is reduced to https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=22d329ce0ffc65f73e6f54e95ee63b0a99f56b44&tochange=1f6dc95932ad44dfc2ce7ffd613d707ab56f60de

(successful push https://treeherder.mozilla.org/jobs?repo=try&revision=bc58a8efc836a67c4fb52f59aeb96e48413a15fc)

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Another beta sim based on the change which landed in bug 1823907 succeeded. Only bug 1824726 and bug 1882089 landed between that and the first failure.

Alexandre, could investigate if this bug is a regression from bug 1824726?

Comment 8

[Julian Descottes [:jdescottes]]

Nicolas, can you try running some of those tests on windows with a beta simulation?

Comment 10

[Nicolas Chevobbe [:nchevobbe]]

(In reply to Julian Descottes [:jdescottes] from comment #8)

Nicolas, can you try running some of those tests on windows with a beta simulation?

I tried a bit but I don't reproduce :/

Comment 11

[Intermittent Failures Robot]

6 failures in 4577 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 6

Platform and build breakdown:

• windows11-64-2009-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1902326&startday=2024-06-17&endday=2024-06-23&tree=all

Comment 12

[Julian Descottes [:jdescottes]]

Try push with some nightly-only necko prefs to false on Nightly helps to reproduce the issue: https://treeherder.mozilla.org/jobs?repo=try&revision=5cf518f4bfbf9d57b12c5b6f7ab3eed10e665af7

Edit: Does not seem to trigger the issue, unfortunately.

Comment 13

[Julian Descottes [:jdescottes]]

If we don't have time to find the root cause of the issue, we will need to skip the failing tests on windows+beta for now. Let's see if it's only about the 2 which failed so far: https://treeherder.mozilla.org/jobs?repo=try&revision=15289293e28c6e8cfc95f026af44655444aabd46

Comment 14

[Julian Descottes [:jdescottes]]

Interesting thing to note is that the throttling of server side resources normally doesn't impact the network events because they are emitted by the Watcher actor, not the target actor, and the watcher actor is not throttled yet.

So in practice the main difference for netmonitor tests with bug 1824726 is that we lost the client side throttling for resources.
But adding some logs, I can't even see the updates for the missing event timings.

I think for now we should skip the frequently failing tests on beta + win, and investigate in a followup.

Comment 15

[Julian Descottes [:jdescottes]]

Attachment: Bug 1902326 - [devtools] Skip netmonitor tests frequently failing on windows + beta 129

After introducing resource throttling on the server side, we removed client side throttling which seems to frequently fail netmonitor tests on windows + beta

Comment 16

[Serban Stanca [:SerbanS]]

Patch is working in today's central as beta simulation.

Comment 17

[Intermittent Failures Robot]

2 failures in 4116 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 1
  • opt: 1
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1902326&startday=2024-06-24&endday=2024-06-30&tree=all

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/882e75547c1f
[devtools] Skip netmonitor tests frequently failing on windows + beta 129 r=ochameau,devtools-reviewers,nchevobbe

Comment 19

[tszentpeteri]

https://hg.mozilla.org/mozilla-central/rev/882e75547c1f

Comment 20

[Norisz Fay [:noriszfay]]

Verified and resolved fixed as seen in today's central as beta sim