1917426 - [win perma] toolkit/components/processtools/tests/xpcshell/test_total_cpu_time.js | test_totalCpuTime_in_parent - [test_totalCpuTime_in_parent : 38] The CPU time reported through Glean should be at least as much as the CPU time reported by ProcInfo ...

Status: RESOLVED FIXED

(Thunderbird :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mkmelin [at] iki.fi
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=473333199&repo=comm-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/ZhvmQfrISHCAeF6_IIY7CA/runs/0/artifacts/public/logs/live_backing.log

---

Comment 1

[Intermittent Failures Robot]

8 failures in 3668 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 8

Platform and build breakdown:

• windows11-64-2009-shippable-qr: 1
  • opt: 1
• windows11-64-2009-qr: 3
  • debug: 3
• windows11-32-2009-qr: 2
  • opt: 1
  • debug: 1
• windows11-32-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-09-02&endday=2024-09-08&tree=all

Comment 2

[Magnus Melin [:mkmelin]]

Seems first pushes with this is https://treeherder.mozilla.org/jobs?repo=comm-central&tochange=34c1074463a8383fb7f1ee5dadad4a5d953ccf3d&fromchange=0bd88e159c2d293749b8b1e3ca4c20f32c95e923&selectedTaskRun=YnlywbetQI-20DS8xttBZw.0 corresponding to https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=cbfd8f13ada7739ef07dd4873cfd87e69e5c91d2

Comment 3

[Intermittent Failures Robot]

21 failures in 4448 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 21

Platform and build breakdown:

• linux1804-64-qr: 1
  • opt: 1
• windows11-64-2009-qr: 7
  • debug: 6
  • opt: 1
• windows11-32-2009-qr: 9
  • opt: 4
  • debug: 5
• windows11-64-2009-shippable-qr: 3
  • opt: 3
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-09-09&endday=2024-09-15&tree=all

Comment 4

[Magnus Melin [:mkmelin]]

I don't see what would have caused it.
The windows failure is here: https://searchfox.org/comm-central/rev/94a4fb022ccd5586762df7f422be2e01243b6173/mozilla/toolkit/components/processtools/tests/xpcshell/test_total_cpu_time.js#41
"The CPU time reported through Glean should be at least as much as the CPU time reported by ProcInfo before asking Glean for the data - 346 >= 349"
I see a note "50 is an arbitrary value, but the resolution is 16ms on Windows" a few lines below. Could that have some impact here?? Florian, as original creator of this patch, any ideas?

Comment 5

[Intermittent Failures Robot]

46 failures in 4860 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 46

Platform and build breakdown:

• windows11-32-2009-qr: 18
  • opt: 6
  • debug: 12
• windows11-64-2009-qr: 22
  • debug: 18
  • opt: 4
• windows11-32-2009-shippable-qr: 1
  • opt: 1
• windows11-64-2009-shippable-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-09-16&endday=2024-09-22&tree=all

Comment 6

[Intermittent Failures Robot]

40 failures in 4889 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #39 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 40

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 2
  • opt: 2
• windows11-64-2009-qr: 16
  • debug: 15
  • opt: 1
• windows11-64-2009-shippable-qr: 4
  • opt: 4
• windows11-32-2009-qr: 12
  • debug: 11
  • opt: 1
• windows11-64-2009-ccov-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-09-23&endday=2024-09-29&tree=all

Comment 7

[Intermittent Failures Robot]

35 failures in 3574 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #45 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 35

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 1
  • opt: 1
• windows11-32-2009-qr: 10
  • debug: 8
  • opt: 2
• windows11-64-2009-qr: 16
  • debug: 14
  • opt: 2
• windows11-64-2009-shippable-qr: 3
  • opt: 3
• windows11-64-2009-ccov-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-09-30&endday=2024-10-06&tree=all

Comment 8

[Florian Quèze [:florian]]

(In reply to Magnus Melin [:mkmelin] from comment #4)

I don't see what would have caused it.
The windows failure is here: https://searchfox.org/comm-central/rev/94a4fb022ccd5586762df7f422be2e01243b6173/mozilla/toolkit/components/processtools/tests/xpcshell/test_total_cpu_time.js#41
"The CPU time reported through Glean should be at least as much as the CPU time reported by ProcInfo before asking Glean for the data - 346 >= 349"
I see a note "50 is an arbitrary value, but the resolution is 16ms on Windows" a few lines below. Could that have some impact here?? Florian, as original creator of this patch, any ideas?

This bug feels very similar to bug 1880373, but I don't see why the fixes we landed there wouldn't work on comm-central.

Are you running these tests on real hardware or on virtual machines?
Virtual machines usually don't report having a constant_tsc, which means we can't get a resolution higher than 1/64 s for CPU times on them. There's a hack at https://searchfox.org/mozilla-central/rev/906cf04e249dd79ee496f01f9ed79cc16b1ff78c/toolkit/components/processtools/ProcInfo_win.cpp#61-67 to force using a high resolution in automation. I don't see a reason for that hack to not work on comm-central.

Is there any chance xpc::IsInAutomation could be broken on comm-central? (I assume not, as that would likely break many other things)

Comment 9

[Magnus Melin [:mkmelin]]

Are you running these tests on real hardware or on virtual machines?

Rob, do you know?

The test does not fail locally (at least for me).

Comment 10

[Geoff Lankow (:darktrojan)]

(In reply to Florian Quèze [:florian] from comment #8)

Is there any chance xpc::IsInAutomation could be broken on comm-central? (I assume not, as that would likely break many other things)

I'm pretty sure it's not broken. Using it for something at the moment and it works as expected.

Comment 12

[Rob Lemley [:rjl]]

(In reply to Magnus Melin [:mkmelin] from comment #9)

Are you running these tests on real hardware or on virtual machines?

Rob, do you know?

The test does not fail locally (at least for me).

Windows tests run on virtual. It's the same basic "win11-64-2009" image that Firefox tests run on. (No gpu or anything fancy)

Comment 13

[Intermittent Failures Robot]

55 failures in 4679 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #34 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 52
• try-comm-central: 3

Platform and build breakdown:

• windows11-32-2009-qr: 16
  • debug: 14
  • opt: 2
• windows11-64-2009-qr: 25
  • opt: 7
  • debug: 18
• windows11-64-2009-shippable-qr: 6
  • opt: 6
• windows11-64-2009-ccov-qr: 5
  • opt: 5
• linux1804-64-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-10-07&endday=2024-10-13&tree=all

Comment 14

[Intermittent Failures Robot]

16 failures in 955 pushes (0.017 failures/push) were associated with this bug yesterday.

Repository breakdown:

• comm-central: 13
• try-comm-central: 3

Platform and build breakdown:

• windows11-64-2009-qr: 9
  • opt: 3
  • debug: 6
• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-32-2009-qr: 3
  • debug: 3
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-10-15&endday=2024-10-15&tree=all

Comment 15

[Magnus Melin [:mkmelin]]

Attachment: Bug 1917426 - Disable test_totalCpuTime_in_parent for Thunderbird. r=florian

Comment 16

[Intermittent Failures Robot]

18 failures in 897 pushes (0.02 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• comm-central: 18

Platform and build breakdown:

• windows11-32-2009-qr: 4
  • debug: 4
• windows11-32-2009-shippable-qr: 1
  • opt: 1
• windows11-64-2009-qr: 8
  • debug: 7
  • opt: 1
• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-64-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-10-17&endday=2024-10-17&tree=all

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/52b544b6424e
Disable test_totalCpuTime_in_parent for Thunderbird. r=florian

Comment 18

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/52b544b6424e

Comment 19

[Intermittent Failures Robot]

48 failures in 4579 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #50 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 45
• try-comm-central: 3

Platform and build breakdown:

• windows11-32-2009-qr: 11
  • debug: 11
• windows11-32-2009-shippable-qr: 3
  • opt: 3
• windows11-64-2009-qr: 22
  • debug: 18
  • opt: 4
• windows11-64-2009-ccov-qr: 10
  • opt: 10
• windows11-64-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1917426&startday=2024-10-14&endday=2024-10-20&tree=all