1740597 - Intermittent GeckoProfiler.CPUUsage | Expected: (threadCPUDeltaZeroCount) > (threadCPUDeltaNonZeroCount), actual: 7 vs 14

Status: RESOLVED FIXED

(Core :: Gecko Profiler, defect, P2)

Description

[Treeherder Bug Filer]

Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=357672284&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/PFSEp0dBS1WfWLvIR0THrQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-11-10T19:42:42.944Z] 19:42:42     INFO -  TEST-START | GeckoProfiler.CPUUsage
[task 2021-11-10T19:42:43.027Z] 19:42:43  WARNING -  TEST-UNEXPECTED-FAIL | GeckoProfiler.CPUUsage | Expected: (threadCPUDeltaZeroCount) > (threadCPUDeltaNonZeroCount), actual: 7 vs 14
[task 2021-11-10T19:42:43.030Z] 19:42:43     INFO -  There should be more zero-CPUs than non-zero @ /builds/worker/checkouts/gecko/tools/profiler/tests/gtest/GeckoProfiler.cpp:4304
[task 2021-11-10T19:42:43.121Z] 19:42:43  WARNING -  TEST-UNEXPECTED-FAIL | GeckoProfiler.CPUUsage | test completed (time: 178ms)

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

As feared, the test needs tweaking -- But the Firefox code is not at cause here, so no need to backout.

NI:myself, I will wait a few days to collect more test failures, which will give us more information to better adjust the test expectations.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1736616

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1740597 - Tweak GeckoProfiler.CPUUsage gtest expectations of zero/non-zero values - r?canaltinova

The expectation of 50% idle stacks was too optimistic, since we cannot control how the OS interact with idle threads and may still had CPU time to them unexpectedly.
The patch now only expects at least one zero and at least one non-zero.

The non-zero should be guaranteed by the work necessary to go around the loop.
The zero is still hopeful, in practice we've seen a minimum of 7 (out of 20) zeroes. But in the worst case it could still theoretically fail; If that happens we may need to increase the loop count, or try the whole test a few times.

Comment 4

[Intermittent Failures Robot]

7 failures in 2723 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-asan-qr: 1
  • opt: 1
• macosx1015-64-qr: 3
  • debug: 3
• linux1804-32-shippable-qr: 1
  • opt: 1
• linux1804-64-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1740597&startday=2021-11-08&endday=2021-11-14&tree=all

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/73f75f5755d3
Tweak GeckoProfiler.CPUUsage gtest expectations of zero/non-zero values - r=canaltinova

Comment 6

[Cristina Cozmuta (:CrissCozmuta)]

https://hg.mozilla.org/mozilla-central/rev/73f75f5755d3

Comment 7

[Intermittent Failures Robot]

2 failures in 3259 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1
• linux1804-32-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1740597&startday=2021-11-15&endday=2021-11-21&tree=all