1991910 - toolkit/components/extensions/test/xpcshell/test_MatchPattern.js fails frequently in parallel

Status: RESOLVED FIXED

(WebExtensions :: General, defect, P3)

Description

[Florian Quèze [:florian]]

This happens about 20 times a day, and isn't visible on treeherder because the xcpshell tests failing in parallel then re-run a second time sequentially.

Here's a profile of a failure: https://share.firefox.dev/46x0A0P
The failure message was: [test_MatchGlob_redundant_wildcards_backtracking : 527] First matching duration: 402ms (limit: 20ms) - 402 < 20

This happened on a Mac machine that was running slowly (I suspect it of over heating).

https://searchfox.org/firefox-main/rev/21d3e8ab8b61715ddd39ac04c62a846fa79deddd/toolkit/components/extensions/test/xpcshell/test_MatchPattern.js#490-540 seems non-deterministic. Especially, it attempts to measure performance by measuring wall clock time. I think it could be more reliable if it measured CPU time instead (especially for tests that are running in parallel with 30 other tests running at the same time).

Here's some example code of how to measure CPU time from an xpcshell test:

// Get total CPU time for the current process and all its children
async function getTotalCpuTime() {
  let procInfo = await ChromeUtils.requestProcInfo();
  let totalCpuTime = procInfo.cpuTime;

  // Add CPU time from all child processes
  for (let child of procInfo.children) {
    totalCpuTime += child.cpuTime;
  }

  return totalCpuTime;
}

Calling it instead of Date.now() should provide something more deterministic.

Comment 1

[Tomislav Jovanovic :zombie]

Attachment: Bug 1991910 - Use CPU time instead of wall clock in test_MatchPattern.js r?robwu

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/15e98967a272
https://hg.mozilla.org/integration/autoland/rev/e257ac5742d3
Use CPU time instead of wall clock in test_MatchPattern.js r=robwu

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/3d3016fb3463
https://hg.mozilla.org/integration/autoland/rev/585b48d93122
Revert "Bug 1991910 - Use CPU time instead of wall clock in test_MatchPattern.js r=robwu" for causing xpcshell failures on test_MatchPattern.js

Comment 4

[Cristina Horotan [:chorotan]]

Backed out for causing xpcshell failures on test_MatchPattern.js

Backout link

Push with failures

Failure log

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/8fa15c02db08
https://hg.mozilla.org/integration/autoland/rev/426ed2e56aa8
Use CPU time instead of wall clock in test_MatchPattern.js r=robwu

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/426ed2e56aa8