1723430 - Perma xpcshell [tier 2] windows10-64-2004-ccov-qr [taskcluster:error] Task aborted - max run time exceeded

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Alexandru Michis [:malexandru]]

Failure log

[task 2021-08-02T01:07:58.274Z] 01:07:58     INFO -  testKnownFail (__main__.XPCShellTests.runSelfTest.<locals>.XPCShellTestsTests)
[task 2021-08-02T01:08:10.235Z] 01:08:10     INFO -  Check that a simple failing test marked as known-fail in the manifest ... ok
[task 2021-08-02T01:08:10.237Z] 01:08:10     INFO -  testLogCorrectFileName (__main__.XPCShellTests.runSelfTest.<locals>.XPCShellTestsTests)
[task 2021-08-02T01:08:24.266Z] 01:08:24     INFO -  Make sure a meaningful filename and line number is logged ... ok
[task 2021-08-02T01:08:24.268Z] 01:08:24     INFO -  testMissingHeadFile (__main__.XPCShellTests.runSelfTest.<locals>.XPCShellTestsTests)
[task 2021-08-02T01:08:24.453Z] 01:08:24     INFO -  Ensure that missing head file results in fatal error. ... ok
[task 2021-08-02T01:08:24.456Z] 01:08:24     INFO -  testMozinfo (__main__.XPCShellTests.runSelfTest.<locals>.XPCShellTestsTests)
[task 2021-08-02T01:08:35.788Z] 01:08:35     INFO -  Check that mozinfo.json is loaded ... ok
[task 2021-08-02T01:08:35.790Z] 01:08:35     INFO -  testNoRunTestAddTask (__main__.XPCShellTests.runSelfTest.<locals>.XPCShellTestsTests)
<...>
[task 2021-08-02T01:42:52.913Z] 01:42:52     INFO -  TEST-START | services/common/tests/unit/test_utils_encodeBase32.js
[taskcluster:error] Aborting task...
[taskcluster 2021-08-02T01:43:02.706Z] ERROR: The process with PID 3952 (child process of PID 10624) could not be terminated.
[taskcluster 2021-08-02T01:43:02.706Z] Reason: There is no running instance of the task.
<...>
[taskcluster 2021-08-02T01:43:02.706Z] ERROR: The process with PID 9624 (child process of PID 3896) could not be terminated.
[taskcluster 2021-08-02T01:43:02.706Z] Reason: There is no running instance of the task.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 3896 (child process of PID 8628) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 4200 (child process of PID 7424) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 8628 (child process of PID 7424) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 7424 (child process of PID 9044) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 8772 (child process of PID 5896) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 9044 (child process of PID 5896) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] SUCCESS: The process with PID 5896 (child process of PID 8928) has been terminated.
[taskcluster 2021-08-02T01:43:02.706Z] 
[taskcluster:warn 2021-08-02T01:43:02.708Z] exit status 128
[taskcluster 2021-08-02T01:43:02.708Z] === Task Finished ===
[taskcluster 2021-08-02T01:43:02.708Z] Task Duration: 45m0.3573561s
[taskcluster 2021-08-02T01:43:03.079Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2022-08-02T00:57:54.868Z
[taskcluster 2021-08-02T01:43:03.562Z] Uploading artifact public/test_info/2a630cdc-a6e2-412d-823a-3a41b92b1ce1.dmp from file build\blobber_upload_dir\2a630cdc-a6e2-412d-823a-3a41b92b1ce1.dmp with content encoding "gzip", mime type "application/octet-stream" and expiry 2022-08-02T00:57:54.868Z
[taskcluster 2021-08-02T01:43:04.048Z] Uploading artifact public/test_info/2a630cdc-a6e2-412d-823a-3a41b92b1ce1.extra from file build\blobber_upload_dir\2a630cdc-a6e2-412d-823a-3a41b92b1ce1.extra with content encoding "gzip", mime type "application/octet-stream" and expiry 2022-08-02T00:57:54.868Z
[taskcluster 2021-08-02T01:43:04.241Z] Uploading artifact public/test_info/system-info.log from file build\blobber_upload_dir\system-info.log with content encoding "gzip", mime type "text/plain" and expiry 2022-08-02T00:57:54.868Z
[taskcluster 2021-08-02T01:43:04.428Z] Uploading artifact public/test_info/xpcshell_errorsummary.log from file build\blobber_upload_dir\xpcshell_errorsummary.log with content encoding "gzip", mime type "text/plain" and expiry 2022-08-02T00:57:54.868Z
[taskcluster 2021-08-02T01:43:04.660Z] Uploading artifact public/test_info/xpcshell_raw.log from file build\blobber_upload_dir\xpcshell_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2022-08-02T00:57:54.868Z
[taskcluster 2021-08-02T01:43:04.878Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2022-08-02T00:57:54.868Z
[taskcluster:error] Task aborted - max run time exceeded

masterwayz, could you please take a look?

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1718292

Comment 2

[Michelle Goossens [:masterwayz]]

Investigating this with the team that works on the migration along with me. I have a patch prepared already to move xpcshell Windows ccov back to the old workers should this be needed. Waiting to hear from them and then will update the bug.

Comment 3

[Intermittent Failures Robot]

17 failures in 561 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 17

Platform and build breakdown:

• windows10-64-2004-ccov-qr: 17
  • opt: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1723430&startday=2021-08-03&endday=2021-08-03&tree=all

Comment 4

[Intermittent Failures Robot]

25 failures in 580 pushes (0.043 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 25

Platform and build breakdown:

• windows10-64-2004-ccov-qr: 25
  • opt: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1723430&startday=2021-08-04&endday=2021-08-04&tree=all

Comment 5

[Intermittent Failures Robot]

17 failures in 625 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 17

Platform and build breakdown:

• windows10-64-2004-ccov-qr: 17
  • opt: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1723430&startday=2021-08-05&endday=2021-08-05&tree=all

Comment 6

[Michelle Goossens [:masterwayz]]

This should be fixed with the landing of Bug 1723531. Can you keep an eye on it and see if it has indeed solved the problem?

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Thank you, Windows XPCshell tasks are back to successful runs.

Comment 8

[Intermittent Failures Robot]

65 failures in 2829 pushes (0.023 failures/push) were associated with this bug in the last 7 days.

This is the #16 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 65

Platform and build breakdown:

• windows10-64-2004-ccov-qr: 65
  • opt: 65

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1723430&startday=2021-08-02&endday=2021-08-08&tree=all