1754563 - Perma [tier2] fxrecord [taskcluster:error] exit status 1

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ctuns [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=367274491&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/RtewiRSxTIq-ChgtvV39wg/runs/3/artifacts/public/logs/live_backing.log

---

[taskcluster 2022-02-09T18:10:12.952Z] Worker Type (performance-hardware/gecko-t-fxrecorder) settings:
[taskcluster 2022-02-09T18:10:12.952Z]   {
[taskcluster 2022-02-09T18:10:12.952Z]     "config": {
[taskcluster 2022-02-09T18:10:12.952Z]       "deploymentId": ""
[taskcluster 2022-02-09T18:10:12.952Z]     },
[taskcluster 2022-02-09T18:10:12.952Z]     "generic-worker": {
[taskcluster 2022-02-09T18:10:12.952Z]       "engine": "multiuser",
[taskcluster 2022-02-09T18:10:12.952Z]       "go-arch": "amd64",
[taskcluster 2022-02-09T18:10:12.952Z]       "go-os": "windows",
[taskcluster 2022-02-09T18:10:12.952Z]       "go-version": "go1.16.3",
[taskcluster 2022-02-09T18:10:12.952Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v44.0.0",
[taskcluster 2022-02-09T18:10:12.952Z]       "revision": "faf6f319c05a1fa96ee1bc38bab2dd31d75e8a73",
[taskcluster 2022-02-09T18:10:12.952Z]       "source": "https://github.com/taskcluster/taskcluster/commits/faf6f319c05a1fa96ee1bc38bab2dd31d75e8a73",
[taskcluster 2022-02-09T18:10:12.952Z]       "version": "44.0.0"
[taskcluster 2022-02-09T18:10:12.952Z]     }
[taskcluster 2022-02-09T18:10:12.952Z]   }
[taskcluster 2022-02-09T18:10:12.953Z] Task ID: RtewiRSxTIq-ChgtvV39wg
[taskcluster 2022-02-09T18:10:12.953Z] === Task Starting ===
[taskcluster 2022-02-09T18:10:15.089Z] Uploading redirect artifact public/logs/live.log to URL https://firefoxci-websocktunnel.services.mozilla.com/test-worker-group.fxrecorder01.60099/log/GkCBAqLiT3qo4cAH62JggQ with mime type "text/plain; charset=utf-8" and expiry 2022-02-09T18:55:14.997Z
[taskcluster 2022-02-09T18:10:15.158Z] Executing command 0: C:\fxrecorder\fxrecorder.exe --config C:\fxrecorder\fxrecord.toml record
C:\Users\task_164443003153966>C:\fxrecorder\fxrecorder.exe --config C:\fxrecorder\fxrecord.toml record 
2022-02-09 18:10:15.339Z INFO read command-line options
  options: Options { config_path: "C:\\fxrecorder\\fxrecord.toml", command: Record(RecordOptions { task_id: "P5WTaJQcQo658XeEuIZEIA", profile_path: None, prefs: [], skip_idle: false, keep_video: false }), output_path: None }
2022-02-09 18:10:15.351Z INFO Connected
  peer: fxrunner01.corp.tor1.mozilla.com:8888
2022-02-09 18:10:15.351Z INFO Requesting new session
2022-02-09 18:10:15.872Z INFO Downloading build ...
2022-02-09 18:10:26.511Z INFO Build download complete; extracting build ...
2022-02-09 18:11:08.871Z INFO Build extracted
2022-02-09 18:11:12.387Z INFO No profile to send
2022-02-09 18:11:12.468Z INFO Runner is restarting...
2022-02-09 18:11:12.469Z INFO Disconnected from runner. Waiting to reconnect...
2022-02-09 18:11:42.470Z INFO Attempting re-connection to runner...
2022-02-09 18:13:03.475Z INFO Attempting re-connection to runner...
2022-02-09 18:15:05.485Z INFO Attempting re-connection to runner...
2022-02-09 18:15:05.489Z INFO Re-connected
  peer: fxrunner01.corp.tor1.mozilla.com:8888
2022-02-09 18:15:05.489Z INFO Resuming session
2022-02-09 18:15:17.671Z INFO Waiting for runner to become idle...
2022-02-09 18:15:35.249Z ERROR Runner could not become idle
  error: timed out waiting for CPU and disk to become idle
2022-02-09 18:15:35.253Z ERROR unexpected error
  error: a remote error occurred: timed out waiting for CPU and disk to become idle
[taskcluster 2022-02-09T18:15:35.276Z]    Exit Code: 1
[taskcluster 2022-02-09T18:15:35.276Z]    User Time: 15.625ms
[taskcluster 2022-02-09T18:15:35.276Z]  Kernel Time: 0s
[taskcluster 2022-02-09T18:15:35.276Z]    Wall Time: 5m20.1168553s
[taskcluster 2022-02-09T18:15:35.276Z]       Result: FAILED
[taskcluster 2022-02-09T18:15:35.277Z] === Task Finished ===
[taskcluster 2022-02-09T18:15:35.277Z] Task Duration: 5m20.1187056s
[taskcluster 2022-02-09T18:15:35.498Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2023-02-09T16:14:22.749Z
[taskcluster:error] exit status 1

Comment 1

[Cristian Tuns]

Hello Barret, can you please take a look at this?

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This turned green on the next merge to mozilla-central.

Comment 3

[Barret Rennie [:barret] (they/them)]

Lookinag at the logs, the first job restarted the fxrunner laptop and was not able to reconnect. The next to retries were also not able to reconnect. The last retry was able to reconnect but the fxrunner machine did not become idle.

If I had to guess, I'd say that this was maybe a Windows Update running and causing the reboot to take a long time (which explains why it couldn't reconnect in time) and then some of the update stuff running post reboot (which explains why it could not go idle in the 3rd retry).

Since the job has gone green on the latest m-c, I've retriggered the failed job and expect it to succeed.

Comment 4

[Cosmin Sabou [:CosminS]]

Green also on the next merge. Let's consider this fixed.

Comment 5

[Intermittent Failures Robot]

1 failures in 3711 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows2012-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1754563&startday=2022-02-07&endday=2022-02-13&tree=all