1738369 - No workers available for gecko-t-fxrecorder causing tasks to fail with deadline exceeded

Status: RESOLVED FIXED

(Taskcluster :: Workers, defect)

Description

[Cosmin Sabou [:CosminS]]

https://firefox-ci-tc.services.mozilla.com/provisioners/performance-hardware/worker-types/gecko-t-fxrecorder lists no workers
this is causing Desktop startup recorder jobs like these to fail as an exception with deadline exceeded, for eg: https://firefox-ci-tc.services.mozilla.com/tasks/HNFx69B-QrKDAM0mQm39tQ
last green one was here

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This got added in bug 1715643 a few months ago. Is the machine in the Toronto office unreachable?

Comment 2

[Aki Sasaki (not active)]

Not sure. Barret has a nick of barret | I have 2 broken arms on Matrix, though, so it seems likely that the pool might not be maintained as well as when they were around.

Comment 3

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Dave, do you know who can you help with this unresponsive performance machine in the Toronto office?

Comment 4

[Intermittent Failures Robot]

1 failures in 2845 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows2012-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1738369&startday=2021-10-25&endday=2021-10-31&tree=all

Comment 5

[Greg Mierzwinski [:sparky]]

I don't see us using this task for anything atm (no perfherder data is being submited, and no artifacts are being created). We could probably disable it depending on what :davehunt thinks.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I've opened bug 1739397 for investigating the hardware in the Toronto server room. We can use this bug to temporarily disable the jobs running on mozilla-central. This configuration can be found here: https://searchfox.org/mozilla-central/source/taskcluster/ci/fxrecord/kind.yml

Comment 7

[Intermittent Failures Robot]

1 failures in 3030 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows2012-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1738369&startday=2021-11-01&endday=2021-11-07&tree=all

Comment 8

[:dhouse]

Is this working now? re jlin's turning them on in bug 1739397