1441820 - Quarantine on GECKO-T-W10-64-HW instances

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Zsolt Fay [:zfay]]

I have quarantined a number of instances from gecko-t-win10-64-hw. These instances had the last 10+ tasks resolved as exception (orange). Can relops take a look at them?

These instances are:
T-W1064-MS-117
T-W1064-MS-119
T-W1064-MS-121
T-W1064-MS-127
T-W1064-MS-152
T-W1064-MS-163
T-W1064-MS-167
T-W1064-MS-197
T-W1064-MS-207
T-W1064-MS-208
T-W1064-MS-218
T-W1064-MS-258
T-W1064-MS-269

Thanks in advance!

Comment 1

[Pete Moore [:pmoore][:pete]]

I also quarantined some today. The full list of releng-hardware/gecko-t-win10-64-hw quarantined workers is currently:

T-W1064-MS-034
T-W1064-MS-037
T-W1064-MS-075
T-W1064-MS-112
T-W1064-MS-117
T-W1064-MS-119
T-W1064-MS-121
T-W1064-MS-127
T-W1064-MS-129
T-W1064-MS-152
T-W1064-MS-167
T-W1064-MS-197
T-W1064-MS-200
T-W1064-MS-207
T-W1064-MS-208
T-W1064-MS-218
T-W1064-MS-258
T-W1064-MS-269

This list can be refreshed by visiting https://tools.taskcluster.net/provisioners/releng-hardware/worker-types/gecko-t-win10-64-hw and choosing the filter "quarantined".

Comment 2

[Bogdan Crisan [:bcrisan] (EEST - GMT + 3)]

I also quarantined some of them (add them to the list from comment 1):
T-W1064-MS-078
T-W1064-MS-198
T-W1064-MS-221

Comment 3

[Danut Labici [:dlabici]]

First day of spring, so lets add the following quarantined instances to the list:
T-W1064-MS-113
T-W1064-MS-123
T-W1064-MS-153
T-W1064-MS-158
T-W1064-MS-164
T-W1064-MS-084
T-W1064-MS-169
T-W1064-MS-215

Comment 4

[Bogdan Crisan [:bcrisan] (EEST - GMT + 3)]

The last 10+ resolved tasks by these machines was marked as exceptions so I quarantined them.

T-W1064-MS-065
T-W1064-MS-072
T-W1064-MS-082
T-W1064-MS-086
T-W1064-MS-088
T-W1064-MS-111
T-W1064-MS-156
T-W1064-MS-174
T-W1064-MS-214
T-W1064-MS-248
T-W1064-MS-256
T-W1064-MS-257

Comment 5

[Zsolt Fay [:zfay]]

Added these machines to the quarantine list since they have the entire first page of tasks solved as exception.

5th March 2018:

T-W1064-MS-207
T-W1064-MS-221
T-W1064-MS-258

Comment 6

[Roland Mutter Michael (:rmutter)]

Adding the following:

T-W1064-MS-078
T-W1064-MS-129
T-W1064-MS-153
T-W1064-MS-219

Comment 7

[Kendall Libby [:fubar] (he/him)]

Mark's been working on the moonshots to fix a bunch of the issues that came up last week (when this was originally filed), and has been re-installing them. Once that's finished, these will need to be unquarantined to pick up jobs; but we should also make sure that there aren't any other lingering issues.

Comment 8

[Mark Cornmesser [:markco] OOO 2024/04/15]

Is there documentation on how to quarantine and unquarantine workers? I would like to quarantine several more while I am testing, and then after I am done testing I can go through an unquarantine all.

Comment 9

[Zsolt Fay [:zfay]]

(In reply to Mark Cornmesser [:markco] from comment #8)
> Is there documentation on how to quarantine and unquarantine workers? I
> would like to quarantine several more while I am testing, and then after I
> am done testing I can go through an unquarantine all.

Hey Mark, we recently created a page about this. Hope it helps. Feel free to contact me as well if you stumble across anything else build duty related.
https://wiki.mozilla.org/BuildDuty:QuarantineInstances

Comment 10

[Mark Cornmesser [:markco] OOO 2024/04/15]

I believe all these are now pickup tasks.