1041354 - Some panda devices not in graph server

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P2)

Description

[Nick Thomas [:nthomas] (UTC+12)]

eg: panda-908
https://tbpl.mozilla.org/php/getParsedLog.php?id=44232915&tree=Mozilla-Aurora#error4

15:23:01 CRITICAL -  FAIL: Graph server unreachable (5 attempts)
15:23:01     INFO -  RETURN:send failed, graph server says:
15:23:01 CRITICAL -  RETURN:No machine_name called 'panda-0908' can be found

I suspect panda-0900 through to 0908 were disabled in scl1, but enabled since moving to scl3 in bug 1027839. Certainly they have short job histories in slave health. Now they're failing (ie red on tbpl) on any talos-type job which needs to do a graph server submission, and this in the graph db:

mysql> select max(name) from machines where name like 'panda%';
+------------+
| max(name)  |
+------------+
| panda-0899 |
+------------+
1 row in set (0.00 sec)

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Disabled 0900 to 0909 in slavealloc.

Comment 2

[Chris Cooper [:coop] (he/him)]

INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0900');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0901');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0902');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0903');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0904');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0905');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0906');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0907');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0908');
INSERT INTO machines (os_id, is_throttling, cpu_speed, is_active, date_added, name) VALUES (29,0,1.0,1,unix_timestamp(),'panda-0909');

Comment 3

[Chris Cooper [:coop] (he/him)]

Added to both staging and production databases.