If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

[tracker] Panda Move Train 1

RESOLVED FIXED

Status

Infrastructure & Operations
RelOps
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: arr, Assigned: arr)

Tracking

({meta})

Details

(Whiteboard: [kanban:https://kanbanize.com/ctrl_board/4/451] , URL)

Attachments

(1 attachment)

(Assignee)

Description

3 years ago
For all work related to Panda Train 1. This includes panda racks p1, p2, and p4.

Updated

3 years ago
Whiteboard: [kanban]

Updated

3 years ago
Whiteboard: [kanban] → [kanban:https://kanbanize.com/ctrl_board/4/451]

Updated

3 years ago
Depends on: 1031847
Depends on: 1031936
Created attachment 8448013 [details]
train_1-down.sh

script to downtime hosts under old host names (in scl1)
Unable to find some of those hosts:
Downtime for foopy40.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy41.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy42.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy43.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy44.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy45.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy46.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy47.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy48.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy49.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy50.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy51.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy52.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy60.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy61.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy62.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy63.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy64.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy65.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for foopy66.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for mobile-imaging-001.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for mobile-imaging-002.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for mobile-imaging-004.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0033.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0082.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0083.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0084.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0085.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0086.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0087.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0088.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0089.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0090.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0091.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0092.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0093.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0094.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0095.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0096.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0097.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0098.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0099.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0100.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0101.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0102.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0103.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0104.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0105.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0106.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0107.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0108.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0109.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0110.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0111.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0112.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0113.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0114.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0115.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0116.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0117.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0118.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0119.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0120.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0121.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0122.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0123.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0124.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0125.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0126.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0127.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0128.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0129.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0130.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0131.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0132.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0133.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0134.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0135.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0136.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0137.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0138.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0139.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0140.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0141.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0142.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0143.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0144.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0145.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0146.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0147.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0148.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0149.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0150.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0151.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0152.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0153.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0154.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0155.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0156.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0157.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0158.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0159.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0160.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0161.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0162.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0163.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0164.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0165.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0166.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0167.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0168.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0169.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0170.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0171.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0173.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0174.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0175.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0176.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0177.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0178.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0179.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0180.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0181.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0182.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0183.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0184.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0185.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0186.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0187.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0188.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0189.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0190.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0191.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0192.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0193.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0194.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0195.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0196.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0197.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0198.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0199.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0200.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0201.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0202.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0203.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0204.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0205.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0206.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0207.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0208.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0209.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0210.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0211.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0212.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0213.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0214.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0215.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0216.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0217.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0218.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0219.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0220.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0221.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0222.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0223.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0224.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0225.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0226.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0227.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0228.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0229.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0230.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0231.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0232.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0233.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0234.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0235.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0236.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0237.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0238.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0239.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0240.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0241.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0242.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0243.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0244.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0245.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0246.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0247.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0248.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0249.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Can't find a host by that name.
Downtime for panda-0251.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0252.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0253.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0254.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0255.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0256.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0257.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0346.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0347.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0348.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0349.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0350.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0351.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0352.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0353.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0354.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0355.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0356.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0357.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0358.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0359.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0360.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0361.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0362.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0363.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0364.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0365.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0366.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0367.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0368.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0369.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0370.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0372.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0373.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0374.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0375.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0376.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0377.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0378.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0379.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0380.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0381.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0382.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0383.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0384.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0385.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0386.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0387.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0388.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0389.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0390.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0391.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0392.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0393.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0394.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0396.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0397.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0398.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0399.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0400.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0401.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0402.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0403.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0404.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0405.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0406.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0407.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0408.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0409.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0410.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0411.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0412.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0413.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0414.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0415.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0416.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0417.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0418.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0419.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0420.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0421.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0422.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0423.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0424.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0425.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0426.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0427.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0428.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0429.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0430.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0431.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0432.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-0433.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Can't find a host by that name.
Downtime for panda-relay-007.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-008.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-009.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-010.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-011.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-012.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-013.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-014.p1.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-015.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-016.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-017.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-018.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-019.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-020.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-021.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-022.p2.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-031.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-032.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-033.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-034.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-035.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-036.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-037.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Downtime for panda-relay-038.p4.releng.scl1.mozilla.com of 172800 seconds scheduled.
Seems we are not monitoring the actual chassis's so no change there.

Found bug with dc-move.sh script, if a host is also a parent, it will grab ALL the records which is also a parent of. This is now fixed (ie only grabs the 1st and ONLY correct record. Same thing with hostgroups, only exact matches obtained now.

Revisions:
p1 panda linux (r89812.)
p1 panda boards (r89813.)
p1 panda relays (r89815.)
p2 panda linux (r89817.)
p2 panda boards,relays (r89818.)
p4 panda linux,boards,relays (r89819.)
Parent fixing (r89820.)

Following switches added to nagios (r89821,r89824)
switch1.r602-2.ops.releng.scl3.mozilla.net
switch1.r602-3.ops.releng.scl3.mozilla.net
switch1.r602-4.ops.releng.scl3.mozilla.net

Hosts downtimed in nagios for 3 days, etherpad updated and announcement made in #it && #releng
Err, links;
https://nagios.mozilla.org/releng-scl3/cgi-bin/status.cgi?hostgroup=pandas&style=overview
https://nagios.mozilla.org/releng-scl3/cgi-bin/status.cgi?hostgroup=mobile-imaging&style=overview

Essentially, would be ideal to have the actual chassis's grouped in nagios (ie P1, P2, P4 etc have their own hostgroups). Is this wanted? or perhaps there's a reason this wasn't done.
Amy, Callek -- any opinion on comment 4?
Flags: needinfo?(bugspam.Callek)
Flags: needinfo?(arich)
I'm not a fan of "pod" specific hostgroups for these, since as far as releng is concerned they are all a single group (all pandas are part of the same pool), etc.

IFF we can do hostgroups as sub groups of other hostgroups, e.g. "panda" --> "panda pod 1" --> 'panda-{}', then sure. Otherwise I feel the current way is best.
Flags: needinfo?(bugspam.Callek)
(Assignee)

Comment 7

3 years ago
AJ: the reason to have hostgroups is to associate them with service checks. They all get the exact same service checks, so it doesn't make sense to split them up and have to replicate that information 10 times (once per rack).

I suppose the only time having them split into additional hostgroups (not replacing the pandas hostgroup) would be useful is when downtiming (so you could do them all at once), but then you could split that a few different ways, per relay, per chassis, per pod, etc. I'm not sure that's entirely valuable since we almost never downtime machines at that level. We're only doing it here because they're moving.
Flags: needinfo?(arich)
(Assignee)

Updated

3 years ago
Depends on: 1033251
(Assignee)

Updated

3 years ago
Depends on: 1033350
One can have hostgroups just for labeling (ie no alerts associated with them) and hosts could have multiple hosts groups. It's really just for eye candy (ie easier to see if a particular pod is having issues etc).

Side note, removed the SCL1 switches that were related to this move (r89912.)
Removed the PDUs as well (r89953.)
(Assignee)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.