add more linux32 and linux64 aws masters

RESOLVED FIXED

Status

RESOLVED FIXED
3 years ago
9 months ago

People

(Reporter: kmoir, Assigned: kmoir)

Tracking

Details

Attachments

(4 attachments, 1 obsolete attachment)

(Assignee)

Description

3 years ago
catlee mentioned that master lag is high (see bug 1049657)

https://www.hostedgraphite.com/da5c920d/86a8384e-d9cf-4208-989b-9538a1a53e4b/grafana/#/dashboard/db/buildbot-masters

It looks we could use some new linux64 and and even linux32 masters
(Assignee)

Updated

3 years ago
Blocks: 1205785
(Assignee)

Updated

3 years ago
Assignee: nobody → kmoir
(Assignee)

Comment 3

3 years ago
Created attachment 8662984 [details] [diff] [review]
bug1205409tools.patch
Attachment #8662617 - Attachment is obsolete: true
(Assignee)

Updated

3 years ago
Attachment #8662983 - Flags: review?(rail)
(Assignee)

Updated

3 years ago
Attachment #8662984 - Flags: review?(rail)

Comment 4

3 years ago
Comment on attachment 8662984 [details] [diff] [review]
bug1205409tools.patch

Review of attachment 8662984 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm. Bonus point for removing trailing spaces. ;)
Attachment #8662984 - Flags: review?(rail) → review+

Updated

3 years ago
Attachment #8662983 - Flags: review?(rail) → review+
(Assignee)

Updated

3 years ago
Attachment #8662983 - Flags: checked-in+
(Assignee)

Updated

3 years ago
Attachment #8662984 - Flags: checked-in+
(Assignee)

Comment 5

3 years ago
In production
(Assignee)

Updated

3 years ago
Depends on: 1206222
(Assignee)

Updated

3 years ago
Depends on: 1207411
(Assignee)

Comment 6

3 years ago
Created attachment 8665103 [details] [diff] [review]
bug1205409enable.patch

patch to enable masters once I sort out the failing nagios checks
Attachment #8665103 - Flags: review?(rail)

Comment 7

3 years ago
Comment on attachment 8665103 [details] [diff] [review]
bug1205409enable.patch

\o/
Attachment #8665103 - Flags: review?(rail) → review+
(Assignee)

Comment 8

3 years ago
It looks like command_runner.py and pulse_publisher.py do not exist on these masters so there is something else wrong with the puppet config. Investigating.
(Assignee)

Comment 9

3 years ago
Created attachment 8666197 [details] [diff] [review]
bug1205409fixtypo.patch

found problem

must fix script that generates the puppet snippets too
Attachment #8666197 - Flags: review?(rail)

Updated

3 years ago
Attachment #8666197 - Flags: review?(rail) → review+
(Assignee)

Comment 10

3 years ago
Comment on attachment 8666197 [details] [diff] [review]
bug1205409fixtypo.patch

and merged to production
Attachment #8666197 - Flags: checked-in+
(Assignee)

Comment 11

3 years ago
Comment on attachment 8665103 [details] [diff] [review]
bug1205409enable.patch

I locked some machines to the master yesterday and it looks like the jobs that ran last night were okay.
Attachment #8665103 - Flags: checked-in+
(Assignee)

Comment 12

3 years ago
In production, masters enabled in slavealloc and running jobs. Nagios downtime cancelled.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.