Closed
Bug 1027437
Opened 10 years ago
Closed 10 years ago
add more slave names to non jacuzzi builders
Categories
(Release Engineering :: General, defect, P1)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jlund, Assigned: rail)
References
Details
Attachments
(2 files)
1.09 KB,
patch
|
catlee
:
review+
rail
:
checked-in+
|
Details | Diff | Splinter Review |
21.76 KB,
text/plain
|
Details |
we had a tree closure lasting over a few hours due to aws pending jobs (compile/build jobs) not finding a valid spot.
Theories included spot bidding prices being too high, not enough ip's in the AZ subnet group, and not enough slave names for non jacuzzi builders.
Patches were put in place for the first two theories but it seems like only upping the ondemand limit actually solved the issue. This is of course is not a solution so we should look at adding more slave names to our non-jacuzzi builders.
Apparently we added locked a lot of slaves to our jacuzzis in the the last week and this closure may be the result of that bug.
see https://bugzilla.mozilla.org/show_bug.cgi?id=1027308#c16 for more details
Reporter | ||
Comment 1•10 years ago
|
||
also rail, we added a subnet that appeared missing in our config for east-1c: https://hg.mozilla.org/build/cloud-tools/rev/ec66195a91fd
and, we should probably revert these two band aid fixes once we fix this bug:
https://hg.mozilla.org/build/cloud-tools/rev/eae3f6598284
https://hg.mozilla.org/build/cloud-tools/rev/64155873112f
Assignee | ||
Updated•10 years ago
|
Assignee: nobody → rail
Severity: normal → major
Priority: -- → P1
Assignee | ||
Comment 3•10 years ago
|
||
Once this is in production I'll need to add the slave to slavealloc, that's it.
Attachment #8442882 -
Flags: review?(catlee)
Updated•10 years ago
|
Attachment #8442882 -
Flags: review?(catlee) → review+
Assignee | ||
Comment 4•10 years ago
|
||
Comment on attachment 8442882 [details] [diff] [review]
spots.diff
https://hg.mozilla.org/build/buildbot-configs/rev/a51f0c4920c4
Attachment #8442882 -
Flags: checked-in+
Assignee | ||
Comment 5•10 years ago
|
||
To be added to slavealloc
Assignee | ||
Comment 6•10 years ago
|
||
(In reply to Rail Aliiev [:rail] from comment #5)
> Created attachment 8442900 [details]
> spot-bld.csv
>
> To be added to slavealloc
Done
Reporter | ||
Comment 7•10 years ago
|
||
buildbot-config patch live in production :)
Assignee | ||
Updated•10 years ago
|
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Updated•8 years ago
|
Component: Tools → General
You need to log in
before you can comment on or make changes to this bug.
Description
•