Closed
Bug 1013961
Opened 10 years ago
Closed 9 years ago
Get catlee's cool fabric actions in production for e.g. managing pulse queues on buildbot masters
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: pmoore, Assigned: pmoore)
References
Details
Attachments
(3 files)
8.31 KB,
patch
|
Details | Diff | Splinter Review | |
1.07 KB,
patch
|
nthomas
:
review+
Callek
:
review+
pmoore
:
checked-in+
|
Details | Diff | Splinter Review |
618 bytes,
patch
|
coop
:
review+
pmoore
:
checked-in+
|
Details | Diff | Splinter Review |
Catlee has written some extra fabric actions for managing buildbot masters, such as manipulating pulse queues. https://github.com/catlee/tools/compare/master...fabric#diff-8fd5c265985d88a725d36d2e7fddc64aR303 Let's get these reviewed and in production. :)
Attachment #8426236 -
Flags: review?(bugspam.Callek)
Attachment #8426236 -
Flags: feedback?(catlee)
Assignee | ||
Comment 1•10 years ago
|
||
Catlee says there are new dependencies introduced by the patch, which need addressing before it is reviewed. I'll remove the review flags...
Assignee | ||
Updated•10 years ago
|
Attachment #8426236 -
Flags: review?(bugspam.Callek)
Attachment #8426236 -
Flags: feedback?(catlee)
Updated•10 years ago
|
Blocks: byebyebuildduty
Assignee | ||
Comment 2•10 years ago
|
||
This is a subset of the actions that nthomas kindly gave me, which I was able to use to reset the queues on 1 Sep 2014 with bug 1061208. Not obsoleting the other patch, in case there are other parts we still also want to grab.
Attachment #8484322 -
Flags: review?(nthomas)
Comment 3•10 years ago
|
||
Comment on attachment 8484322 [details] [diff] [review] bug1013961_tools_v1.patch action_oldest_twistdlog and action_master_health are actually my hackery, but r+ on landing action_retry_dead_queue.
Attachment #8484322 -
Flags: review?(nthomas) → review+
Comment 4•10 years ago
|
||
Comment on attachment 8484322 [details] [diff] [review] bug1013961_tools_v1.patch Review of attachment 8484322 [details] [diff] [review]: ----------------------------------------------------------------- since nick disavows ability to r+ the others, allow me to do so.
Attachment #8484322 -
Flags: review+
Comment 5•10 years ago
|
||
action_oldest_twistdlog is really just cruft from a one-off weird case.
Assignee | ||
Comment 6•10 years ago
|
||
OK, I'll land action_master_health and action_retry_dead_queue...
Assignee | ||
Comment 7•10 years ago
|
||
Comment on attachment 8484322 [details] [diff] [review] bug1013961_tools_v1.patch Checked in, with the exception of the action_oldest_twistdlog method... I'll leave this bug open for now, as there may be other methods we want to mine from the original attachment.
Attachment #8484322 -
Flags: checked-in+
Assignee | ||
Comment 8•10 years ago
|
||
Cheekily tagging this onto this bug here, as an extra action I've added for seeing uptime of buildbot masters...
Assignee | ||
Comment 9•10 years ago
|
||
Output looks like this: (fabric)pmoore@Elisandra:~/git/tools master $ manage_masters.py -f buildfarm/maintenance/production-masters.json -R all -j 32 uptime bm85-build1 09:29:15 up 324 days, 18:00, 1 user, load average: 0.08, 0.05, 0.02 bm82-build1 09:29:15 up 324 days, 18:02, 1 user, load average: 0.03, 0.05, 0.06 bm81-build_scheduler 09:29:15 up 436 days, 2:37, 2 users, load average: 0.22, 0.23, 0.25 bm81-tests_scheduler 09:29:15 up 436 days, 2:37, 1 user, load average: 0.22, 0.23, 0.25 bm84-build1 09:29:15 up 324 days, 18:00, 1 user, load average: 0.20, 0.08, 0.04 bm83-try1 09:29:15 up 324 days, 18:02, 1 user, load average: 0.20, 0.17, 0.07 bm05-tests1-linux32 09:29:15 up 2:25, 1 user, load average: 0.04, 0.13, 0.07 bm06-tests1-linux32 09:29:15 up 1:36, 1 user, load average: 0.00, 0.02, 0.04 bm04-tests1-linux32 09:29:15 up 1:40, 1 user, load average: 0.00, 0.03, 0.08 bm53-tests1-linux64 09:29:15 up 547 days, 37 min, 1 user, load average: 1.14, 1.24, 1.23 bm68-tests1-linux64 09:29:15 up 534 days, 11 min, 1 user, load average: 1.22, 1.18, 1.14 bm54-tests1-linux64 09:29:15 up 546 days, 23:50, 1 user, load average: 1.07, 1.16, 1.16 bm72-build1 09:29:15 up 22:39, 2 users, load average: 0.04, 0.06, 0.11 bm73-build1 09:29:15 up 204 days, 6:00, 1 user, load average: 0.28, 0.17, 0.12 bm79-try1 09:29:15 up 203 days, 2:11, 1 user, load average: 0.16, 0.07, 0.05 bm74-build1 09:29:15 up 204 days, 6:17, 1 user, load average: 0.02, 0.24, 0.25 bm78-try1 09:29:15 up 203 days, 3:20, 1 user, load average: 0.00, 0.07, 0.05 bm86-build1 09:29:16 up 324 days, 18:00, 1 user, load average: 0.08, 0.10, 0.09 bm87-try1 09:29:16 up 324 days, 18:00, 1 user, load average: 0.08, 0.05, 0.01 bm89-tests1-panda 09:29:16 up 282 days, 23:51, 1 user, load average: 0.19, 0.14, 0.15 bm52-tests1-linux64 09:29:16 up 223 days, 23:12, 1 user, load average: 1.06, 1.23, 1.11 bm51-tests1-linux64 09:29:16 up 547 days, 1:32, 1 user, load average: 0.97, 1.06, 0.95 bm71-build1 09:29:16 up 204 days, 7:04, 1 user, load average: 1.20, 0.57, 0.36 bm67-tests1-linux64 09:29:16 up 534 days, 44 min, 1 user, load average: 0.71, 0.78, 0.87 bm77-build1 09:29:16 up 203 days, 4:21, 1 user, load average: 0.42, 0.47, 0.35 bm76-try1 09:29:16 up 203 days, 4:38, 1 user, load average: 0.07, 0.12, 0.19 bm01-tests1-linux32 09:29:16 up 23:16, 1 user, load average: 1.68, 1.97, 1.92 bm75-try1 09:29:16 up 191 days, 22:59, 1 user, load average: 0.01, 0.12, 0.18 bm03-tests1-linux32 09:29:16 up 191 days, 22:03, 1 user, load average: 1.91, 1.82, 1.76 bm02-tests1-linux32 09:29:16 up 191 days, 22:04, 1 user, load average: 1.87, 1.73, 1.66 bm70-build1 09:29:16 up 207 days, 18:21, 1 user, load average: 1.06, 0.37, 0.36 bm91-build1 09:29:16 up 3:03, 1 user, load average: 0.08, 0.11, 0.14 bm100-tests1-panda 09:29:17 up 280 days, 23:36, 1 user, load average: 0.23, 0.14, 0.14 bm102-tests1-panda 09:29:17 up 280 days, 23:41, 1 user, load average: 0.13, 0.13, 0.15 bm101-tests1-panda 09:29:17 up 281 days, 1:21, 1 user, load average: 0.05, 0.09, 0.12 bm104-tests1-linux 09:29:17 up 224 days, 22:38, 1 user, load average: 0.12, 0.12, 0.09 bm103-tests1-linux 09:29:17 up 224 days, 21:46, 1 user, load average: 0.00, 0.00, 0.00 bm105-tests1-linux 09:29:17 up 224 days, 22:38, 1 user, load average: 0.00, 0.00, 0.00 bm106-tests1-macosx 09:29:17 up 224 days, 22:37, 1 user, load average: 0.26, 0.57, 0.64 bm108-tests1-macosx 09:29:17 up 224 days, 22:36, 1 user, load average: 0.62, 0.79, 0.75 bm107-tests1-macosx 09:29:17 up 224 days, 22:37, 1 user, load average: 1.47, 0.99, 0.78 bm109-tests1-windows 09:29:18 up 224 days, 22:35, 1 user, load average: 1.51, 1.22, 1.11 bm110-tests1-windows 09:29:18 up 224 days, 22:35, 1 user, load average: 1.02, 1.10, 1.12 bm112-tests1-windows 09:29:18 up 224 days, 22:34, 1 user, load average: 1.57, 1.41, 1.22 bm111-tests1-windows 09:29:18 up 224 days, 22:29, 1 user, load average: 1.11, 1.13, 1.14 bm115-tests1-linux64 09:29:18 up 63 days, 14:02, 1 user, load average: 0.68, 0.76, 0.91 bm94-build1 09:29:18 up 191 days, 17:32, 1 user, load average: 0.09, 0.13, 0.28 bm116-tests1-linux64 09:29:19 up 60 days, 17:20, 1 user, load average: 0.62, 0.85, 1.02 bm118-tests1-linux64 09:29:19 up 3:23, 1 user, load average: 0.59, 0.67, 0.88 bm114-tests1-linux64 09:29:19 up 60 days, 17:12, 1 user, load average: 0.67, 0.79, 0.81 bm113-tests1-linux64 09:29:19 up 60 days, 17:20, 1 user, load average: 0.85, 0.67, 0.69 bm117-tests1-linux64 09:29:19 up 66 days, 19:26, 1 user, load average: 0.98, 0.93, 0.79 ================================================================================ (fabric)pmoore@Elisandra:~/git/tools master $
Updated•10 years ago
|
Attachment #8496049 -
Flags: review?(coop) → review+
Assignee | ||
Comment 10•10 years ago
|
||
Comment on attachment 8496049 [details] [diff] [review] bug1013961_tools_uptime_v1.patch http://hg.mozilla.org/build/tools/rev/2d2c52fdfc4b
Attachment #8496049 -
Flags: checked-in+
Updated•9 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Updated•7 years ago
|
Component: Tools → General
You need to log in
before you can comment on or make changes to this bug.
Description
•