Make sure that we only add as many builders as strictly necessary

RESOLVED WONTFIX

Status

P3
normal
RESOLVED WONTFIX
7 years ago
3 months ago

People

(Reporter: armenzg, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments, 1 obsolete attachment)

(Reporter)

Description

7 years ago
I have seen few cases on mozilla-tests/config.py and mozilla/project_branches.py

I will hopefully make it make more sense.

This is necessary as I have few more suites to add and I am hitting the 1024 limit.
(Reporter)

Comment 1

7 years ago
Created attachment 608480 [details] [diff] [review]
force project branches to be more explicit about what they require

This is my first pass.
I will try to get in touch with project owners and update their requirements as some branches like jaegermonkey are abandoned.
Attachment #608480 - Flags: review?(aki)
Created attachment 608536 [details] [diff] [review]
diff off removed/added builders

I'm not entirely sure what you're trying to do here, so I'm going to wait til you're back to verify.  It does look like you want to explicitly list what's in each project branch, which I approve of.  It's hard to figure out what our current programmatic config is doing.

In the meantime, I've run dump_masters to see what the net effect here will be.

First, run dump_masters to get clean output:

 ~/src/clean/braindump/buildbot-related/dump_masters.sh 2>&1 >../clean.dump

Then hg import and get patched output:

 hg import --no-commit ~/Desktop/less_builders.v4.diff
 ~/src/clean/braindump/buildbot-related/dump_masters.sh 2>&1 >../armen.dump

Try to only get the information of changed builders:

 egrep '^[><] [a-zA-Z]' factory.diff >x

Separate added and removed factory entries:

 grep '^>' x | sed -e 's/^> \(.*\)$/\1/'| sort | uniq >added
 grep '^<' x | sed -e 's/^< \(.*\)$/\1/'| sort | uniq >removed

Get a summary of added/removed builders:

 diff -U0 removed added > summary

Looks like we're removed 890 builders, which is great.  Does this list look correct?
Attachment #608536 - Flags: feedback?(armenzg)
Hm, I got factory diff via

 diff ../{clean,armen}.dump | grep -i factory > ../factory.diff

which I'm not sure is accurate, but then I doublechecked:

 diff clean.dump armen.dump | egrep '^[><] [a-zA-Z]' > X
 diff x X

They're the same.  I think the above file is still accurate.
see also bug 737661 where we have jetpack testers for platforms we don't build
(Reporter)

Comment 5

7 years ago
Comment on attachment 608536 [details] [diff] [review]
diff off removed/added builders

Thanks for taking the time.
I think my eyes were getting tired and just focusing on few branches.
I will have an updated patch in few hours.

BTW I use builder_list.py from braindump.py and comment out few lines to only output the build names.
Attachment #608536 - Flags: feedback?(armenzg) → feedback-
Attachment #608480 - Flags: review?(aki)
(Reporter)

Comment 6

7 years ago
Created attachment 609390 [details] [diff] [review]
remove unused branches

I will reach project owners for "graphics" and "jaegermonkey".

I would like to remove "Pine" from the available lists and add it when extremely necessary.
(Reporter)

Comment 7

7 years ago
I see "birch" (unused branch) as 'birch': {}, but I see all of these builders:
 Android birch build
 Android Debug birch build 
 Android XUL birch build 
 Linux birch build 
 Linux birch leak test build 
 Linux x86-64 birch build 
 Linux x86-64 birch leak test build 
 Linux QT birch build 
 OS X 10.5.2 birch leak test build 
 OS X 10.6.2 birch build 
 OS X 10.6.2 birch leak test build 
 WINNT 5.2 birch build 
 WINNT 5.2 birch leak test build 
 WINNT 6.1 x86-64 birch build

These are just from the Windows testers:
 Rev3 WINNT 5.1 birch talos nochrome.2 
 Rev3 WINNT 5.1 birch talos chrome.2 
 Rev3 WINNT 5.1 birch talos dromaeo 
 Rev3 WINNT 5.1 birch talos svg 
 Rev3 WINNT 5.1 birch talos tpr_responsiveness 
 Rev3 WINNT 5.1 birch talos dirty 
 Rev3 WINNT 5.1 birch opt test mochitests-1/5 
 Rev3 WINNT 5.1 birch opt test mochitests-2/5 
 Rev3 WINNT 5.1 birch opt test mochitests-3/5 
 Rev3 WINNT 5.1 birch opt test mochitests-4/5 
 Rev3 WINNT 5.1 birch opt test mochitests-5/5 
 Rev3 WINNT 5.1 birch opt test mochitest-other 
 Rev3 WINNT 5.1 birch opt test reftest 
 Rev3 WINNT 5.1 birch opt test crashtest 
 Rev3 WINNT 5.1 birch opt test xpcshell 
 Rev3 WINNT 5.1 birch opt test jsreftest 
 Rev3 WINNT 5.1 birch debug test mochitests-1/5 
 Rev3 WINNT 5.1 birch debug test mochitests-2/5 
 Rev3 WINNT 5.1 birch debug test mochitests-3/5 
 Rev3 WINNT 5.1 birch debug test mochitests-4/5 
 Rev3 WINNT 5.1 birch debug test mochitests-5/5 
 Rev3 WINNT 5.1 birch debug test mochitest-other 
 Rev3 WINNT 5.1 birch debug test reftest 
 Rev3 WINNT 5.1 birch debug test crashtest 
 Rev3 WINNT 5.1 birch debug test xpcshell 
 Rev3 WINNT 5.1 birch debug test jsreftest 
 Rev3 WINNT 6.1 birch talos nochrome.2 
 Rev3 WINNT 6.1 birch talos chrome.2 
 Rev3 WINNT 6.1 birch talos dromaeo 
 Rev3 WINNT 6.1 birch talos svg 
 Rev3 WINNT 6.1 birch talos tpr_responsiveness 
 Rev3 WINNT 6.1 birch talos dirty 
 Rev3 WINNT 6.1 birch opt test mochitests-1/5 
 Rev3 WINNT 6.1 birch opt test mochitests-2/5 
 Rev3 WINNT 6.1 birch opt test mochitests-3/5 
 Rev3 WINNT 6.1 birch opt test mochitests-4/5 
 Rev3 WINNT 6.1 birch opt test mochitests-5/5 
 Rev3 WINNT 6.1 birch opt test mochitest-other 
 Rev3 WINNT 6.1 birch opt test reftest 
 Rev3 WINNT 6.1 birch opt test crashtest 
 Rev3 WINNT 6.1 birch opt test xpcshell 
 Rev3 WINNT 6.1 birch opt test jsreftest 
 Rev3 WINNT 6.1 birch opt test reftest-no-accel 
 Rev3 WINNT 6.1 birch debug test mochitests-1/5 
 Rev3 WINNT 6.1 birch debug test mochitests-2/5 
 Rev3 WINNT 6.1 birch debug test mochitests-3/5 
 Rev3 WINNT 6.1 birch debug test mochitests-4/5 
 Rev3 WINNT 6.1 birch debug test mochitests-5/5 
 Rev3 WINNT 6.1 birch debug test mochitest-other 
 Rev3 WINNT 6.1 birch debug test reftest 
 Rev3 WINNT 6.1 birch debug test crashtest 
 Rev3 WINNT 6.1 birch debug test xpcshell 
 Rev3 WINNT 6.1 birch debug test jsreftest n
(Reporter)

Comment 8

7 years ago
Comment on attachment 609390 [details] [diff] [review]
remove unused branches

I got OK through emails from owners.
Attachment #609390 - Flags: review?(coop)
Attachment #609390 - Flags: review?(coop) → review+
(Reporter)

Comment 9

7 years ago
Comment on attachment 609390 [details] [diff] [review]
remove unused branches

559f2545345b landed on "default"
Attachment #609390 - Flags: checked-in+
(Reporter)

Comment 10

7 years ago
This went live around 11:55AM PDT.
(Reporter)

Updated

7 years ago
Priority: P1 → P3
(Reporter)

Comment 11

6 years ago
Created attachment 615361 [details] [diff] [review]
(optional) Check each existing attachment made obsolete by your new attachment. 608480: force project branches to be more explicit about what they require 608536: diff off removed/added builders 6093
Attachment #608480 - Attachment is obsolete: true
(Reporter)

Comment 12

6 years ago
I have my hands full.
I left attachment 615361 [details] [diff] [review] in case anyone wants to pick this up later on (even myself).
Assignee: armenzg → nobody
(Reporter)

Comment 13

6 years ago
Created attachment 619049 [details] [diff] [review]
[wip] talos jobs should be explicitely called out for project branches

I am just refreshing the patch.
I haven't touched this for a while.
I don't have time to work on it yet.
(Assignee)

Updated

5 years ago
Product: mozilla.org → Release Engineering
Probably not worth spending time on now that we're moving so much to Taskcluster.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
(Assignee)

Updated

3 months ago
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.