586664 - Normalize builder names

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect, P3)

Description

[Chris AtLee [:catlee]]

Our builder names are a bit of a hodgepodge of different formats, making it hard to write tools that analyze build history.

Some examples:

'try' vs 'tryserver'

Using '-' as the branch/platform/job type delimiter even though '-' also appears in branch names and job types.

I propose using this format:
$branch_$platform_$jobtype

For mobile, you can have
$branch+$mobilebranch_$platform_$jobtype

'tryserver' should go away and be replaced by 'try'

Comment 1

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #0)
> For mobile, you can have
> $branch+$mobilebranch_$platform_$jobtype

I tried to include the mobile branch in the builder names when I originally landed generateMobileBranchObjects and there was pushback for that being too long.  If I remember correctly, the builder name is what shows up as the title of the tinderbox waterfall.

Examples would be:

mozilla-central+mobile-browser_maemo5-gtk_nightly
mozilla-central+mobile-browser_maemo5-gtk_talos_tdhtml


If we care about what things look like on plain tinderbox, we should test that this properly wraps a long string and verify that this is OK with people who consume the mobile tinderbox page.

Are we going to say that $branch == repoPath and $mobilebranch == mobileRepoPath or are we going to do $branch == repoPath.rstrip('/').split('/')[-1]

Comment 2

[Chris AtLee [:catlee]]

(In reply to comment #1)
> (In reply to comment #0)
> > For mobile, you can have
> > $branch+$mobilebranch_$platform_$jobtype
> 
> I tried to include the mobile branch in the builder names when I originally
> landed generateMobileBranchObjects and there was pushback for that being too
> long.  If I remember correctly, the builder name is what shows up as the title
> of the tinderbox waterfall.

There are two formats to the builder name, the short name (above), and the long pretty name, e.g. Rev3 Linux mozilla-central XXXX.  The long name shows up on tinderbox, and in the schedulerdb.  The short name shows up on slaves, and in the statusdb.  They should be made consistent as well.  Any reason to not make short name == long name?

> 
> Examples would be:
> 
> mozilla-central+mobile-browser_maemo5-gtk_nightly
> mozilla-central+mobile-browser_maemo5-gtk_talos_tdhtml
> 
> 
> If we care about what things look like on plain tinderbox, we should test that
> this properly wraps a long string and verify that this is OK with people who
> consume the mobile tinderbox page.
> 
> Are we going to say that $branch == repoPath and $mobilebranch ==
> mobileRepoPath or are we going to do $branch ==
> repoPath.rstrip('/').split('/')[-1]

Comment 3

[Aki Sasaki (not active)]

It also shows up on ftp.m.o I think.
What do we gain by putting the mobile branch in there?

Negatives for me include
* longer sendchanges
* longer ftp directory names
* puts even more roadblocks in front of merging mobile into m-c
* we hit directory name limits on wince/winmo builds on a recurring basis, forcing us to use a name->dirname map
* this seems to encourage people to ask the question "why can't we build multiple mobile branches against multiple mozilla branches"

Comment 4

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #3)
> It also shows up on ftp.m.o I think.

According to http://hg.mozilla.org/build/buildbotcustom/file/c568cf50e7f3/misc.py#l2002 it is not going to lengthen the FTP location at all.  We could add the mobile branch name to the ftp location, but that is not related to the builder name.


> What do we gain by putting the mobile branch in there?

On the off chance that we build mozilla-central with mobile-2.0 (if it is created) and mobile-browser.  I hope we don't do that but it is a possibility

> 
> Negatives for me include
> * longer sendchanges
> * longer ftp directory names
> * puts even more roadblocks in front of merging mobile into m-c

agreed, but while we are building in this configuration, it seems useful to have this information.

> * we hit directory name limits on wince/winmo builds on a recurring basis,
> forcing us to use a name->dirname map

We could use a shorter builddir on the builders, alternatively using slavebuilddir to have the slave side build directory to be a shortened version of the masterside build dir.  I.e. transform mozilla-central+mobile-browser_maemo5-gtk_build to cent_brow_5-gtk_bld if we need to.

> * this seems to encourage people to ask the question "why can't we build
> multiple mobile branches against multiple mozilla branches"

Maybe what we can do is have short-name == long-name for desktop and branch_platform_type for the long name on mobile and branch+mobilebranch_platform_type for the shortname.  I know this makes the long name shorter than the short name, but this would allow us to have a unified outward appearance as well as giving our analysis tools enough information to work with on the backend.

Comment 5

[Aki Sasaki (not active)]

(In reply to comment #4)
> > What do we gain by putting the mobile branch in there?
> 
> On the off chance that we build mozilla-central with mobile-2.0 (if it is
> created) and mobile-browser.  I hope we don't do that but it is a possibility

If that's the only thing, could we/should we hold off til that is more of a reality?

mozilla-2.0 is looming, so long term a mobile-2.0 could be built off that.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

What if we did the following:

-desktop shortname == desktop longname == new format
-mobile shortname == new format
-mobile longname uses the current format

Do we know what the plan for fennec-2.0 is for branching?

Comment 7

[Aki Sasaki (not active)]

Discussed w/ jhford in person.

If
- "new format" for mobile shortname includes +mobile-branch
- "current format" for mobile longname is equivalent to the desktop new format
- there isn't an added benefit we're missing (we only gain the ability to have a many-to-many relationship between mobile branches and mozilla branches)

then there isn't as urgent a reason to change the mobile shortname immediately.

I don't know the fennec 2.0 branch plan, probably because it's still in discussion. We have the potential to have a long-running alpha relbranch, or create new repos, or deal with building the next alpha or beta off of default tip.

Comment 8

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Found in triage. 

A lot has changed in the 3 years since this bug was filed, including merging mobile-> m-c. Moving to GeneralAutomation where we can figure out what is still to do here.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

This seems pretty straightforward. Any objections to me taking this on?

Comment 10

[Gregory Szorc [:gps]]

Bug 913658 is similar and I /think/ the the underlying request is the same: ability to extract metadata about a builder. Personally, I'd rather see bug 913658 over this bug. But the treeherder people are more important than me.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

I don't see why making the relatively simple and straightforward changes here has to be an either/or proposition with bug 913658. I will point out that I'm volunteering to do this *now* vs. "somebody" doing bug 913658 whenever they get around to it or are told to do so by their manager.

Comment 12

[Gregory Szorc [:gps]]

I'm just saying that I /think/ the net effect of both the bugs is likely similar. Bug 913658 is likely the more robust and reusable solution.

Comment 13

[Jonathan Griffin (:jgriffin)]

I think it would be good to fix, but this may impact pulse and the logparser, so we'll have to check those carefully before deploying changes.

Comment 15

[[e-mail] (:bhearsum)]

Does this bug still matter and/or is it worthwhile given the move to Taskcluster?