Closed Bug 1205241 Opened 9 years ago Closed 9 years ago

Make linux64 and MacOSX64 tc(B) run on integration branches, at tier 2

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Assigned: dustin)

Details

Attachments

(1 file)

MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek
40 bytes, text/x-review-board-request
dustin
: review+
Details 
      No description provided.
Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r?ted.mielczarek
Attachment #8661728 - Flags: review?(ted)
https://reviewboard.mozilla.org/r/19433/#review17391

I built the graphs (`./mach taskcluster-graph`) and diffed them.  We are already building linux64 opt/dbg on m-c (hidden), but not m-i, so the diff only showed MacOSX64 added to m-c, and all three added to m-i.
Summary: Make linux64 and MacOSX64 tc(B) run on integration branches → Make linux64 and MacOSX64 tc(B) run on integration branches, at tier 2
I think we'll need to request that these be hidden after this lands (AIUI from philor you can't request a hide before the task runs?).

From the Tier-2 Checklist:

> Has an active owner
> 
>     Who is committed to ensuring the other requirements are met not just initially, but over the long term.
>     Who will ensure the new job type is switched off to save resources should we stop finding it useful in the future.

Who should this be?  I feel like I'm stepping in to get this done because it needs to be done, but in a year I don't want to be the guy who gets pinged every time Linux builds go orange.

> Usable job logs
> ...

Not an issue since it's the same logs that we get from BB, more or less

> Has sufficient documentation
> ...

Ditto
Attachment #8661728 - Flags: review?(ted) → review+
Comment on attachment 8661728 [details]
MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

https://reviewboard.mozilla.org/r/19435/#review17399

::: testing/taskcluster/tasks/branches/base_jobs.yml:102
(Diff revision 1)
> +        task: tasks/builds/dbg_macosx64.yml

Note that my Mac task doesn't have the same clobber bits as yours. I think Mozharness will still respect the in-tree clobber, but this might make Try builds less useful. I don't know if that needs to block making these visible or not.
(In reply to Dustin J. Mitchell [:dustin] from comment #3)
> > Has an active owner
> > 
> >     Who is committed to ensuring the other requirements are met not just initially, but over the long term.
> >     Who will ensure the new job type is switched off to save resources should we stop finding it useful in the future.
> 
> Who should this be?  I feel like I'm stepping in to get this done because it
> needs to be done, but in a year I don't want to be the guy who gets pinged
> every time Linux builds go orange.

I hope we can say that RelEng and build peers are jointly tasked with ownership here.
Oh, right, I need to only use the clobber versions on try.  I can take a swing at setting up the same thing for OS X.  I'll re-draft.
I think ehsan had most of that done in his patch in...that bug about static analysis builds whose number escapes me.
Comment on attachment 8661728 [details]
MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r?ted.mielczarek

Properly splitting clobber (try) and non-clobber builds
Comment on attachment 8661728 [details]
MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

This didn't automatically re-flag you for review, sorry.
Attachment #8661728 - Flags: review+ → review?(ted)
Comment on attachment 8661728 [details]
MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

https://reviewboard.mozilla.org/r/19435/#review17683

::: testing/taskcluster/tasks/branches/try/job_flags.yml:191
(Diff revision 2)
> +        task: tasks/builds/dbg_macosx64_clobber.yml

I don't see dbg_macosx64_clobber.yml in this patch anywhere. Did it not get added?
Attachment #8661728 - Flags: review?(ted)
It was a rename.
I'm still not sure exactly how to deploy this.  When I land this, we'll have (hopefully green) letters showing up on treeherder.  However, I'm not sure how or where to indicate what should happen when those turn orange.  In other words, how do I indicate that a build is Tier 2?
Flags: needinfo?(dburns)
redirecting to camd.
Flags: needinfo?(dburns) → needinfo?(cdawson)
(In reply to Dustin J. Mitchell [:dustin] from comment #11)
> It was a rename.

OK, but it's not in the patch. (I checked the raw diff.)
Comment on attachment 8661728 [details]
MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

Properly splitting clobber (try) and non-clobber builds
Attachment #8661728 - Attachment description: MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r?ted.mielczarek → MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek
Attachment #8661728 - Flags: review?(ted)
I fixed up the confusion of dbg/opt_macosx64 in the patch and pushed to try:
  https://treeherder.mozilla.org/#/jobs?repo=try&revision=62cd34aa23f3
Comment on attachment 8661728 [details]
MozReview Request: Bug 1205241: enable linux64 opt/dbg and MacOSX64 builds on m-i and m-c; r=ted.mielczarek

"good if it works in try" from ted in irc
Attachment #8661728 - Flags: review?(ted) → review+
green, so just waiting on info from camd before landing this
Hey guys--  So to make a job Tier-2, there are 2 ways.

1. If using the API (like Task Cluster, not Buildbot) when you can post your jobs with a ``tier`` field value of, you guessed it: 2

2. If that's not an option, then Dustin is right.  You can only specify if AFTER the job has been posted to Treeherder once.  At that time, you can modify the Exclusion profile called ``Tier-2`` to include your jobs.

P.S. - I acknowledge option 2 is a bit hacky (a lot bit) but it was the best/fastest option we had to support Buildbot at the time.  :)
Flags: needinfo?(cdawson)
Whoa!  I just looked on production and I don't see the Tier-2 exclusion profile anymore.  I see a few Job Exclusions named "Tier-2 ..." included in the default exclusion profile.  So, those jobs WILL be hidden.  But that works differently.  They won't get their ``tier`` value marked as 2 by the process in #2 above.

Checking a few things to see what happened...
I went ahead and re-created that Exclusion profile hoping it was deleted by accident.  Anyway, probably not your concern, because you can have Task Cluster report them directly as Tier-2, I'm guessing.  But if not, please create a job exclusion for them after they're in, and add it to the "Tier-2" exclusion profile.

Note: Members of the "Tier-2" exclusion profile are not actually excluded by default.  You must ALSO add that Job Exclusion to the default exclusion profile for that.
I'm not sure what you mean by "excluded" - I would like these builds to be visible to developers in the default treeherder view, so that if they go orange in a try run they are inclined to try to fix them.  Isn't that generally what tier 2 entails?

And yes, I can easily add the "tier: 2" key -- pretty cool!
Ah, the "Tier 2" key is *not* functional, actually.  So I'll land this and we can figure out how to set up its profiles appropriately -- dunno if that's a TH patch I need to submit, or some TH UI I may or may not have access to.
A try run passed after rebasing, so I'll land.
Keywords: leave-open
With Cameron's help, I created a new job profile and hooked it up to the tier-2 exclusion profile.  We can easily add more jobs to this job profile as necessary.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: