self-serve should be able to disable coalescing (aka retriggering builds manually should never coalesce)

RESOLVED FIXED

Status

Release Engineering
General
RESOLVED FIXED
7 years ago
5 years ago

People

(Reporter: philor, Assigned: emorley)

Tracking

({buildapi})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [self-serve][sheriff-want])

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

7 years ago
I'm not sure whether it should always avoid coalescing, or should just have the option of avoiding it, but for my use of self-serve, which is primarily "I need to know which of these three pushes to back out because it caused this failure that happens on between one in four and two in four runs of a suite, so I need to have four runs of the suite on each of two or three pushes" coalescing makes the task absolutely miserable. Not only do I have to only trigger one run at a time, having to wait until it's picked up before triggering a second one (third, fourth, twelfth), but every time between the trigger and the pickup I have to worry about the way that any more recent push might finish a build and schedule the same suite and coalesce away my job.
(Reporter)

Comment 1

7 years ago
Probably I do want an option, since really that "somebody's getting backed out" use is when I really desperately want it and want it now, but the vast majority of my actual use by number of jobs is retriggering Android jobs from within tbpl, where I generally don't much care whether or not it gets coalesced, just that it runs again after that infra failure, and gets a chance to get as far as a code failure instead. So an option (somehow jammed) in buildapi/self-serve/, where I usually am for regression hunting since I'm having to trigger things that got coalesced the first run, too, would work fine.
I think this could work:

* self-serve accepts optional parameter to disable coalescing; sets this as a buildset property on the new build request (e.g. 'mergeable': False)

* modify our mergeRequests function (http://hg.mozilla.org/build/buildbotcustom/file/74df664b9053/misc.py#l348) to check for this property and return False if either request has it set
Whiteboard: [self-serve]

Updated

7 years ago
Priority: -- → P5
(Reporter)

Updated

6 years ago
Blocks: 755192
(Assignee)

Comment 3

6 years ago
This bites myself, philor and RyanVM quite a lot.
It would be great if manually retriggered jobs would never get coalesced :-)
Severity: enhancement → normal
Priority: P5 → --
Summary: self-serve should be able to disable coalescing → self-serve should be able to disable coalescing (aka retriggering builds manually should never coalesce)
(Assignee)

Updated

6 years ago
Whiteboard: [self-serve] → [self-serve][sheriff-want]
(Assignee)

Updated

6 years ago
Blocks: 784348
(Assignee)

Comment 4

6 years ago
Thinking about comment 1 - I'm not that sure we really do need an option any more (given the reduced Android failure rate) - which would simplify the implementation (we'd otherwise need two sets of rebuild columns on the self-serve columns, or else have to tuck the option away somewhere, or ...?) and mean I can still use TBPL's UI.

I'm trying to pick through the buildbotcustom/buildbot repo now, but I know absolutely nothing about them. Is there an easy way to tell from http://hg.mozilla.org/build/buildbotcustom/file/74df664b9053/misc.py#l348 whether the request came from self-serve as opposed to the schedulers? If so we could just never merge if it was a self-serve request and resolve this pretty easily.
(Assignee)

Comment 5

6 years ago
Created attachment 653782 [details] [diff] [review]
Don't coalesce on retriggers

I'm a complete buildbot (and python) n00b, so this is probably wrong - but hey :-)
Assignee: nobody → bmo
Status: NEW → ASSIGNED
Attachment #653782 - Flags: feedback?(catlee)
Comment on attachment 653782 [details] [diff] [review]
Don't coalesce on retriggers

Review of attachment 653782 [details] [diff] [review]:
-----------------------------------------------------------------

Yeah, this is the right place to do it. There are some other strings that show up in the reason field that we may want to use as well. E.g.
"Requested via self-serve by foo@example.com" for new builds rather than rebuilds

Alternatively, we could make sure that all actions performed via self-serve include "self-serve" in the reason. e.g. http://hg.mozilla.org/build/buildapi/file/6a71c42d0895/buildapi/scripts/selfserve-agent.py#l237
Attachment #653782 - Flags: feedback?(catlee) → feedback+
(Assignee)

Updated

6 years ago
Depends on: 786290
(Assignee)

Comment 7

6 years ago
Created attachment 656025 [details] [diff] [review]
Don't coalesce on retriggers v2

Good suggestion, I've filed bug 786290 to add a new 'Self-serve' prefix.

Patch updated to use this new string.
Attachment #653782 - Attachment is obsolete: true
Attachment #656025 - Flags: review?(catlee)

Updated

6 years ago
Attachment #656025 - Flags: review?(catlee) → review+
(Assignee)

Comment 8

6 years ago
Comment on attachment 656025 [details] [diff] [review]
Don't coalesce on retriggers v2

Thank you for the review :-)

http://hg.mozilla.org/build/buildbotcustom/rev/08ab818fd5a1
Attachment #656025 - Flags: checked-in+

Comment 9

6 years ago
Live in production.
(Assignee)

Comment 10

6 years ago
I tested this earlier but the builds still seemed to coalesce.
Philor/Ryan, could you keep an eye out and see if it is working next time we have a retrigger-party? :-)
If/when this happens again, if you could point to particular revisions and builder names, that would help a lot!
(Assignee)

Comment 12

6 years ago
(In reply to Chris AtLee [:catlee] from comment #11)
> If/when this happens again, if you could point to particular revisions and
> builder names, that would help a lot!

Will do :-)

(In this instance I was mid-eating dinner so ended up losing track of which tab/revision/..., but I'll pay more attention next time)
(Assignee)

Comment 13

6 years ago
Seems to be working fine now :-)
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
(Assignee)

Updated

6 years ago
Keywords: buildapi
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.