Closed Bug 774799 Opened 8 years ago Closed 8 years ago

Double builds scheduled for each platform on try

Categories

(Release Engineering :: General, defect)

defect
Not set
critical

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Unassigned)

References

Details

(Whiteboard: [buildduty][tryserver][capacity])

For every Try push after and inclusive of:
https://tbpl.mozilla.org/?tree=Try&rev=f0096d1ad4b0
2012-07-17 11:06 PDT

There have been two builds scheduled for each platform.

This is going to start to cause issues with Try load very soon :-(
I notice that none of the secondary builds have completed yet, though that may just be a matter of time.
Then again, that includes the B2G builds, which should have been done for a while.
This looks similar to bug 719260.
Talked to catlee, he looked at the logs and found that at 11:37:20 we got a 500 error from hg.m.o/try

/builds/buildbot/build_scheduler/master/twistd.log.5:2012-07-17 11:37:20-0700 [HTTPPageGetter,client] http://hg.mozilla.org/try has been reset

He suspects this is the root cause and subsequent builds should be fine.
Ok, thank you :-)

Do we need to do anything to mitigate this in the future?
I opened a bug to address this - see bug 774862.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
This is happening again; I pushed changesets f307976a7ae6 and 744b65845b23 both of which got double-builds queued. I manually cancelled the ones I don't care about.
the try poller reset itself again at 2012-07-18 07:20:54-0700. for some reason the server reports that the previous revision doesn't exist temporarily.

the revision it failed on happened to be yours actually...it was polling http://hg.mozilla.org/try/json-pushes?full=1&fromchange=f307976a7ae68117ad3cf651ad07ceb92cc49cd0&tipsonly=1
We had another batch of double builds on try last night, eg:
https://tbpl.mozilla.org/?tree=Try&rev=eaf06d352463
Blocks: 772458
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Summary: Double builds scheduled for each platform on try, for every push after 1100 PDT → Double builds scheduled for each platform on try
Ah, due to bug 770811, most of you won't be able to load that page easily :-(

The push in question:
amarchesini@moco – Fri Jul 20 03:23:21 2012 (UTC+1)
eaf06d352463 Andrea Marchesini – try: -b do -p all -u all -t none 

(There were many more like it, this was just the most recent push on which it occurred)
I've cancelled all doubled up running and pending tests (approx 100, all Windows).

There were over 30 pushes last night (UTC) that had doubled up builds; one plus side is that once this is resolved at least bug 772458 gets an easy win :-)
Component: Release Engineering → Release Engineering: Developer Tools
QA Contact: lsblakk
Whiteboard: [buildduty][tryserver][capacity]
this shouldn't be a problem now
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: Tools → General
You need to log in before you can comment on or make changes to this bug.