Double builds scheduled for each platform on try

RESOLVED FIXED

Status

Release Engineering
General
--
critical
RESOLVED FIXED
6 years ago
7 months ago

People

(Reporter: emorley, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [buildduty][tryserver][capacity])

(Reporter)

Description

6 years ago
For every Try push after and inclusive of:
https://tbpl.mozilla.org/?tree=Try&rev=f0096d1ad4b0
2012-07-17 11:06 PDT

There have been two builds scheduled for each platform.

This is going to start to cause issues with Try load very soon :-(
I notice that none of the secondary builds have completed yet, though that may just be a matter of time.
Then again, that includes the B2G builds, which should have been done for a while.

Comment 3

6 years ago
This looks similar to bug 719260.

Comment 4

6 years ago
Talked to catlee, he looked at the logs and found that at 11:37:20 we got a 500 error from hg.m.o/try

/builds/buildbot/build_scheduler/master/twistd.log.5:2012-07-17 11:37:20-0700 [HTTPPageGetter,client] http://hg.mozilla.org/try has been reset

He suspects this is the root cause and subsequent builds should be fine.
(Reporter)

Comment 5

6 years ago
Ok, thank you :-)

Do we need to do anything to mitigate this in the future?

Comment 6

6 years ago
I opened a bug to address this - see bug 774862.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
This is happening again; I pushed changesets f307976a7ae6 and 744b65845b23 both of which got double-builds queued. I manually cancelled the ones I don't care about.
the try poller reset itself again at 2012-07-18 07:20:54-0700. for some reason the server reports that the previous revision doesn't exist temporarily.

the revision it failed on happened to be yours actually...it was polling http://hg.mozilla.org/try/json-pushes?full=1&fromchange=f307976a7ae68117ad3cf651ad07ceb92cc49cd0&tipsonly=1

Updated

6 years ago
Depends on: 774862
(Reporter)

Comment 9

6 years ago
We had another batch of double builds on try last night, eg:
https://tbpl.mozilla.org/?tree=Try&rev=eaf06d352463
Blocks: 772458
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Summary: Double builds scheduled for each platform on try, for every push after 1100 PDT → Double builds scheduled for each platform on try
(Reporter)

Comment 10

6 years ago
Ah, due to bug 770811, most of you won't be able to load that page easily :-(

The push in question:
amarchesini@moco – Fri Jul 20 03:23:21 2012 (UTC+1)
eaf06d352463 Andrea Marchesini – try: -b do -p all -u all -t none 

(There were many more like it, this was just the most recent push on which it occurred)
(Reporter)

Comment 11

6 years ago
I've cancelled all doubled up running and pending tests (approx 100, all Windows).

There were over 30 pushes last night (UTC) that had doubled up builds; one plus side is that once this is resolved at least bug 772458 gets an easy win :-)

Updated

6 years ago
Component: Release Engineering → Release Engineering: Developer Tools
QA Contact: lsblakk
Whiteboard: [buildduty][tryserver][capacity]
this shouldn't be a problem now
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago5 years ago
Resolution: --- → FIXED
(Assignee)

Updated

4 years ago
Product: mozilla.org → Release Engineering
(Assignee)

Updated

7 months ago
Component: Tools → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.