Closed Bug 1023314 Opened 10 years ago Closed 10 years ago

Increase Android 2.3 and B2G reftest chunks

Categories

(Testing :: General, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gbrown, Assigned: gbrown)

Details

Attachments

(4 files)

Android 2.3 reftests sometimes exceed the 60 minute max time. This caused some aggravation in bug 1022002.

Android 2.3 reftest times vary considerably from one chunk to another, and sometimes from one run to another of the same chunk. Current times are in the range of 40 minutes to 60 minutes.

I don't want to increase the number of chunks again (more setup time, more slaves used per push, etc) without good reason, but it seems like there may be good reason.

Alternatively, I am really starting to wonder if we ought to change the 60 minute limit.
Funny, I was recently thinking about filing this exact bug for B2G as well (all running in 60-75min now). Don't suppose we can kill two birds here?
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #1)
> Don't suppose we can kill two birds here?

Sure - it makes sense.

Do we know what has happened? More tests? Something running slower?
I don't know, unfortunately.
I chatted with :ryanvm on irc earlier today and he clarified that the concern is not only that long-running jobs are in danger of failing, but also that they simply take so long to complete. A test job that takes longer to fail allows more changes to be landed on top of it, potentially complicating regression analysis and backouts. It also takes longer to verify a bustage fix, causing longer tree closures.
Summary: Consider increasing Android 2.3 reftest chunks → Increase Android 2.3 and B2G reftest chunks
:jgriffin -- I'm thinking of increasing B2G reftest chunks from 15 to 20. Does that seem reasonable?
Flags: needinfo?(jgriffin)
Yes, that makes sense to me.
Flags: needinfo?(jgriffin)
I don't usually modify B2G configs, but this looked simple...hope I am not missing anything.
Attachment #8437953 - Flags: review?(aki)
Attachment #8437937 - Flags: review?(kmoir) → review+
Attachment #8437938 - Flags: review?(kmoir) → review+
Attached file bug1023314builder.diff
I thought I was asked to review all the patches and so I looked at the b2g one too.  It looked good on my dev-master, here is the builder diff for reference. Anyways, I'll let aki review.
Checked in commits went live with reconfig on 2014-06-11 10:16 PT
Comment on attachment 8437953 [details] [diff] [review]
increase B2G reftest chunks from 15 to 20

Sorry for the delay, I was trying to figure out why we don't use totalChunks in b2g_config.py like we do in config.py: http://hg.mozilla.org/build/buildbot-configs/file/0939a9fa05fb/mozilla-tests/config.py#l293

That would save a lot of copy/pasting.  I don't have headspace for that right now, though, so I'm going to stop blocking this patch from landing.
Attachment #8437953 - Flags: review?(aki) → review+
Thanks :aki. totalChunks looks useful, but I don't have insight into why it's not used here. Another day...

https://hg.mozilla.org/build/buildbot-configs/rev/3ba6e52f8444
In prod with reconfig on 2014-06-12 10:46 PT
Verified on mozilla-inbound.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: