Increase Android 2.3 and B2G reftest chunks

RESOLVED FIXED

Status

RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: gbrown, Assigned: gbrown)

Tracking

Trunk
x86_64
Linux
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

(Assignee)

Description

5 years ago
Android 2.3 reftests sometimes exceed the 60 minute max time. This caused some aggravation in bug 1022002.

Android 2.3 reftest times vary considerably from one chunk to another, and sometimes from one run to another of the same chunk. Current times are in the range of 40 minutes to 60 minutes.

I don't want to increase the number of chunks again (more setup time, more slaves used per push, etc) without good reason, but it seems like there may be good reason.

Alternatively, I am really starting to wonder if we ought to change the 60 minute limit.
Funny, I was recently thinking about filing this exact bug for B2G as well (all running in 60-75min now). Don't suppose we can kill two birds here?
(Assignee)

Comment 2

5 years ago
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #1)
> Don't suppose we can kill two birds here?

Sure - it makes sense.

Do we know what has happened? More tests? Something running slower?
I don't know, unfortunately.
(Assignee)

Comment 4

5 years ago
I chatted with :ryanvm on irc earlier today and he clarified that the concern is not only that long-running jobs are in danger of failing, but also that they simply take so long to complete. A test job that takes longer to fail allows more changes to be landed on top of it, potentially complicating regression analysis and backouts. It also takes longer to verify a bustage fix, causing longer tree closures.
Summary: Consider increasing Android 2.3 reftest chunks → Increase Android 2.3 and B2G reftest chunks
(Assignee)

Comment 5

5 years ago
Created attachment 8437937 [details] [diff] [review]
increase Android 2.3 reftest chunks from 12 to 16 (buildbot)
Attachment #8437937 - Flags: review?(kmoir)
(Assignee)

Comment 6

5 years ago
Created attachment 8437938 [details] [diff] [review]
increase Android 2.3 reftest chunks from 12 to 16 (mozharness)
Attachment #8437938 - Flags: review?(kmoir)
(Assignee)

Comment 7

5 years ago
:jgriffin -- I'm thinking of increasing B2G reftest chunks from 15 to 20. Does that seem reasonable?
Flags: needinfo?(jgriffin)
Yes, that makes sense to me.
Flags: needinfo?(jgriffin)
(Assignee)

Comment 9

5 years ago
Created attachment 8437953 [details] [diff] [review]
increase B2G reftest chunks from 15 to 20

I don't usually modify B2G configs, but this looked simple...hope I am not missing anything.
Attachment #8437953 - Flags: review?(aki)

Updated

5 years ago
Attachment #8437937 - Flags: review?(kmoir) → review+

Updated

5 years ago
Attachment #8437938 - Flags: review?(kmoir) → review+
Created attachment 8438465 [details]
bug1023314builder.diff

I thought I was asked to review all the patches and so I looked at the b2g one too.  It looked good on my dev-master, here is the builder diff for reference. Anyways, I'll let aki review.
Checked in commits went live with reconfig on 2014-06-11 10:16 PT
Comment on attachment 8437953 [details] [diff] [review]
increase B2G reftest chunks from 15 to 20

Sorry for the delay, I was trying to figure out why we don't use totalChunks in b2g_config.py like we do in config.py: http://hg.mozilla.org/build/buildbot-configs/file/0939a9fa05fb/mozilla-tests/config.py#l293

That would save a lot of copy/pasting.  I don't have headspace for that right now, though, so I'm going to stop blocking this patch from landing.
Attachment #8437953 - Flags: review?(aki) → review+
(Assignee)

Comment 15

5 years ago
Thanks :aki. totalChunks looks useful, but I don't have insight into why it's not used here. Another day...

https://hg.mozilla.org/build/buildbot-configs/rev/3ba6e52f8444
In prod with reconfig on 2014-06-12 10:46 PT
(Assignee)

Comment 17

5 years ago
Verified on mozilla-inbound.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.