790510 - Add a fourth chunk to Android reftests

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Phil Ringnalda (:philor)]

If you look at the logs for the links pasted into bug 663657, where reftests time out after 1200 seconds with no output, you'll see that they are reftest-3.

If you look at the logs for the links pasted into bug 689856, where a job is running along, and then suddenly after 61 minutes it stops in midstream, you'll see that they are reftest-3.

If you look at the total time for Android test jobs, you'll see that it's around 30 minutes per suite, except that reftest-2 is 45 minutes, and reftest-3 is around 60-80 minutes. I'd assert that that's longer than you can hold the attention of a tegra, and we need to add reftest-4 to balance them out to small enough chunks.

Obviously, that's a releng thing, but if I file it there it'll get bounced to you to say whether or not to do it, so just move it there when you agree.

Comment 1

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Please define "you".

It sounds fine to me.

Comment 2

[Phil Ringnalda (:philor)]

Sorry, that was more clear while filing it, where the cc field and the comment field are close together - the you that releng will want to have pre-approve it is jmaher.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

This looks like a good thing to do!

Comment 4

[Phil Ringnalda (:philor)]

Attachment: configs

Slightly silly since the real problem is that skip-if(Android) isn't evenly distributed, so R1 does hardly any of its tests and R3 does almost all of them, but porting the reftest manifest parser to buildbot seems... unlikely to go well :)

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 660499 [details] [diff] [review]
configs

This looks good for android.
Did you want to do the same thing for android-armv6, or is that one fine?
If so, the relevant code is http://hg.mozilla.org/build/buildbot-configs/file/7d811e29a112/mozilla-tests/config.py#l632 .

Comment 6

[Phil Ringnalda (:philor)]

You're right, I do - despite the small number of runs so far, it shows ample evidence of needing the same thing.

Comment 7

[Phil Ringnalda (:philor)]

http://hg.mozilla.org/build/buildbot-configs/rev/50cb97f019e5

Comment 8

[Kim Moir [:kmoir] ET]

In production.

Comment 9

[Phil Ringnalda (:philor)]

Not sure this was a good idea after all - setup time is insane (I'm looking at a 65 minute R1 where the actual remotereftest.py step was 20 minutes), and the fix for bug 791427 may have actually been all we needed.

Comment 10

[Phil Ringnalda (:philor)]

But, despite the still-awful times, it did drop end-to-end time.