820958 - Increase B2G emulator reftest chunks

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Andrew Halberstadt [:ahal]]

I've enabled a subset of important reftests according to cjones. However with the current 6 chunks we have the jobs take 45-60+ minutes. I think we should bump the chunks to either 8, 9 or 10

Comment 1

[Andrew Halberstadt [:ahal]]

(this is currently running on cedar only)

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to Andrew Halberstadt [:ahal] from comment #0)
> I've enabled a subset of important reftests according to cjones. However
> with the current 6 chunks we have the jobs take 45-60+ minutes. I think we
> should bump the chunks to either 8, 9 or 10

Ideally, we like each chunk to take ~25-30mins to complete. Given the timings you presented above, it sounds like 10? 12? chunks would be appropriate. Hows that sound to you?

Comment 3

[Andrew Halberstadt [:ahal]]

(In reply to John O'Duinn [:joduinn] from comment #2)
> Ideally, we like each chunk to take ~25-30mins to complete. Given the
> timings you presented above, it sounds like 10? 12? chunks would be
> appropriate. Hows that sound to you?

That sounds good. 12 will leave a bit more room to enable additional tests in the future, but might be a bit overkill. I'll defer to releng's judgement here.

This will hopefully only be until we have pandas running reftests reliably.

Comment 4

[Aki Sasaki (not active)]

Attachment: b2g emulator reftest chunks -> 10

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 691478 [details] [diff] [review]
b2g emulator reftest chunks -> 10

http://hg.mozilla.org/build/buildbot-configs/rev/9492b9665248

Comment 6

[Phil Ringnalda (:philor)]

I don't suppose there's any way to trim any of the crazy 10 minutes of setup/teardown time on those runs? That took us from burning an hour of overhead per push to burning 100 minutes, which given the 12-24 hour wait times for linux32 talos slaves on try right now is going to be painful when these run on more branches.

Comment 7

[Andrew Halberstadt [:ahal]]

(In reply to Phil Ringnalda (:philor) from comment #6)
> I don't suppose there's any way to trim any of the crazy 10 minutes of
> setup/teardown time on those runs? That took us from burning an hour of
> overhead per push to burning 100 minutes, which given the 12-24 hour wait
> times for linux32 talos slaves on try right now is going to be painful when
> these run on more branches.

Actually now that busybox is on tooltool we can use that to speed up the process of installing gecko. I don't know what the actual wins would be, but I'll work on getting that finished by the end of the week.

Comment 8

[Aki Sasaki (not active)]

This is in production.