821012 - B2G automation should use busybox during the install_gecko step if it's available

Status: RESOLVED FIXED

Description

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

The setup time is about 6-10 minutes. There are now 18 B2G emulator chunks across the various tests which equals almost 3 hours of slave time per push wasted on setup. Lets try and speed this up.

One quick win is to install busybox on the emulators and use that to copy all the gecko files over at once. I don't know how much faster it will be.

Comment 1

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Created attachment 697935 [details] [diff] [review]
Patch 1.0 - install with busybox

I'll push this to ash to test it out since it requires a mozharness change too.

Comment 2

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Created attachment 697945 [details] [diff] [review]
Patch 1.0 (mozharness) - pass in busybox

Comment 3

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Pushed to ash for testing:
https://tbpl.mozilla.org/?tree=Ash&rev=dcefa32b7944

Comment 4

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Preliminary results seem to show that this cuts down 2-3 minutes per chunk. That's a bigger win than I was expecting.

Comment 5

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

I'll have to land the gecko patch everywhere before landing the mozharness one. I'll wait till next week when I'll be around all day to supervise it.

Comment 6

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

In order to make sure we don't get any failures I had to commit a part 1 gecko patch which is just the changes to add the busybox arg to the reftest and mochitest harnesses.

https://hg.mozilla.org/integration/mozilla-inbound/rev/7744b9769935
https://hg.mozilla.org/releases/mozilla-aurora/rev/f705a043ad9c
https://hg.mozilla.org/releases/mozilla-b2g18/rev/171baeed6231

Once this lands on central I'll commit the mozharness patch, then I can safely commit the marionette portion of the gecko patch.

Comment 7

[Ed Morley [:emorley]]

Part 1 landed with an incorrect bug number (bug 812012). Happened to spot it by chance, but not until after the script had closed that bug. 

Ideally please can we have a backout + reland (n the same push, with DONTBUILD on the topmost changeset), so hg annotate shows the correct bug number?

https://hg.mozilla.org/mozilla-central/rev/7744b9769935

Comment 8

[Ed Morley [:emorley]]

(Also note that part 1 missed the merge of mozilla20 to aurora; so needs to land there too)

Comment 9

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Re-pushed: https://tbpl.mozilla.org/?rev=e7cdf5e3ee87
https://hg.mozilla.org/mozilla-central/rev/1780bd0c3dee
https://hg.mozilla.org/mozilla-central/rev/e7cdf5e3ee87

Pushed to aurora:
https://hg.mozilla.org/releases/mozilla-aurora/rev/489f8fb0b4fa

Comment 10

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

mozharness:
https://hg.mozilla.org/build/mozharness/rev/1feb32c9c1b5

Comment 11

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

gecko patch part 2:
https://hg.mozilla.org/integration/mozilla-inbound/rev/999039502afb

I probably won't bother landing on aurora/b2g18 as it isn't something that's critical.

Comment 12

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Crap. I just realized that the install_gecko step is happening before the install_busybox step. This means a third gecko patch is needed.

Comment 13

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

Created attachment 699347 [details] [diff] [review]
Patch 1.1 - install_busybox step needs to happen before install_gecko step

Comment 14

[Andrew Halberstadt [:ahal] (PTO until May 1st)]

gecko patch part 3:
https://hg.mozilla.org/integration/mozilla-inbound/rev/564336aa34f6

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/999039502afb
https://hg.mozilla.org/mozilla-central/rev/564336aa34f6

Comment 16

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/rev/d6d3e797d1ae