835517 - Investigate moving B2G tests to Ubuntu 64 VM test pool

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Andrew Halberstadt [:ahal]]

Since B2G tests are overloading the Fedora 32 slave pool we should look into moving mochitests, xpcshell tests and marionette/webapi tests to the new Ubuntu 64 VM pool. Reftests will likely have to remain on physical hardware and cannot be moved over.

Comment 1

[Andrew Halberstadt [:ahal]]

I had to install ia32-libs in order to run the adb/emulator binary, but works out of the box other than that.

Mochitests all seem to pass except for 4 failures in chunk 6 (though need to do a few more runs to make sure).

Comment 2

[Chris AtLee [:catlee]]

fwiw, we are also creating a 32-bit pool of Ubuntu VMs on EC2 if that makes this work any easier.

Comment 3

[Andrew Halberstadt [:ahal]]

The original purpose of moving the tests was to free up load on the linux 32 pool, but if this new ubuntu 32 pool will be enough to handle the extra load and re-imaging the Ubuntu 64 pool with ia32-libs is a pain then that works too. Doesn't really matter to me.

Comment 4

[Andrew Halberstadt [:ahal]]

I ran mochitests in 6 chunks a couple times each. They all passed except one time (out of 3) chunk 6 had 4 failures and took ~68min to finish (the other two times it passed and took ~43min). The shortest chunk was 35min and the longest was 44min.

Comment 5

[Jonathan Griffin (:jgriffin)]

(In reply to Andrew Halberstadt [:ahal] from comment #4)
> I ran mochitests in 6 chunks a couple times each. They all passed except one
> time (out of 3) chunk 6 had 4 failures and took ~68min to finish (the other
> two times it passed and took ~43min). The shortest chunk was 35min and the
> longest was 44min.

Were the failures timeouts?  A 20-minute difference is pretty strange.

Comment 6

[Andrew Halberstadt [:ahal]]

(In reply to Jonathan Griffin (:jgriffin) from comment #5)
> Were the failures timeouts?  A 20-minute difference is pretty strange.
Unfortunately I don't know. I forgot to bump the scrollback of the terminal on the VM and I couldn't see which tests failed (but I would assume it was timeouts)

Comment 7

[Andrew Halberstadt [:ahal]]

I also ran xpcshell tests. They all pass except for toolkit/mozapps/update/test_timermanager/unit/test_0010_timermanager.js. This test also failed on my local machine (also 64bit) but works fine on the Fedora 32 pool. If we do end up running them on the 64bit machines we can just disable it.

The tests take around 9 minutes. This time is without the install gecko step which would likely add 5-10 minutes of setup time (and come to think about it the mochitest times in comment 4 are also not including the install_gecko step).

Comment 8

[Andrew Halberstadt [:ahal]]

The marionette/webapi tests all pass as well and take around ~6-7 minutes (not counting the install_gecko step).

Jgriffin, did you want me to look into anything else? Or can I relinquish the VM back to releng?

Comment 9

[Jonathan Griffin (:jgriffin)]

I think that's enough to prove that we can run them, thanks ahal.