908874 - Add mach command to run remote mochitests

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Andrew Halberstadt [:ahal]]

We need to modify the existing mochitest mach_commands.py to be capable of running b2g mochitests.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Patch 1.0 - Add a class to store common conditions in mozbuild

I think conditions that depend on mozbuild in any way should be stored alongside MachCommandBase. Gps, if you are hesitant to take this (or something like this), I'll probably just store the conditions inside the mach_commands.py files for now and file a bug to figure this out later.

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Patch 2.0 - refactor mach_commands.py

Adds ability to run remote mochitests.

Comment 3

[Jonathan Griffin (:jgriffin)]

Comment on attachment 794857 [details] [diff] [review]
Patch 2.0 - refactor mach_commands.py

Review of attachment 794857 [details] [diff] [review]:
-----------------------------------------------------------------

Nice!

::: testing/mochitest/mach_commands.py
@@ +68,5 @@
> +        options.b2gPath = b2g_home
> +        options.consoleLevel = 'INFO'
> +        options.logcat_dir = self.mochitest_dir
> +        options.httpdPath = self.mochitest_dir
> +        options.xrePath = os.path.join(b2g_home, 'gaia', 'xulrunner-sdk', 'bin')

This is going to suffer the same problem as bug 908698; maybe we should add logic to look for any xulrunner-sdk* directory under gaia?   The best way to get this would be to get the value of XULRUNNER_DIRECTORY from gaia's make file, but I think that's more-or-less impossible in this context.

@@ +322,5 @@
> +    func = deviceport(func)
> +
> +    deviceip= CommandArgument('--deviceIP', default=None,
> +        help='ip address of remote device to test')
> +    func = deviceip(func)

deviceIP and devicePort are currently unused (they're designed to be used with SUTAgent), so I think we can omit them here, for the sake of reducing potential confusion.

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 794854 [details] [diff] [review]
Patch 1.0 - Add a class to store common conditions in mozbuild

Review of attachment 794854 [details] [diff] [review]:
-----------------------------------------------------------------

I like where this is going!

Comment 5

[Andrew Halberstadt [:ahal]]

Attachment: Patch 2.1 - fix nits, minor refactor

Going to flag interdiff for review as I ended up doing some light refactoring in addition to the review comments.

In this patch, I defer finding of the xre_path to the bootstrap script which will pass it in via context. I also added a condition to the command so it will only be runnable if an emulator is configured (I'll file a bug to make it work with all devices).

Comment 6

[Jonathan Griffin (:jgriffin)]

Comment on attachment 796179 [details] [diff] [review]
Patch 2.1 - fix nits, minor refactor

Review of attachment 796179 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.  Is the reason we're currently requiring an emulator simply because the device path doesn't work correctly?

Comment 7

[Andrew Halberstadt [:ahal]]

(In reply to Jonathan Griffin (:jgriffin) from comment #6)
> Looks good.  Is the reason we're currently requiring an emulator simply
> because the device path doesn't work correctly?

Yeah, right now --emulator arm will just always get added. The reason I went this route is because I'm pushing really hard to get *something* landed before I leave for PTO. Supporting all devices and the testing that goes with it would make this bigger than a two day job.

Though if you wanted to try and get all devices working before the Oslo work week, the patch should be as simple as passing 'device_name' through to the run_b2g_test method and making sure a proper command line is formulated if device_name != 'emulator'.

Comment 9

[Andrew Halberstadt [:ahal]]

Rolled all three patches into one since they depend on each other:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2cc63b8363a5

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/2cc63b8363a5