1120727 - Add mach commands for mochitest-plain and mochitest-chrome on Android

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, defect)

Description

[Geoff Brown [:gbrown]]

On Android, we have a mach command to run robocop tests, but none for plain mochitests. Plain mochitests can be run locally by calling runtestsremote.py directly, or with the mochitest-remote make target.

With patches landed in bug 1026290, it is now possible to run mochitest-chrome tests on Android via runtestsremote.py, but there is no make target and no mach command.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: wip - still need to account for bug 1046992

Comment 2

[Andrew Halberstadt [:ahal]]

Just to note, I plan on completely overhauling the mochitest mach_commands.py very soon. I'm going to move all of the logic out of mach_commands.py and into the test harness. So depending how close you are to landing this, either I should wait for you to land, or you should wait for me (though I haven't started writing the patch yet, so you're likely closer).

Comment 3

[Geoff Brown [:gbrown]]

(In reply to Andrew Halberstadt [:ahal] from comment #2)
> I'm going to move all of the logic out of
> mach_commands.py and into the test harness. 

I like that idea.

Comment 4

[Geoff Brown [:gbrown]]

Attachment: add mochitest-plain and mochitest-chrome support for Android

This adds rudimentary support for "mochitest-plain" and "mochitest-chrome" on Android.

Many of the advanced options supported on desktop (chunking, tags, etc) are ignored on Android; we can always follow-up.

As for B2G, if more than one test path is requested, only the first is used and a warning is printed.

I have not added any Android-specific options. Prime candidates would be for selecting device manager, device IP, etc. Newer Android tests on treeherder are using adb and sutagent runs into trouble with newer androids because of selinux security, so I am starting to think that we should retire sutagent, or at least de-emphasize it.

I have tested:
 - mochitest-plain and mochitest-chrome continue to work on desktop
 - on Android:
     ./mach mochitest-plain
     ./mach mochitest-plain <test-dir>
     ./mach mochitest-plain <test-file>
     ./mach mochitest-chrome
     ./mach mochitest-chrome <test-dir>
     ./mach mochitest-chrome <test-file>
     ./mach robocop
     ./mach robocop <test-name>
 - on Android, all of the above complain if MOZ_HOST_BIN is not defined, or defined incorrectly.

Comment 5

[Andrew Halberstadt [:ahal]]

Comment on attachment 8590380 [details] [diff] [review]
add mochitest-plain and mochitest-chrome support for Android

Review of attachment 8590380 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mochitest/mach_commands.py
@@ +1044,5 @@
>  
>          mochitest = self._spawn(MochitestRunner)
>          return mochitest.run_b2g_test(test_paths=test_paths, **kwargs)
>  
> +    def run_mochitest_android(self, test_paths, chrome=False, **kwargs):

Because this is using _st_parser, people will see the full set of options when running --help. These show up in kwargs, but will be ignored which could cause confusion. It would be good to do something like [1] so they get forwarded on to the harness. But if you want to land this as is for now and do the rest in a follow up, that's fine with me.

[1] https://dxr.mozilla.org/mozilla-central/source/testing/mochitest/mach_commands.py#185

Comment 6

[Geoff Brown [:gbrown]]

No changes to reviewed patch; filed 1152944 for improvements.

https://hg.mozilla.org/integration/mozilla-inbound/rev/5cfb3f15fdc9

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/5cfb3f15fdc9