2013743 - Support missing android test types with './mach test'

Status: NEW

(Testing :: General, task)

Description

[Ted Campbell [:tcampbell]]

Existing ./mach test <path-to-test> support is very limited for the Android test suites.

• *.java files are unsupported
• GeckoView unit-tests are unsupported
• Tests not in a 'test' packages are unsupported
• Fenix/Focus/AC androidTest tests are unsupported

Firstly, the ./mach android-test command needs to be enriched, then the ./mach test command can use it in more cases.

Comment 1

[Ted Campbell [:tcampbell]]

Attachment: WIP: Bug 2013743 - Part 0.1: Remove ANDROID_INSTRUMENTATION_MANIFESTS support

This was old infrastructure from the Fennec RoboCop test
suites that are long gone.

Comment 2

[Ted Campbell [:tcampbell]]

Attachment: WIP: Bug 2013743 - Part 0.2: Remove RobocopTarget Annotations from GeckoView

These are no longer connected to anything and the robocop tests
are long gone.

Comment 3

[Ted Campbell [:tcampbell]]

Attachment: WIP: Bug 2013743 - Part 1.1: Add tests for './mach android-test'

Add some tests for the android-test command. This also fixes handling of .java
file extensions. Includes some minor refactoring to prepare for subsequent
changes.

Comment 4

[Ted Campbell [:tcampbell]]

Attachment: WIP: Bug 2013743 - Part 1.2: Add geckoview support to './mach android-test'

Add support for geckoview (unit) test suite to 'android-test' command. Note
that the 'geckoview-junit' command still exists and does similar things.

Cleanup how subprojects are handled and derive it automatically if a test path
is specified. Cleanup the gradle commands to separate gradle options from task
options and avoid duplicate or redundant tasks.

Comment 5

[Ted Campbell [:tcampbell]]

Attachment: WIP: Bug 2013743 - Part 1.3: Add --gradle-variant to './mach android-test'

The default gradle-variant we use for tests is 'Debug', but allow overriding
it on command line.

Comment 6

[Ted Campbell [:tcampbell]]

Attachment: WIP: Bug 2013743 - Part 1.4: Add instrumented test support to './mach android-test'

Teach the './mach android-test' command to recognize and to run Android
instrumentation tests using the connectedAndroidTest family of Gradle tasks.

Comment 7

[Ted Campbell [:tcampbell]]

Attachment: Bug 2013743 - Part 3: Cleanup GeckoView support in './mach test'. r?jmaher

• Add recognition of '.java' files.
• Use the 'android-test' command instead of 'geckoview-junit'
• Turn the 'junit' suite into an alias of 'geckoview'
• Support unit and android tests for geckoview. We were running the android
  tests already, so also run the very small collection of unit tests.
• Be explicit that other android suites currently run unit tests only.

Comment 8

[Ted Campbell [:tcampbell]]

Attachment: Bug 2013743 - Part 4: Support Fenix/Focus/AC androidTests with './mach test'. r?jmaher

Updates the resolver logic to check the androidTest path as well. Instead of
requiring that tests use a test package name, just require the name ends in
Test.{kt,java}. Note that the existing presets (like 'fenix') still only run
local unit tests. If a path is given instead, a mix of tests can be run.

Comment 9

[Phabricator Automation]

Comment on attachment 9541589 [details]
WIP: Bug 2013743 - Part 0.1: Remove ANDROID_INSTRUMENTATION_MANIFESTS support

Revision D281345 was moved to bug 2013802. Setting attachment 9541589 [details] to obsolete.

Comment 10

[Phabricator Automation]

Comment on attachment 9541590 [details]
WIP: Bug 2013743 - Part 0.2: Remove RobocopTarget Annotations from GeckoView

Revision D281346 was moved to bug 2013802. Setting attachment 9541590 [details] to obsolete.

Comment 11

[Phabricator Automation]

Comment on attachment 9541591 [details]
WIP: Bug 2013743 - Part 1.1: Add tests for './mach android-test'

Revision D281347 was moved to bug 2013803. Setting attachment 9541591 [details] to obsolete.

Comment 12

[Phabricator Automation]

Comment on attachment 9541592 [details]
WIP: Bug 2013743 - Part 1.2: Add geckoview support to './mach android-test'

Revision D281348 was moved to bug 2013803. Setting attachment 9541592 [details] to obsolete.

Comment 13

[Phabricator Automation]

Comment on attachment 9541593 [details]
WIP: Bug 2013743 - Part 1.3: Add --gradle-variant to './mach android-test'

Revision D281349 was moved to bug 2013803. Setting attachment 9541593 [details] to obsolete.

Comment 14

[Phabricator Automation]

Comment on attachment 9541594 [details]
WIP: Bug 2013743 - Part 1.4: Add instrumented test support to './mach android-test'

Revision D281350 was moved to bug 2013803. Setting attachment 9541594 [details] to obsolete.

Comment 15

[Ted Campbell [:tcampbell]]

Attachment: Bug 2013743 - Part 1: Be consistent about android test naming. r?ohall

Test suites use names ending with 'Test.kt', so rename some of the inconsistent
names to make it easier to distinguish tests from support code.

Comment 16

[Ted Campbell [:tcampbell]]

Attachment: Bug 2013743 - Part 2: Add some test for android './mach test' behaviour. r?jmaher

Add some tests for the cases that currently work so they don't regress will
supporting new cases. Fix some minor defects in existing code while we are at it.