1170332 - |mach robocop SINGLE_TEST| broken after Bug 1165497

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Nick Alexander :nalexander [he/him]]

Fallout from Bug 1164597: unfortunately, |mach robocop SINGLE_TEST| is broken.  There are few ways to address this, including:
* rolling back parts of that ticket;
* making Robocop a proper Mochitest flavour.

Comment 1

[Nick Alexander :nalexander [he/him]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8f615247e10e

Comment 2

[Nick Alexander :nalexander [he/him]]

Attachment: MozReview Request: Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

This patch declares robocop.ini an instrumentation manifest.  It's not
currently possible to declare tests that don't correspond to files, so
we include the .java extension.  (This could be revisited.)

In |mach robocop|, we use the generic test resolving infrastructure to
select the 'instrumentation'/'robocop' flavor/subsuite tests.  In
|runtestsremote.py|, we fall back to robocop.ini, as we always have.

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8613723 [details]
MozReview Request: Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

https://reviewboard.mozilla.org/r/9781/#review8595

The TestResolver changes look good to me! Ship it assuming the INSTRUMENTATION_MANIFEST change is ok with gbrown.

Comment 4

[Nick Alexander :nalexander [he/him]]

Comment on attachment 8613723 [details]
MozReview Request: Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

This patch declares robocop.ini an instrumentation manifest.  It's not
currently possible to declare tests that don't correspond to files, so
we include the .java extension.  (This could be revisited.)

In |mach robocop|, we use the generic test resolving infrastructure to
select the 'instrumentation'/'robocop' flavor/subsuite tests.  In
|runtestsremote.py|, we fall back to robocop.ini, as we always have.

Comment 5

[Nick Alexander :nalexander [he/him]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b84246ab327c

Comment 6

[Andrew Halberstadt [:ahal]]

Comment on attachment 8613723 [details]
MozReview Request: Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

https://reviewboard.mozilla.org/r/9781/#review8641

Ship It!

Comment 7

[Geoff Brown [:gbrown]]

Comment on attachment 8613723 [details]
MozReview Request: Bug 1170332 - Fix |mach robocop SINGLE_TEST|. r?ahal,gbrown

https://reviewboard.mozilla.org/r/9781/#review8661

::: testing/mochitest/mach_commands.py:595
(Diff revision 2)
> +        driver.install_tests(remove=False)

Is this necessary? I find the extra messaging and pause disconcerting (but it's not a big issue).

Comment 8

[Nick Alexander :nalexander [he/him]]

https://reviewboard.mozilla.org/r/9781/#review8697

> Is this necessary? I find the extra messaging and pause disconcerting (but it's not a big issue).

It's not really necessary, but it does make it easier to add a new test.  It's a pity it does so much work (30000 files) for Robocop alone.

Comment 9

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/integration/fx-team/rev/0d93a77bb153

Comment 10

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/0d93a77bb153