1097125 - make |mach reftest| support testing single files

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[Nathan Froyd [:froydnj]]

It bugs me that mach doesn't support this straightforward thing.  Let's fix it.

Comment 1

[Nathan Froyd [:froydnj]]

Attachment: part 0 - fix cut-and-pasted comment

Just something I noticed while I was in the vicinity.

Comment 2

[Nathan Froyd [:froydnj]]

Attachment: part 1 - add filtering support to |mach reftest-{remote,b2g-desktop}|

We're going to (ab)use --filter to support single-file tests, and therefore we
need --filter support for remote-ish reftests, too.

One wonders why the b2g reftest stuff uses a completely separate
@CommandProvider for this stuff, but maybe that can be fixed up another day.

Comment 3

[Nathan Froyd [:froydnj]]

Attachment: part 2 - make |mach reftest| and related commands support single files

It turns out that the easy workaround (prior to this bug) is to do:

  mach reftest [options] --filter=$(basename $TEST) $(dirname $TEST)

but we can just make mach do that, and make life easier for people.

As written, the patch silently permits you to test non-existent files; I guess
we could add a check to guard against that if you like.

Comment 4

[(not currently active) Ted Mielczarek]

Comment on attachment 8520732 [details] [diff] [review]
part 2 - make |mach reftest| and related commands support single files

Review of attachment 8520732 [details] [diff] [review]:
-----------------------------------------------------------------

::: layout/tools/reftest/mach_commands.py
@@ +94,3 @@
>          return files[suite]
>  
>      def _find_manifest(self, suite, test_file):

You might want to add a short docstring here documenting what this function returns now, since it's no longer obvious.

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/140c9b452bf5
https://hg.mozilla.org/mozilla-central/rev/f941c8c85ae0
https://hg.mozilla.org/mozilla-central/rev/3d75b3fcfb90