make |mach reftest| support testing single files

RESOLVED FIXED in mozilla37


4 years ago
4 years ago


(Reporter: froydnj, Assigned: froydnj)



Firefox Tracking Flags

(Not tracked)



(3 attachments)



4 years ago
It bugs me that mach doesn't support this straightforward thing.  Let's fix it.

Comment 1

4 years ago
Created attachment 8520729 [details] [diff] [review]
part 0 - fix cut-and-pasted comment

Just something I noticed while I was in the vicinity.
Attachment #8520729 - Flags: review?(ted)

Comment 2

4 years ago
Created attachment 8520730 [details] [diff] [review]
part 1 - add filtering support to |mach reftest-{remote,b2g-desktop}|

We're going to (ab)use --filter to support single-file tests, and therefore we
need --filter support for remote-ish reftests, too.

One wonders why the b2g reftest stuff uses a completely separate
@CommandProvider for this stuff, but maybe that can be fixed up another day.
Attachment #8520730 - Flags: review?(ted)

Comment 3

4 years ago
Created attachment 8520732 [details] [diff] [review]
part 2 - make |mach reftest| and related commands support single files

It turns out that the easy workaround (prior to this bug) is to do:

  mach reftest [options] --filter=$(basename $TEST) $(dirname $TEST)

but we can just make mach do that, and make life easier for people.

As written, the patch silently permits you to test non-existent files; I guess
we could add a check to guard against that if you like.
Attachment #8520732 - Flags: review?(ted)
Attachment #8520729 - Flags: review?(ted) → review+
Attachment #8520730 - Flags: review?(ted) → review+
Comment on attachment 8520732 [details] [diff] [review]
part 2 - make |mach reftest| and related commands support single files

Review of attachment 8520732 [details] [diff] [review]:

::: layout/tools/reftest/
@@ +94,3 @@
>          return files[suite]
>      def _find_manifest(self, suite, test_file):

You might want to add a short docstring here documenting what this function returns now, since it's no longer obvious.
Attachment #8520732 - Flags: review?(ted) → review+
You need to log in before you can comment on or make changes to this bug.