Closed Bug 1365621 Opened 7 years ago Closed 7 years ago

|mach test-info| could be more flexible

Categories

(Testing :: General, enhancement)

enhancement
Not set
normal

Tracking

(firefox55 fixed)

RESOLVED FIXED
mozilla55
Tracking Status
firefox55 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

Details

Attachments

(3 files)

3 simple ideas:
 - support multiple tests at once
 - display manifest info
 - more command line arguments to filter categories of information
Attachment #8868606 - Flags: review?(jmaher)
For example:

$ ./mach test-info test_app_constants.html test_simple.js --show-info
===== test_app_constants.html =====
Found mobile/android/tests/browser/chrome/test_app_constants.html in source control.
mobile/android/tests/browser/chrome/test_app_constants.html found in manifest mobile/android/tests/browser/chrome/chrome.ini
  flavor: chrome
  skip-if: os != 'android'
===== test_simple.js =====
Found netwerk/test/unit/test_simple.js in source control.
netwerk/test/unit/test_simple.js found in manifest netwerk/test/unit/xpcshell.ini
  flavor: xpcshell
Comment on attachment 8868605 [details] [diff] [review]
support test-info for more than one test at a time

Review of attachment 8868605 [details] [diff] [review]:
-----------------------------------------------------------------

great!
Attachment #8868605 - Flags: review?(jmaher) → review+
Comment on attachment 8868606 [details] [diff] [review]
add test manifest info to test-info

Review of attachment 8868606 [details] [diff] [review]:
-----------------------------------------------------------------

my comment is more a question and maybe a followup bug.

::: testing/mach_commands.py
@@ +1004,5 @@
> +                print("  flavor: %s" % tests[0]['flavor'])
> +            if tests[0].get('skip-if'):
> +                print("  skip-if: %s" % tests[0]['skip-if'])
> +            if tests[0].get('fail-if'):
> +                print("  fail-if: %s" % tests[0]['fail-if'])

how does this work if we fail/skip on certain platforms?  do we iterate through the platforms, or print the condition?  Will this work for mochitest,reftest,web-platform-tests?
Attachment #8868606 - Flags: review?(jmaher) → review+
Comment on attachment 8868609 [details] [diff] [review]
allow customization for categories of test-info

Review of attachment 8868609 [details] [diff] [review]:
-----------------------------------------------------------------

nice!
Attachment #8868609 - Flags: review?(jmaher) → review+
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/af8c7cffc22a
Allow multiple test arguments for |mach test-info|; r=jmaher
https://hg.mozilla.org/integration/mozilla-inbound/rev/347c5cc8e367
Display manifest info in |mach test-info|; r=jmaher
https://hg.mozilla.org/integration/mozilla-inbound/rev/fb4daf2e29dc
Allow specification of info categories for |mach test-info|; r=jmaher
(In reply to Joel Maher ( :jmaher) from comment #6)
> Comment on attachment 8868606 [details] [diff] [review]
> add test manifest info to test-info
> 
> Review of attachment 8868606 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> my comment is more a question and maybe a followup bug.
> 
> ::: testing/mach_commands.py
> @@ +1004,5 @@
> > +                print("  flavor: %s" % tests[0]['flavor'])
> > +            if tests[0].get('skip-if'):
> > +                print("  skip-if: %s" % tests[0]['skip-if'])
> > +            if tests[0].get('fail-if'):
> > +                print("  fail-if: %s" % tests[0]['fail-if'])
> 
> how does this work if we fail/skip on certain platforms?  do we iterate
> through the platforms, or print the condition?  

test['skip-if'] seems to provide the text of the skip-if annotation, as found in the manifest. In that sense it is platform agnostic. However, if a test is skipped on the platform associated with the mach session that is running test-info, then the whole test manifest lookup will fail: test-info will report "<test> not found in any test manifest!". That is inconvenient, so I am looking for a way around that.

> Will this work for
> mochitest,reftest,web-platform-tests?

Yes, I think so. It leverages the TestResolver class, which has a bunch of harness- and flavor-specific logic. It seems to work for everything. :)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: