1476363 - Add --show-tasks support to 'mach test-info'

Status: RESOLVED FIXED

(Testing :: General, enhancement, P1)

Description

[Geoff Brown [:gbrown]]

'mach test-info' provides access to a variety of test information for a specified test name. 

Let's add support for retrieving the names of test tasks where the specified test runs -- information readily available in ActiveData. This would be useful for answering questions like:
 - what platforms does this test run on?
 - what test chunk does this test run in, on each platform?

Comment 1

[Geoff Brown [:gbrown]]

Attachment: add '--show-tasks' option for 'mach test-info'

Simple addition to 'mach test-info':

$ ./mach test-info --show-tasks test_jsimplemented_subclassing.html
===== test_jsimplemented_subclassing.html =====
Found dom/bindings/test/test_jsimplemented_subclassing.html in source control.
dom/bindings/test/test_jsimplemented_subclassing.html found in manifest dom/bindings/test/mochitest.ini
  flavor: mochitest
Querying ActiveData...
Found records matching 'dom/bindings/test/test_jsimplemented_subclassing.html' in ActiveData.

Test tasks for dom/bindings/test/test_jsimplemented_subclassing.html on mozilla-central,mozilla-inbound,autoland between 2018-07-10 and 2018-07-17
android/debug:                           test-android-em-4.3-arm7-api-16/debug-mochitest-7 in 31 runs
android/opt:                             test-android-em-4.3-arm7-api-16/opt-mochitest-4 in 37 runs
linux32/debug-e10s:                      test-linux32/debug-mochitest-e10s-3 in 38 runs
linux32/debug:                           test-linux32/debug-mochitest-3 in 29 runs
linux32/opt-e10s:                        test-linux32-nightly/opt-mochitest-e10s-1 in 6 runs
                                         test-linux32/opt-mochitest-e10s-1 in 33 runs
linux64/[u'debug', u'ccov']-e10s:        test-linux64-ccov/debug-mochitest-e10s-3 in 5 runs
linux64/[u'debug', u'qr']-e10s:          test-linux64-qr/debug-mochitest-e10s-3 in 41 runs
linux64/[u'opt', u'asan']-e10s:          test-linux64-asan/opt-mochitest-e10s-2 in 24 runs
linux64/[u'opt', u'jsdcov']-e10s:        test-linux64-jsdcov/opt-mochitest-e10s-2 in 4 runs
linux64/[u'opt', u'pgo']-e10s:           test-linux64-pgo/opt-mochitest-e10s-1 in 29 runs
linux64/[u'opt', u'qr']-e10s:            test-linux64-qr/opt-mochitest-e10s-1 in 83 runs
linux64/debug-e10s:                      test-linux64/debug-mochitest-e10s-3 in 34 runs
                                         test-linux64/debug-mochitest-plain-headless-e10s-3 in 26 runs
linux64/opt-e10s:                        test-linux64/opt-mochitest-e10s-1 in 39 runs
                                         test-linux64-nightly/opt-mochitest-e10s-1 in 1 runs
                                         test-linux64/opt-mochitest-plain-headless-e10s-1 in 32 runs
maxosx64/debug-e10s:                     test-macosx64/debug-mochitest-e10s-1 in 34 runs
maxosx64/opt-e10s:                       test-macosx64-nightly/opt-mochitest-e10s-1 in 3 runs
                                         test-macosx64/opt-mochitest-e10s-1 in 32 runs
win32/[u'debug', u'msvc']-e10s:          test-windows7-32-msvc/debug-mochitest-e10s-1 in 5 runs
win32/[u'opt', u'msvc']-e10s:            test-windows7-32-msvc/opt-mochitest-e10s-1 in 8 runs
win32/[u'opt', u'pgo']-e10s:             test-windows7-32-pgo/opt-mochitest-e10s-1 in 33 runs
win32/debug-e10s:                        test-windows7-32/debug-mochitest-e10s-1 in 28 runs
win32/opt-e10s:                          test-windows7-32-nightly/opt-mochitest-e10s-1 in 2 runs
                                         test-windows7-32/opt-mochitest-e10s-1 in 27 runs
win64/[u'debug', u'ccov']-e10s:          test-windows10-64-ccov/debug-mochitest-e10s-2 in 3 runs
win64/[u'debug', u'msvc']-e10s:          test-windows10-64-msvc/debug-mochitest-plain-headless-e10s-1 in 4 runs
                                         test-windows10-64-msvc/debug-mochitest-e10s-1 in 2 runs
win64/[u'opt', u'asan']-e10s:            test-windows10-64-asan/opt-mochitest-e10s-1 in 40 runs
win64/[u'opt', u'msvc']-e10s:            test-windows10-64-msvc/opt-mochitest-e10s-1 in 4 runs
                                         test-windows10-64-msvc/opt-mochitest-plain-headless-e10s-1 in 6 runs
win64/[u'opt', u'pgo']-e10s:             test-windows10-64-pgo/opt-mochitest-e10s-1 in 27 runs
win64/[u'opt', u'qr']-e10s:              test-windows10-64-qr/opt-mochitest-e10s-1 in 110 runs
win64/debug-e10s:                        test-windows10-64/debug-mochitest-plain-headless-e10s-1 in 41 runs
                                         test-windows10-64/debug-mochitest-e10s-1 in 34 runs
win64/opt-e10s:                          test-windows10-64/opt-mochitest-plain-headless-e10s-1 in 31 runs
                                         test-windows10-64/opt-mochitest-e10s-1 in 31 runs
                                         test-windows10-64-nightly/opt-mochitest-e10s-1 in 3 runs

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8992709 [details] [diff] [review]
add '--show-tasks' option for 'mach test-info'

Review of attachment 8992709 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mach_commands.py
@@ +953,5 @@
> +                    tasks += task
> +                    tasks += " in %d runs" % consolidated[key][task]
> +                print("%-40s %s" % (key, tasks))
> +        else:
> +            print("No test tasks found.")

this is good, I would like to see activedata recipes for this sometime

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/58719fdf1b95
Add --show-tasks support to 'mach test-info'; r=jmaher

Comment 4

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/58719fdf1b95