1049085 - Robocop SUITE-START message wrong when TEST_PATH is set

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Geoff Brown [:gbrown]]

This is not a big deal, but a little disconcerting, when running a single robocop test locally:

gbrown@mozpad:~/objdirs/droid$ export TEST_PATH=testNewTab
gbrown@mozpad:~/objdirs/droid$ make mochitest-robocop
0 INFO Device info: {}
1 INFO Test root: /sdcard/tests
2 INFO Android sdk version '12'; will use this to filter manifests
3 INFO SUITE-START | Running 61 tests
pk12util: PKCS12 IMPORT SUCCESSFUL
...

Comment 1

[Geoff Brown [:gbrown]]

This is a little odd on tbpl too, presumably because of chunking?

https://tbpl.mozilla.org/php/getParsedLog.php?id=45256473&tree=Mozilla-Central&full=1

09:43:30     INFO -  3 INFO SUITE-START | Running 61 tests
09:43:30     INFO -  4 INFO Running tests 1-16/61

https://tbpl.mozilla.org/php/getParsedLog.php?id=45257118&tree=Mozilla-Central&full=1

10:00:10     INFO -  3 INFO SUITE-START | Running 61 tests
10:00:10     INFO -  4 INFO Running tests 17-32/61

Comment 2

[Ahmed Kachkach [:akachkach]]

This seems to be because of some ulterior filtering done to the robocop tests:

http://dxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#683

I'll try a quick fix for this.

Comment 3

[Ahmed Kachkach [:akachkach]]

Attachment: 0001-Bug-1049085-Correct-list-of-tests-for-the-robocop-su.patch

This should get it done.

Try run: https://tbpl.mozilla.org/?tree=Try&rev=20fa002d71fb

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8468004 [details] [diff] [review]
0001-Bug-1049085-Correct-list-of-tests-for-the-robocop-su.patch

Review of attachment 8468004 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mochitest/runtestsremote.py
@@ +675,5 @@
>              dm._checkCmd(["install", "-r", options.robocopApk])
>  
> +        active_tests = [t['name'] for t in robocop_tests if not (options.testPath and options.testPath != t['name']) and
> +                                                            t['name'] in my_tests and
> +                                                            'disabled' in t]

this really should be formatted easier.  While it is understandable we have 4 conditions we are checking for and we should either simplify that list or make this a for loop.

Comment 5

[Ahmed Kachkach [:akachkach]]

Attachment: 0001-Bug-1049085-Correct-list-of-tests-for-the-robocop-su.patch

Updated patch.

Try run here: https://tbpl.mozilla.org/?tree=Try&rev=37145f5b7895
(could you confirm the numbers shown make sense?)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8468732 [details] [diff] [review]
0001-Bug-1049085-Correct-list-of-tests-for-the-robocop-su.patch

Review of attachment 8468732 [details] [diff] [review]:
-----------------------------------------------------------------

much better.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/fx-team/rev/3e86e8d97314

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/3e86e8d97314

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/6b801f094ae6