1258816 - Autophone - allow trigger to submit all chunks for a test

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

Attachment: trigger.patch

Currently, trigger_runs.py takes a --test argument which specifies the name of a test to be executed in autophone. This works for tests without chunks by simply specifying the full test name, e.g. autophone-s1s2. For tests with chunks, the current code requires that the test name include the chunk as in autophone-reftest-1. Simply specifying autophone-reftest will not match any tests.

This patch adds a check to allow the base name of a chunked test to be specified which will then submit all of the chunks to be tested.

While looking at this, I realized that the chunk argument to PhoneTest.match is not used and is redundant due to the nature of the name/chunking selection so I have also removed it as part of this patch.

I first began thinking of the need for this ability when attempting try runs on the normally inactive tests such as mochitest, reftest, jstest, etc and also due to gbrown's work on specifying tests in the mach autophone command.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8733508 [details] [diff] [review]
trigger.patch

Review of attachment 8733508 [details] [diff] [review]:
-----------------------------------------------------------------

this looks good, the additional clause confuses me, maybe a comment or just reply in the bug.

::: phonetest.py
@@ +213,5 @@
>  
>          for test in tests:
> +            if (test_name and
> +                test_name != test.name and
> +                "%s%s" % (test_name, test.name_suffix) != test.name):

why did we add this extra clause here?

Comment 2

[Bob Clary [:bc] (inactive)]

I tried several comments but they were really long and verbose. They mostly went with my overly long and verbose patch which turned out to be unnecessary. But still trying to word this without making it several paragraphs seems difficult.

When matching on a test_name like autophone-reftest where we want to get all of the chunks, the extra clause will attempt to match the current test 

{test_name}{test.name_suffix} against {test.name}.

If test_name is autophone-reftest and test.name is autophone-reftest-1, that means that test.name_suffix is "-1". So, in this example we would attempt to match autophone-reftest-1 against autophone-reftest-1 and find a match. This works for all of the chunks.

If we had already specified a chunk in the test_name, e.g. autophone-reftest-1, we would hit the test_name != test.name clause and only pick up the test.name == 'autophone-reftest-1'. Similarly for tests without chunks.

So, the extra clause simply allows all of the chunks of a test to be specified via the "root" test name.

Comment 3

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/5cf53e3b4dc9b1be5af160d91dc2edbbe9ad629a

Comment 4

[Bob Clary [:bc] (inactive)]

deployed 2016-03-23 06:30 PDT