1256343 - autophone should add --subsuite=media to run dom/media mochitests

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

we modified the mochitest manifests to have |suite = media| for all mochitests in dom/media/*

this is solved by adding --subsuite=media to the manifest parser for desktop mochitests.

Comment 1

[Bob Clary [:bc] (inactive)]

I don't understand what you mean by "adding --subsuite=media to the manifest parser for desktop mochitests". My first though would have been to add an optional test_subsuite to the mochitest config files in configs/ and to add it to the arguments passed to mochitest/run*.py if it exists.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I envision a fix that looks like:
https://pastebin.mozilla.org/8863602

how to solve builds on version <48 will not be as trivial, but doable.

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: bug-1256343-v1.patch

test runs:

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=a794e56f887d&filter-tier=1&filter-tier=2&filter-tier=3&exclusion_profile=false&filter-searchStr=autophone

mozilla-aurora, mozilla-beta appeared to run ok but don't display in treeherder:

https://treeherder.allizom.org/#/jobs?repo=mozilla-aurora&revision=5e9097f4bf87&filter-tier=1&filter-tier=2&filter-tier=3&exclusion_profile=false&filter-searchStr=autophone

https://treeherder.allizom.org/#/jobs?repo=mozilla-beta&revision=2a62a6f3237e&filter-tier=1&filter-tier=2&filter-tier=3&exclusion_profile=false&filter-searchStr=autophone

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8730681 [details] [diff] [review]
bug-1256343-v1.patch

Review of attachment 8730681 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with nit for comments.

::: tests/runtestsremote.py
@@ +224,5 @@
> +            try:
> +                manifest_cfg = ConfigParser.RawConfigParser()
> +                manifest_cfg.read("%s/tests/%s" % (
> +                    self.parms['build_dir'], self.parms['test_manifest']))
> +                test_subsuite = manifest_cfg.get('DEFAULT', 'subsuite')

this is a nice way to solve the problem, a comment here would help make it understandable to someone looking at this for the first time.  Specifically that we sometimes have subsuite and if we have it we will use it, otherwise throw an exception.

Comment 5

[Bob Clary [:bc] (inactive)]

I've added the following comment. Ping me quick if you want to change it:

+            # Check if the test manifest defines a subsuite which
+            # should be specified. Although we normally parse these
+            # manifests with manifestparser, that is overkill for our
+            # needs. Using the normal ConfigParser we can determine
+            # the subsuite value if it is defined and add the subsuite
+            # command line argument. Missing section or option errors
+            # simply mean no subsuite is defined.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

looks great, thanks!

Comment 7

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/e15ff66a2d3b63299f22314789c58014211b9ae9

deployed 2016-03-15 06:29

Comment 8

[Bob Clary [:bc] (inactive)]

Attachment: bug-1256343-autophone-media-manifest.patch

Need to add subsuite to testing/mochitest/manifests/autophone-media.ini as well.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8730761 [details] [diff] [review]
bug-1256343-autophone-media-manifest.patch

Review of attachment 8730761 [details] [diff] [review]:
-----------------------------------------------------------------

ah, good catch!

Comment 10

[Bob Clary [:bc] (inactive)]

deployed 2016-03-15 ~22:46

Comment 11

[Bob Clary [:bc] (inactive)]

doh, deployed 2016-03-16 ~22:46

Comment 12

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a0e3d3235849

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a0e3d3235849