Closed Bug 1343999 Opened 3 years ago Closed 3 years ago

Disable twitch tests on TaskCluster for debug builds

Categories

(Testing Graveyard :: external-media-tests, enhancement)

enhancement
Not set

Tracking

(firefox54 fixed, firefox55 fixed)

RESOLVED FIXED
mozilla55
Tracking Status
firefox54 --- fixed
firefox55 --- fixed

People

(Reporter: bryce, Assigned: bryce)

Details

Attachments

(1 file)

Playback performance on debug builds results in stalls when running the twitch external media tests. Because of this we're seeing intermittent failures. I suggest we disable these tests for debug builds, to remove the noise of intermittent failures.

An alternative would be to try and mitigate this in the tests. However, based on discussions I've had within the media term, disabling debug runs is preferred as we want to time box this.
Assignee: nobody → bvandyk
Comment on attachment 8844166 [details]
Bug 1343999 - Disable twitch external media tests for debug builds in taskcluster.

https://reviewboard.mozilla.org/r/117698/#review119410

Aside from the name, the form of this patch is good!

::: taskcluster/ci/test/test-platforms.yml:29
(Diff revision 1)
>      build-platform: linux/opt
>      test-sets:
>          -  linux32-tests
>          -  linux32-opt-tests
>          - external-media-tests
> +        - external-media-tests-opt

Let's name this something other than "..-opt"; perhaps external-media-tests-slow?  I would like to try to name the test-sets for what they contain, not where they run; otherwise this file will devolve into

linux32/opt:
  test-sets:
    - linux32-opt-tests

at which point, it doesn't really tell you anything (and the same test suites are listed multiple times in test-sets.yml)
Attachment #8844166 - Flags: review?(dustin) → review-
Comment on attachment 8844166 [details]
Bug 1343999 - Disable twitch external media tests for debug builds in taskcluster.

https://reviewboard.mozilla.org/r/117698/#review119414
Attachment #8844166 - Flags: review?(dustin) → review+
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1465a237d393
Disable twitch external media tests for debug builds in taskcluster. r=dustin
https://hg.mozilla.org/mozilla-central/rev/1465a237d393
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.