Separate out web extension tests into its own suite

NEW
Unassigned

Status

()

Toolkit
WebExtensions: General
P5
normal
10 months ago
10 months ago

People

(Reporter: zombie, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: triaged)

(Reporter)

Description

10 months ago
+++ This bug was initially created as a clone of Bug #1336757 +++

Can we get our own test designation (is this "suite"?) for all WE tests in one place?  

As things currently stand, mistakes like bug 1336757 are easy to miss and likely to happen again -- it was pure coincidence that I noticed this so quickly, this could have gone unnoticed for weeks.

Also, when hunting for intermittent failures, searching through results of WE tests in mochitest logs 1 through 10 (which also varies between platforms) is an awful dev experience.

Comment 1

10 months ago
Would you know how to do this?
Flags: needinfo?(ahalberstadt)
Priority: -- → P5
Whiteboard: triaged
Yep, you should be able to do it self-serve even:
1) Copy+paste then tweak a config like this:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/test/tests.yml#664

2) Add it to the mozharness configs:
https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/unittests/linux_unittest.py#186

3) Whatever you chose as your string in --subsuite=<value>, just add subsuite=<value> to the [DEFAULT] section of whatever manifests should be run. That's it, test it out by pushing to try.

This will only work with taskcluster tasks (aka linux only for now). Mac and windows support should be coming later this year. Also, there is an effort underway in treeherder to completely re-think how we approach displaying results. As part of this, having to deal with chunks should be a thing of the past.. though that will be a bit longer term.
Flags: needinfo?(ahalberstadt)
You need to log in before you can comment on or make changes to this bug.