Closed Bug 1353218 Opened 8 years ago Closed 7 years ago

stylo: Integrate failure pattern file mechanism into testharness.js framework

Categories

(Testing :: Mochitest, enhancement, P5)

Product:
Testing
Component:
Mochitest
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: xidorn, Unassigned)

References

Details

The failure pattern file mechanism added in bug 1337674 currently only supports SimpleTest framework. We want to expand that to testharness.js framework so that we can extend the range of mochitests we run for stylo.
How much useful test coverage do we stand to gain if we do this? I'd been sorta assuming that, if we fixed all the test failures on the suites we run, we'd be somewhere within reach of enabling stylo globally - so I'm not sure if it's necessarily worth the trouble (and the automation dollars) to expand stylo's coverage to include more than we have. WDYT?
Flags: needinfo?(xidorn+moz)
layout/style/test should be a good coverage for style system, but people working on animation seem to think dom/animation/test is also useful, and we probably should enable. And according to hiro (see bug 1346103 comment 3), there are 64 tests there using testharness.js. That's why I opened this bug. Probably hiro has better understanding about the state of those tests.
Flags: needinfo?(xidorn+moz) → needinfo?(hikezoe)
If those tests are critical to getting animation support working, _and_ we really need the failure annotations for them, we should probably do this. But if we have other tests that exercise most of the same code, or we can enable the tests at a per-file level, then this likely isn't worth it (unless it's really easy).
It's not really easy, otherwise I should have done this in bug 1337674... I believe it took me a while and I eventually decided to give up at that time.
As of today, most tests fail because of transitions (bug 1302946) (another big problem is bug 1326131 I guess). Once transition works on stylo, we can know how many test cases still fails, how many remaining issue there are what we should care about respectively. Anyway, our (animation team) top priority is transition now, after that, we'd like to run all tests in dom/animations/, after the looking the test results we will know how many works there are. So, I don't rush about this at this moment, but this would be really helpful animation works.
Flags: needinfo?(hikezoe)
(In reply to Hiroyuki Ikezoe (:hiro) from comment #5) > As of today, most tests fail because of transitions (bug 1302946) (another > big problem is bug 1326131 I guess). Once transition works on stylo, we can > know how many test cases still fails, how many remaining issue there are > what we should care about respectively. Anyway, our (animation team) top > priority is transition now, after that, we'd like to run all tests in > dom/animations/, after the looking the test results we will know how many > works there are. Ok. Let's wait and see then whether we can enable the tests on a per-file basis, which would mean we wouldn't need the failure pattern file mechanism. P5 until we're sure we need this.
Priority: -- → P5
I guess it is a WONTFIX now. Most of tests disabled for bug 1339656 have been enabled now, and they are not expected to be regressed.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.