Bug 1320841 (stylo-mochitest)

[meta] Pass all mochitests for Stylo

REOPENED
Unassigned

Status

()

defect
P5
normal
REOPENED
3 years ago
2 years ago

People

(Reporter: manishearth, Unassigned)

Tracking

(Depends on 1 bug, {meta})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 2 obsolete attachments)

No description provided.
Comment hidden (mozreview-request)

Comment 2

3 years ago
mozreview-review
Comment on attachment 8815129 [details]
Bug 1320841 - Re-enable mochitests for Stylo builds;

https://reviewboard.mozilla.org/r/96126/#review96706

I would prefer we create a subset of mochitests to run for stylo. Probably create a new test task called mochitest-stylo and gradually move tests we care about into the list until the implementation is good enough to be run on all.

Unlike reftest which is mainly only used to test layout system, mochitest is used to test all kinds of things, so it could be too noisy to enable for now.

I can try to create that new test task.
Attachment #8815129 - Flags: review?(xidorn+moz)
Priority: -- → P2
Priority: P2 → P5
We already run mochitests.
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
Nope. We only run a very limited set of mochitests at the moment (the style system mochitests). We should probably consider enabling more, but I'm still not sure whether enabling wider range of mochitests are worth the work at this stage. By "the work", I mean triaging the reason of the failures.

We probably should still be focusing on reducing failures in style system mochitests.
Assignee: manishearth → nobody
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I plan to enable chrome and browser style system mochitests soonish, as well as enabling them with e10s.
(In reply to Xidorn Quan [:xidorn] (UTC+10) from comment #5)
> I plan to enable chrome and browser style system mochitests soonish, as well
> as enabling them with e10s.

Can you file a bug on this? This has high uncertainty-reduction value, so should probably be P1.
(In reply to Bobby Holley (:bholley) (busy with Stylo) from comment #6)
> Can you file a bug on this? This has high uncertainty-reduction value, so
> should probably be P1.

Bug 1346047 and bug 1346048.
April 24 run of ALL mochitests:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=125e0d61d3353476bdde95cc433707c03e8f4349

Here are the summary:

FAIL	 681
PASS	 2
TIMEOUT	 10
CRASH	 17
(In reply to Shing Lyu [:shinglyu] from comment #9)
> Created attachment 8860842 [details]
> List of tests with unexpected behavior

Nice! Looks like the lion's share of the failures (asides from the ones in layout/style) are in devtools and animation code, both of which we have bugs on. We should be able to make progress on these in the coming weeks.
May 3 test results:

Devtools 342
Misc layout 145
Misc 51
DOM 48
Animation 32
Crashes 16
Timeout 10
Grid 6
Forms 5
Canvas 3
Error 1

There are quite a few in the layout folder, but seems to came from many different root causes.
May 3 all mochitest results
Attachment #8860842 - Attachment is obsolete: true
Attachment #8860843 - Attachment is obsolete: true
Attachment #8865289 - Attachment is patch: true
Attachment #8865289 - Attachment mime type: text/markdown → text/plain
Can we get updated test results here? Thx!
Flags: needinfo?(shing.lyu)
Depends on: 1374872
Depends on: 1370779
Depends on: 1374881
The result from comment 16 shows some failures in dom/base/test/test_intersectionobservers.html but I cannot reproduce them in my local build. Maybe they have been fixed somehow, or they are failing on Linux specifically.
Same for
* dom/base/test/test_mutationobserver_anonymous.html
* dom/base/test/test_reentrant_flush.html
* dom/base/test/test_script_loader_js_cache.html 
* dom/base/test/test_timer_flood.html
Depends on: 1374901
dom/security/test/csp/test_inlinestyle.html also works as expected locally.
Depends on: 1374996
Depends on: 1374999
Depends on: 1375332
Depends on: 1375337
Depends on: 1375338
Depends on: 1375374
The following tests seem to work fine locally:
* editor/libeditor/tests/test_dragdrop.html
* layout/base/tests/test_bug416896.html

And for layout/base/tests/test_reftests_with_caret.html, only bug966992-{1,2,3}.html are failing because of bug 1375374. Other subtests work just fine.
Depends on: 1368381
Depends on: 1375383
Depends on: 1375390
Depends on: 1371395
Depends on: 1381851
Depends on: 1383980
Depends on: 1383985
Depends on: 1383988
Turning this into a meta bug. I'll open another bug for retiring stylo-failures.md and enabling all mochitests.
Keywords: meta
Summary: Re-enable mochitests for Stylo builds → [meta] Pass all mochitests for Stylo
Depends on: 1383992
Alias: stylo-mochitest
Depends on: 1384001
Depends on: 1384002
Depends on: 1384004
Depends on: 1384005
Depends on: 1384009
Depends on: 1384011
Depends on: 1384012
Depends on: 1383998
Depends on: 1385193
Depends on: 1387905
No longer depends on: 1387905
Depends on: 1387908
Depends on: 1387910
Depends on: 1387913
Depends on: 1387932
Depends on: 1387933
Depends on: 1387934
Depends on: 1387935
Depends on: 1387080
Depends on: 1387942
Depends on: 1387993
Depends on: 1387995
Depends on: 1388209
Depends on: 1390046
Depends on: 1390352
Depends on: 1390357
Depends on: 1390364
Depends on: 1390367
Depends on: 1390384
Depends on: 1390409
Depends on: 1390411
Depends on: 1383845
Depends on: 1390694
Depends on: 1390726
No longer depends on: 1383845
Depends on: 1391141
Depends on: 1393819
Depends on: 1394317
Depends on: 1394887
Depends on: 1396483
Depends on: 1417207
You need to log in before you can comment on or make changes to this bug.