1357715 - (stylo-devtools-tests) [meta] Stylo: Pass DevTools tests

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, enhancement, P5)

Description

[J. Ryan Stinnett [:jryans]]

We should start to look at DevTools test failures with Stylo enabled since DevTools uses additional code paths that might not be hit when just styling content normally.

An example try run[1] shows the current state.

[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=752bed7671c8df3c830b37df3115f5aa647c6428

Comment 1

[J. Ryan Stinnett [:jryans]]

Updated run now that a few crashing issues were fixed:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c5f4a1469f213ae25760cebfd5ea5f425b576ec

Comment 2

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f454b5577299a506ea812a53113368ad7ff991cc

Comment 3

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a4b1d029e64f2bc6712a7f156c09f62747cb38b8

Comment 4

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=355298c531857aab509eb9af074ec649a52c13a8

Comment 5

[J. Ryan Stinnett [:jryans]]

Some historical DevTools failure counts (looking at e10s opt only):

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159

So, making progress at least!  Some runs do crash though, so potentially more failures will be revealed once crashes are fixed.

Comment 6

[J. Ryan Stinnett [:jryans]]

Marking leave-open to use this bug to land some DevTools expectation updates for platform features that don't block shipping Stylo.

Comment 7

[J. Ryan Stinnett [:jryans]]

Attachment: Bug 1357715 - Skip DevTools tests on Stylo that use shadow DOM.

Review commit: https://reviewboard.mozilla.org/r/151392/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/151392/

Comment 8

[Cameron McCormack (:heycam)]

Comment on attachment 8880071 [details]
Bug 1357715 - Skip DevTools tests on Stylo that use shadow DOM.

https://reviewboard.mozilla.org/r/151392/#review156538

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a7a5b2fc4f0a
Skip DevTools tests on Stylo that use shadow DOM. r=heycam

Comment 10

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a7a5b2fc4f0a

Comment 11

[Carsten Book [:Tomcat]]

this caused perma failures after this landed on m-c like https://treeherder.mozilla.org/logviewer.html#?job_id=109517923&repo=mozilla-central

Comment 12

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/594cc32b6323
Backed out changeset a7a5b2fc4f0a for causing perma failures after merged to m-c

Comment 13

[J. Ryan Stinnett [:jryans]]

(In reply to Carsten Book [:Tomcat] from comment #11)
> this caused perma failures after this landed on m-c like
> https://treeherder.mozilla.org/logviewer.html#?job_id=109517923&repo=mozilla-
> central

Bug 1375699 made the pref optional again, so we need to keep the try blocks.

Comment 14

[J. Ryan Stinnett [:jryans]]

Comment on attachment 8880071 [details]
Bug 1357715 - Skip DevTools tests on Stylo that use shadow DOM.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/151392/diff/1-2/

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e6dfe16f777b
Skip DevTools tests on Stylo that use shadow DOM. r=heycam

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e6dfe16f777b

Comment 17

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4d05347ab605c7ec8da028ce2dd3e56fec44529e

Comment 18

[J. Ryan Stinnett [:jryans]]

New failure count entry:

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159
2017-06-29: 140

Comment 19

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2fdedc25e9243fb83ecd6b6dab0cb54af2559f6c

Comment 20

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0fc2940d6f2d33a0d589de0b0947ec1e944e73e5

Comment 21

[J. Ryan Stinnett [:jryans]]

New failure count entry:

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159
2017-06-29: 140
2017-07-11: 146

Comment 22

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b31de0d8404ab0f925af3d00e831033391c3551

Comment 23

[J. Ryan Stinnett [:jryans]]

New failure count entry:

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159
2017-06-29: 140
2017-07-11: 146
2017-07-12: 180

Comment 24

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=154e6fc2a776a80ced309d36c06465f7ea37e4b1

Comment 25

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=744fe0f4620dc84ff9cb57dae86152af6f8554e3

Comment 26

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=de0f92fdf53e94f72ef41312c8558a4820b62429

Comment 27

[J. Ryan Stinnett [:jryans]]

New failure count entry:

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159
2017-06-29: 140
2017-07-11: 146
2017-07-12: 180
2017-07-24: 171

Comment 28

[J. Ryan Stinnett [:jryans]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e1995483b7fe673b23aa0fa8fc608910dfd8d1e2

Comment 29

[J. Ryan Stinnett [:jryans]]

New failure count entry:

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159
2017-06-29: 140
2017-07-11: 146
2017-07-12: 180
2017-07-24: 171
2017-07-25: 53

Comment 30

[J. Ryan Stinnett [:jryans]]

New failure count entry:

2017-04-19: 376
2017-05-23: 183
2017-06-07: 172
2017-06-16: 159
2017-06-29: 140
2017-07-11: 146
2017-07-12: 180
2017-07-24: 171
2017-07-25: 53
2017-08-01: 46

Comment 31

[J. Ryan Stinnett [:jryans]]

All DevTools tests are active with Stylo!