1303747 - Update stylo reftest expectations

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect)

Description

[Manish Goregaokar [:manishearth]]

Running bug to keep the expectations up to date.

Comment 1

[Manish Goregaokar [:manishearth]]

Upcoming patch being run at https://treeherder.mozilla.org/#/jobs?repo=try&revision=0a0a2bf7d3d4

(soon pushing this to stylo-incubator)

Comment 2

[Manish Goregaokar [:manishearth]]

Attachment: Bug 1303747 - Update stylo reftest list;

Review commit: https://reviewboard.mozilla.org/r/79532/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/79532/

Comment 3

[Cameron McCormack (:heycam)]

Comment on attachment 8792516 [details]
Bug 1303747 - Update stylo reftest list;

https://reviewboard.mozilla.org/r/79532/#review78262

r=me without the .cargo/config.in change.

::: .cargo/config.in:6
(Diff revision 1)
>  [source.crates-io]
>  registry = 'https://github.com/rust-lang/crates.io-index'
>  replace-with = 'vendored-sources'
>  
>  [source.vendored-sources]
> -directory = '@top_srcdir@/third_party/rust'
> +directory = '@top_srcdir@/third-party/rust'

Why this change?  Seems like it should be in a separate bug if anything.

::: layout/reftests/bugs/reftest-stylo.list:1076
(Diff revision 1)
>  fails skip == 403656-4.html 403656-4.html
>  == 403656-5.html 403656-5.html
>  #== 403657-1.html 403657-1.html
>  # Fails depending on the fonts...
>  == 403733-1.html 403733-1.html
> -== 403962-1.xhtml 403962-1.xhtml
> +skip == 403962-1.xhtml 403962-1.xhtml

Just to confirm, we're only skipping tests that crash, right?  Those that permanently fail should be marked "fails" and intermittent ones as "random".

Comment 4

[Cameron McCormack (:heycam)]

(In reply to Manish Goregaokar [:manishearth] from comment #0)
> Running bug to keep the expectations up to date.

FWIW I think we prefer to use separate bugs for things that we know will land at different times, so if this bug is for more than just the immediate couple of patches left to get us to green now, then maybe we can make this a meta bug (if it's worth keeping) or just file new bugs as we encounter new expectation updates we need (if it's not worth keeping).  (For one thing, MozReview isn't really designed for multiple landings and patch submission cycles.  But in general, keeping to a single back-out-able set of changes per bug makes tracking easier for sheriffs, release managers, etc.)

Comment 5

[Manish Goregaokar [:manishearth]]

Using this as a meta bug, filed bug 1303920.

Undid the config change and addressed the skip thing in IRC (some intermittents get marked as skip because `random` doesn't cover load failures)

Comment 6

[Manish Goregaokar [:manishearth]]

We now have perennially-green reftests. Mostly.