stylo: stylo-to-stylo reftest

RESOLVED FIXED

Status

()

P1
normal
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: shinglyu, Assigned: shinglyu)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments)

(Assignee)

Description

a year ago
We've been running stylo-vs-gecko reftest on linux64-stylo for a while. Now it's time to run stylo-vs-stylo reftest in parallel.

This involves:
1. Add a new test job to the CI manifest
2. Create a separate flag to distinguish stylo-vs-gecko and stylo-vs-stylo
  * I enabled this in Bug 1361645, but got invalidated by Bug 1364863, so I got to rebase 1361645 and land
3. Mark the reftest.list expectations.
Duplicate of this bug: 1367622
In terms of reftest manifest flag names, I think we should replace the current fails-if(stylo) with fails-if(styloVsGecko), and use fails-if(stylo) for stylo-vs-stylo mode.
(Assignee)

Comment 3

a year ago
(In reply to Bobby Holley (:bholley) (busy with Stylo) from comment #2)
> In terms of reftest manifest flag names, I think we should replace the
> current fails-if(stylo) with fails-if(styloVsGecko), and use fails-if(stylo)
> for stylo-vs-stylo mode.

That's the plan for Bug 1361645, I'll land it ASAP.
(Assignee)

Comment 4

a year ago
I tried to enable stylo-to-stylo reftest, doesn't look too bad. https://treeherder.mozilla.org/#/jobs?repo=try&revision=64090709c1c6eede128b1b010c86b56271126abc
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 8

a year ago
Chris, 

Here are the patch for enabling stylo-vs-stylo reftest. I'll be on PTO next week so I'm not very confident to land it right now. If this is very urgent please find someone who know how to update reftest expectation to help land it, because we might need to update some reftest expectations while rebasing.
Flags: needinfo?(cpeterson)
(In reply to Shing Lyu [:shinglyu] from comment #8)
> Here are the patch for enabling stylo-vs-stylo reftest. I'll be on PTO next
> week so I'm not very confident to land it right now. If this is very urgent
> please find someone who know how to update reftest expectation to help land
> it, because we might need to update some reftest expectations while rebasing.

Awesome! Thanks. I'll find someone to land your patches. Is your reftest expectations patch all green (for the moment)? Or will the person who lands this next week need to do fix more reftest expectations?
Assignee: nobody → shing.lyu
Flags: needinfo?(cpeterson) → needinfo?(shing.lyu)
Attachment #8871645 - Flags: review?(cam)
Attachment #8871646 - Flags: review?(cam)
Attachment #8871647 - Flags: review?(cam)

Comment 10

a year ago
Pushed by cmccormack@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/90a28b364ba8
Make reftest "stylo" condition match only non-styloVsGecko mode. r=heycam
https://hg.mozilla.org/integration/autoland/rev/6044cf528f8c
Enable stylo-vs-stylo reftest on CI. r=heycam
https://hg.mozilla.org/integration/autoland/rev/8db66ad9798b
Update stylo-vs-stylo reftest expectations. r=heycam
Attachment #8871645 - Flags: review?(cam)
Attachment #8871646 - Flags: review?(cam)
Attachment #8871647 - Flags: review?(cam)
I pushed this directly to autoland, after updating the test expectations, since I had to base the patches on autoland, but I couldn't push to mozreview on autoland due to some issue with being based on a merge commit.

One notable reftest.list change is that I disabled layout/reftests/text-overflow/block-padding.html, because it was passing in sequential mode but failing in parallel mode, and there's no sandbox flag to condition that failure on.
Flags: needinfo?(shing.lyu)

Comment 12

a year ago
Pushed by cmccormack@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9c5f45aaf4c8
Some more rem-unit test disabling. r=me
Thanks, Cameron! I see the stylo-vs-stylo (R#) and stylo-vs-gecko (Rs#) reftests are now running on Treeherder.
Status: NEW → RESOLVED
Last Resolved: a year ago
Priority: -- → P1
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.