Closed Bug 1322845 Opened 4 years ago Closed 2 years ago

Get reftests running in macOS QR builds in automation

Categories

(Core :: Graphics: WebRender, defect, P3)

53 Branch
x86_64
macOS
defect

Tracking

()

RESOLVED FIXED
mozilla62
Tracking Status
firefox56 --- unaffected
firefox57 --- unaffected
firefox62 --- fixed

People

(Reporter: kats, Assigned: kats)

References

(Blocks 2 open bugs)

Details

(Whiteboard: [wr-reserve] [gfx-noted])

Attachments

(2 files)

On the graphics branch we have Linux QuantumRender jobs going for reftests, but we don't yet have it for OS X. Unfortunately, it looks like TaskCluster doesn't yet support running reftests on OS X (see [1] for example). It looks like people are actively working on this deep in the dependency tree of bug 1298431, so we'll need to wait for that to be done first.

[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=c48c4abfce0d05e430965e974f060d9d20f7b8ab&selectedJob=32493135
OS: Unspecified → Mac OS X
Hardware: Unspecified → x86_64
Priority: P3 → P2
Whiteboard: [gfx-noted] → [wr-mvp] [gfx-noted]
Priority: P2 → P3
Whiteboard: [wr-mvp] [gfx-noted] → [wr-reserve] [gfx-noted]
No longer blocks: stage-wr-tests
Summary: Add jobs in automation for OS X reftests with QR enabled → Get reftests running in macOS QR builds in automation
We should probably get reftests into a runnable shape on macOS because right now we have zero CI coverage of webrender in Firefox on macOS, which leads to problems like bug 1459102.
Assignee: nobody → bugmail
Here's a recent try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2c268d59f033b9109c80249772a2ecddc8840295

There's less than 100 tests with UNEXPECTED results. It should be pretty easy to triage them and turn on this suite.
The reason there were less than 100 tests with UNEXPECTED results in the opt job was because of "Aborting task - max run time exceeded!". The debug jobs have a lot more failures (total 190) because the tests are split into three chunks and they run to completion. So there's more stuff that needs annotating.
This is the try push which just had the first batch of stuff annotated: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c1b4afa7100936c0806d046f58831799fc6ccbd7

And here is one that should have everything annotated: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7a93c04034f207da9341816b3ae750d2b3febd82

I might still need to split the opt job into multiple chunks.
Comment on attachment 8973818 [details]
Bug 1322845 - Enable reftests on macosx64-qr.

https://reviewboard.mozilla.org/r/242182/#review248040
Attachment #8973818 - Flags: review?(a.beingessner) → review+
Comment on attachment 8973817 [details]
Bug 1322845 - Update reftest annotations for WebRender on macOS.

https://reviewboard.mozilla.org/r/242180/#review248042
Attachment #8973817 - Flags: review?(a.beingessner) → review+
We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s 436f1dd0ae9f53a77513862ca6761d390a9d3682 -d ad49e0534fd3: rebasing 462482:436f1dd0ae9f "Bug 1322845 - Update reftest annotations for WebRender on macOS. r=Gankro"
merging layout/reftests/svg/reftest.list
merging layout/reftests/w3c-css/received/reftest.list
warning: conflicts while merging layout/reftests/w3c-css/received/reftest.list! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)
I had to rebase on top of various things including bug 1392106 and the WR update, and I had this try push:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=dfa7df65d835d8a1becc4a397796f6238fe6ddf0

which has a couple of annotations that need updating. Here's a try push with those, hopefully this will be green and landable:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=aad3389936f4961cc88d2e2fbba57b87f4813dd5
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/abd41b3e63e7
Update reftest annotations for WebRender on macOS. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/4fd46cd822bf
Enable reftests on macosx64-qr. r=Gankro
Hm, rebase error. Will fix and reland.
Flags: needinfo?(bugmail)
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/854a37787150
Update reftest annotations for WebRender on macOS. r=Gankro
https://hg.mozilla.org/integration/autoland/rev/c665078b57bd
Enable reftests on macosx64-qr. r=Gankro
https://hg.mozilla.org/mozilla-central/rev/854a37787150
https://hg.mozilla.org/mozilla-central/rev/c665078b57bd
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.