Closed Bug 1436058 Opened 2 years ago Closed 2 years ago

Update webrender to 342bc314db94aa439b2001249c5f24ccfcbccc22

Categories

(Core :: Graphics: WebRender, defect, P3)

60 Branch
defect

Tracking

()

RESOLVED FIXED
mozilla60
Tracking Status
firefox60 --- fixed

People

(Reporter: kats, Assigned: kats)

References

(Blocks 2 open bugs, )

Details

(Whiteboard: [gfx-noted])

Attachments

(5 files)

+++ This bug was initially created as a clone of Bug #1434723 +++

I'm filing this as a placeholder bug for the next webrender update. I may be running a cron script [1] that does try pushes with webrender update attempts, so that we can track build/test breakages introduced by webrender on a rolling basis. This bug will hold the try push links as well as dependencies filed for those breakages, so that we have a better idea going into the update of what needs fixing. I might abort the cron job because once things get too far out of sync it's hard to fully automate fixing all the breakages.

When we are ready to actually land the update, we can rename this bug and use it for the update, and then file a new bug for the next "future update".

[1] https://github.com/staktrace/moz-scripts/blob/master/try-latest-webrender.sh
Blocks: 1435062
Blocks: 1434029
WR @ a492386a97fb73ffc449dc2ee4bee9163238d083

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b518a54e92d91d24fa52740258e77782d8e73605
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c6c64072fee05f10b79607668ae633b52288903f

Now we're busted everywhere. Probably from the env_logger change in a06537e1 (servo/webrender#2385).
No longer blocks: 1434029
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #4)
> Now we're busted everywhere. Probably from the env_logger change in a06537e1
> (servo/webrender#2385).

Ah, it was the log version bump to 0.4. Apparently a lot of stuff in the log crate got renamed. I'm working through it now.
^ Seems to be back to the way things were, i.e. windows bustage and linux debug assertion failures from bug 1432708.
Blocks: 1433839
WR @ 7184d6b6445ba5119acf18cd38c14815f0e8fb16 with a better job of applying the necessary gecko-side patches

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9420c5e883214d72eefa055bb574a3d12e42efcb
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ace60c8b28bdb6a54ed67235ad9e9b5e8fefc441

Windows has gone more red but it might just be that we're failing more reftests than before. I'll look closer. Otherwise things are looking decent.
WR @ 7184d6b6445ba5119acf18cd38c14815f0e8fb16

https://treeherder.mozilla.org/#/jobs?repo=try&revision=bf1f0e821ba910be62e3443687e6e2a7cf288915
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f08626e59d54e2f149ef450fdd1e75bf70bc7403
https://treeherder.mozilla.org/#/jobs?repo=try&revision=94de55fce94e0d6a2f31973397ecaa0eecfdf699
https://treeherder.mozilla.org/#/jobs?repo=try&revision=326b5e9cbc8efa50c62ea087086e3bf42e51ace6
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c9513c4c50d498c748f33de00434dc3b84d8e85a
https://treeherder.mozilla.org/#/jobs?repo=try&revision=147affc252e08cb44f36bf8eebafc5e848c0eb8e
https://treeherder.mozilla.org/#/jobs?repo=try&revision=98d356d779693e0be37d30a1d552f0a27ab7affa
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0136dc220ea9c498760df4090ab5f47f83f1a5d5
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5ad75ddc115a75f102afbba19651409672b6e8c0
https://treeherder.mozilla.org/#/jobs?repo=try&revision=485f8a02c2df32fbc66a13725b0b267a8299df94
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ff0d17d232f7df879cf6929cabdbdb651bae46e1
https://treeherder.mozilla.org/#/jobs?repo=try&revision=bd0c24f524d0a0389294b3b0a57de5b006f2c2a5

Same as before (Linux green, some Windows reftests red)

WR @ d5bf40389e54ee96fc7e313fbe1a590f1464b3a4

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c3ec0fcc24ac51bc3e7f3592f51f8d4500a5cada
https://treeherder.mozilla.org/#/jobs?repo=try&revision=2ecef6debfbd65ffe3fbb0289c14922809cfab3b

Ditto

WR @ eceed87325ef1dbbd220a4ccbb9e7860f2679428

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b5f4754657c4cfc7c1c0e797992a57c06d6a4e3a
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e81014896d5dc6cf83c694f6790d0b957ee9c63e

Two reftests are orange now - one can be fuzzed I think but the other one is all wrong. Probably regression from servo/webrender#2408 since that's the only thing in the range.
So the windows redness seems to be a real issue. Back at d96d02bd9b8141c2fa4a692ee0b35220be0a8873 (comment 2) things were good, in that the windows opt reftests run ran 15720 reftests and 270 of them were unexpected in some way (233 unexpected-fail, 37 unexpected-pass). However as of 7184d6b6445ba5119acf18cd38c14815f0e8fb16 (comment 11) the windows opt reftest run only gets through around 594 reftests (of which 255 are unexpected-fail) before the logfile hits the size limit and the job is aborted. That's what makes it show up as red. So the windows opt run is now failing a *lot* more reftests than it used to, which I consider a regression.

The regression range right now is d96d02bd9b8141c2fa4a692ee0b35220be0a8873..7184d6b6445ba5119acf18cd38c14815f0e8fb16 which is pretty wide but thankfully all the bustage was on the gecko side and so I can do additional try pushes to narrow this down. That's what I'm working on now.


I also kicked off another try push to verify the linux reftest failures came from servo/webrender#2408 and not some random gecko change.
Also for the windows reftest failures, it looks like in a lot of cases the wrong page is getting rendered. This might either be an existing race condition that got aggravated, or could be something like a bug in the texture cache where we're pulling the wrong textures. I downloaded one of those builds and ran it locally and for the most part it seems to work ok. However I did notice at least one problem, on about:newtab the firefox logo on the top-left was getting replaced by a tiled version of the refresh icon from the URL bar. So that might also point to a texture cache problem.
Another problem I noticed with the windows build was that loading layout/reftests/reftest-sanity/647192-1.html shows red for the scrollbars, and it doesn't do that in a current nightly on the same profile or on OS X. This reftest is one of the ones failing in the try push, I picked it at random to see if it would repro locally and it did.
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #13)
> I also kicked off another try push to verify the linux reftest failures came
> from servo/webrender#2408 and not some random gecko change.

This is verified now, left a comment at https://github.com/servo/webrender/pull/2408#issuecomment-364930396
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #13)
> The regression range right now is
> d96d02bd9b8141c2fa4a692ee0b35220be0a8873..
> 7184d6b6445ba5119acf18cd38c14815f0e8fb16 which is pretty wide but thankfully
> all the bustage was on the gecko side and so I can do additional try pushes
> to narrow this down. That's what I'm working on now.

In WR-chronological order:

WR @ 30f2cd5e648bbcadbe8200d479de3f7d76089d0b
https://treeherder.mozilla.org/#/jobs?repo=try&revision=36e1f1cd1c046d07d52b4f129b337788c81f486e
==> good

WR @ 7ae075a3238915530144727896e5d275d4057f34
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3f73d0d02e5fa26bb2e4734c4a85fa8662d03de3
==> good

WR @ b7771429fd8afdea91a0e5d4657bcbaee994909d
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ca5fd387d2ab74a445a9f91ed982f3b9e8dfb7a0
==> good

WR @ a492386a97fb73ffc449dc2ee4bee9163238d083
https://treeherder.mozilla.org/#/jobs?repo=try&revision=daa19eb9727c5011333d391c9555bd4749ab6dd4
==> good

WR @ 342bc314db94aa439b2001249c5f24ccfcbccc22
https://treeherder.mozilla.org/#/jobs?repo=try&revision=bfbd4dad817e82c997ee99aea0d152f938f1ebef
==> good

WR @ 65c68cc1d1d8958c103fa3a528b05d81e27783d3
https://treeherder.mozilla.org/#/jobs?repo=try&revision=7c21bcbd670096d2ef119605b5c51ca6b72f7968
==> bad

WR @ d51d6ec0def765055af6a1e2b2bc8b9bd6e89baf
https://treeherder.mozilla.org/#/jobs?repo=try&revision=2e71f7fa91c2a39d6939f7269bcabf9b9f8b0abb
==> pending

So the regression range is narrowed to 342bc314db94aa439b2001249c5f24ccfcbccc22..65c68cc1d1d8958c103fa3a528b05d81e27783d3:
* 65c68cc1 Auto merge of #2399 - glennw:image-brush, r=kvark
* e0aebb58 Add brush_image shader.

So from servo/webrender#2399 - I'll leave a comment there.
Assignee: nobody → bugmail
No longer blocks: 1405469
Summary: Future webrender update bug → Update webrender to 342bc314db94aa439b2001249c5f24ccfcbccc22
Version: unspecified → 60 Branch
Alias: wr-future-update
Comment on attachment 8950271 [details]
Bug 1436058 - Remove async image pipelines when the renderer notifies pipelines removed.

https://reviewboard.mozilla.org/r/219530/#review225254
Attachment #8950271 - Flags: review?(bugmail) → review+
Comment on attachment 8950272 [details]
Bug 1436058 - Don't remove a WebRender pipeline if we re-created one with the same id.

https://reviewboard.mozilla.org/r/219532/#review225256
Attachment #8950272 - Flags: review?(bugmail) → review+
Comment on attachment 8950268 [details]
Bug 1436058 - Update webrender to 342bc314db94aa439b2001249c5f24ccfcbccc22.

https://reviewboard.mozilla.org/r/219524/#review225258
Attachment #8950268 - Flags: review?(jmuizelaar) → review+
Comment on attachment 8950269 [details]
Bug 1436058 - Update Cargo lockfiles and re-vendor rust dependencies.

https://reviewboard.mozilla.org/r/219526/#review225262
Attachment #8950269 - Flags: review?(jmuizelaar) → review+
Comment on attachment 8950270 [details]
Bug 1436058 - Update for log 0.4 bump in WR PR 2385.

https://reviewboard.mozilla.org/r/219528/#review225266
Attachment #8950270 - Flags: review?(jmuizelaar) → review+
Pushed by kgupta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a4dc76286299
Update webrender to 342bc314db94aa439b2001249c5f24ccfcbccc22. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/9cca7938b734
Update Cargo lockfiles and re-vendor rust dependencies. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/cacc7bcb3220
Update for log 0.4 bump in WR PR 2385. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/a408b8f2f7ab
Remove async image pipelines when the renderer notifies pipelines removed. r=kats
https://hg.mozilla.org/integration/autoland/rev/f53e340b051c
Don't remove a WebRender pipeline if we re-created one with the same id. r=kats
There is a fix / workaround for the mipmap issue here - https://github.com/servo/webrender/pull/2409.

I have a local Windows build with this changeset, but I'm currently unable to reproduce any issues. All of the reftests I've run on Windows appear to run as expected (the sanity, flexbox and layout bugs directory, at least).
I have a possible theory on the Windows bustage.

If I run the flexbox/ reftests on both Linux and Windows the memory usage is relatively stable.

However, when I run the image/test/reftest reftests, there appears to be a significant memory leak. On Windows, the memory usage grows by ~100MB/sec until the process seems to go idle / timeout at ~4 GB allocated. Since the default Windows build is 32 bit, that would seem like an OOM is probably occurring somewhere. On Linux, I see the same memory growth, but since it's a 64-bit build, it doesn't have the same address space issues.

Is it possible that this could be the cause of the Windows issues on CI? There are a couple of patches in this changeset that deal with managing lifetime of image and pipeline resources, could they be related?
Flags: needinfo?(bugmail)
Depends on: 1437750
(Replied on bug 1437572)
Flags: needinfo?(bugmail)
Regressions: 1569724
You need to log in before you can comment on or make changes to this bug.