1582645 - Crash in [@ webrender_bindings::moz2d_renderer::BlobReader::read_entry]

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P1)

Description

[Pascal Chevrel:pascalc (PTO until April 26)]

This bug is for crash report bp-1d03375a-1d36-415b-9b7e-d44250190919.

Top 10 frames of crashing thread:

0 xul.dll GeckoCrash toolkit/xre/nsAppRunner.cpp:5117
1 xul.dll static void gkrust_shared::panic_hook toolkit/library/rust/shared/lib.rs:248
2 xul.dll static void core::ops::function::Fn::call<fn src/libcore/ops/function.rs:69
3 xul.dll static void std::panicking::rust_panic_with_hook src/libstd/panicking.rs:481
4 xul.dll static <NoType> std::panicking::begin_panic<str*> src/libstd/panicking.rs:411
5 xul.dll static struct webrender_bindings::moz2d_renderer::Entry webrender_bindings::moz2d_renderer::BlobReader::read_entry gfx/webrender_bindings/src/moz2d_renderer.rs
6 xul.dll static void webrender_bindings::moz2d_renderer::{{impl}}::update gfx/webrender_bindings/src/moz2d_renderer.rs:641
7 xul.dll static void webrender::resource_cache::ResourceCache::pre_scene_building_update gfx/wr/webrender/src/resource_cache.rs:674
8 xul.dll static union core::option::Option<alloc::boxed::Box<webrender::scene_builder_thread::Transaction>> core::iter::adapters::{{impl}}::next<alloc::boxed::Box<webrender::scene_builder_thread::Transaction>, core::iter::adapters::zip::Zip<core::slice::Iter<webrender_api::api::DocumentId>, alloc::vec::Drain<webrender_api::api::TransactionMsg>>, closure> src/libcore/iter/adapters/mod.rs:570
9 xul.dll static union webrender::render_backend::RenderBackendStatus webrender::render_backend::RenderBackend::process_api_msg gfx/wr/webrender/src/render_backend.rs:1209

Comment 1

[Pascal Chevrel:pascalc (PTO until April 26)]

Calixte, could you identify the source of this regression with Clouseau please? Thanks

Comment 2

[Julien Cristau [:jcristau] (back April 22)]

Bug 1570081 landed in that build (20190918100042).

Comment 3

[Nicolas Silva [:nical]]

Jeff is the most knowledgeable about the blob serialization code.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

This could also be bug 1570435 or bug 1581953

Comment 5

[Calixte Denizet (:calixte)]

According to clouseau, it could be bug 1570435.

Comment 6

[Jeff Muizelaar [:jrmuizel]]

I can reproduce this on bongacams.com

Comment 7

[Jeff Muizelaar [:jrmuizel]]

It looks like this is caused by https://phabricator.services.mozilla.com/D43222. We ignore the update with an empty visible rect. The next time we go round, we use the empty visible rect on content side, but the previous visible rect in the blob merging.

Comment 8

[Jeff Muizelaar [:jrmuizel]]

Nical, do you recall why this change was needed?

Comment 9

[Nicolas Silva [:nical]]

I don't remember exactly which but there are a bunch of places that break with sizes equal to zero in webrender (some divisions, assertions here and there).
We could make webrender robust against these (I don't think it would be difficult). At the time it seemd straightforward to not add something to the display list when we know it won't be rendered, but I didn't anticipate it would break the flow of creating and updating resources.

Comment 10

[Jeff Muizelaar [:jrmuizel]]

Attachment: A test case that shows the problem

Comment 11

[Jeff Muizelaar [:jrmuizel]]

Attachment: svg-empty.html

A further reduced test case

Comment 12

[Jeff Muizelaar [:jrmuizel]]

Attachment: Bug 1582645. Only set mLastVisibleRect after we've sent it to WebRender.

This avoids us setting when we don't send it. e.g. When it's empty.

Comment 13

[Maire Reavy [:mreavy]]

Thanks for jumping on this, Jeff.

Comment 14

[Jeff Muizelaar [:jrmuizel]]

I ran into some reftest failures on try that I'm still trying to figure out.

Comment 15

[Adam Stevenson [:adamopenweb]]

I experienced two crashes in Nightly when using Google docs (slides) on MacOS. My crash report linked to this bug.

Here's a crash report if it helps.
https://crash-stats.mozilla.org/report/index/c482bcbd-4bee-4927-8979-6abde0190926

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43ac974f69db
Only set mLastVisibleRect after we've sent it to WebRender. r=nical

Comment 17

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Crash with this signature on autoland after the patch for this bug landed: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=superseded%2Cretry%2Cusercancel%2Ctestfailed%2Cbusted%2Cexception&revision=1e0685b06c61f350da1973d419223883ba64d3e2&selectedJob=268750332

Comment 18

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/43ac974f69db

Comment 19

[novenary]

I've been having crashes while using Discord since 2019-09-27, running up to date Arch Linux. Worth noting is that I disable the GPU process to work around bug 1572625, and the crash is basically impossible to reproduce if I enable it again (maybe it's crashing silently in the background?).
Crash report: https://crash-stats.mozilla.org/report/index/82050440-8f48-4286-b4d4-df6de0190928
The regression window I found with mozregression seems to point to the fix for this bug: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=aa8f530a1a35ba9b1c84303dbe15107d0026d77c&tochange=1e0685b06c61f350da1973d419223883ba64d3e2

Comment 20

[Intermittent Failures Robot]

13 failures in 4530 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 13

Platform breakdown:

• windows10-64-qr: 9
• linux64-qr: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1582645&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 21

[Cristina Coroiu [:ccoroiu]]

Backed out changeset 43ac974f69db (bug 1582645) for increasing the crash volume

Backout: https://hg.mozilla.org/mozilla-central/rev/de4a32bbbe50d2a21e38eab7a4042378cc435aa8

Comment 22

[Jeff Muizelaar [:jrmuizel]]

Dan, do you have any idea what in Discord was triggering the issue? i.e. what were your steps to reproduce when running the regression window?

Comment 23

[Jeff Muizelaar [:jrmuizel]]

I was able to reproduce on https://play.drova.io/merchant

Comment 24

[novenary]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #22)

Dan, do you have any idea what in Discord was triggering the issue? i.e. what were your steps to reproduce when running the regression window?

I'm not entirely sure, sometimes it crashes on its own and sometimes when I type into the message box. Either way, it happens really fast. With mozregression, I was only able to get a full browser crash with --pref layers.gpu-process.enabled:false. I've seen a lag spike once without (GPU process crashing in the background?), but I have no idea how to get debugging information in that case.

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df723e7120dc
Only set mLastVisibleRect after we've sent it to WebRender. r=nical

Comment 26

[Jeff Muizelaar [:jrmuizel]]

Attachment: Bug 1582645. Add crash test to manifest.

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df8178d120eb
Add crash test to manifest.

Comment 28

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/df723e7120dc
https://hg.mozilla.org/mozilla-central/rev/df8178d120eb

Comment 29

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

This bug has been identified as part of a pilot on determining root causes of blocking and dot release drivers.

It needs a root-cause set for it. Please see the list at https://docs.google.com/document/d/1FFEGsmoU8T0N8R9kk-MXWptOPtXXXRRIe4vQo3_HgMw/.

Add the root cause as a whiteboard tag in the form [rca - <cause> ] and remove the rca-needed keyword.

If you have questions, please contact :tmaity.