1676113 - High CPU and big memory leak on windows 10 WebRender (SVG/Blob/Scenebuilder)

Status: RESOLVED WORKSFORME

(Core :: Graphics: WebRender, defect, P3)

Description

[ivo]

Attachment: FirefoxIssue.html

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36

Steps to reproduce:

Open attached html page. The issue seems to be specific to something in the html page. Other pages I tried work well.

It seems to be related to windows 10. I have tried to reproduce the issue also in Firefox on Mac (Catalina 10.15.6) windows 8 and it seems to be ok there. I have not tried the mobile version.
I have also tried the older version 79 and the issue is not there on window 10.

Actual results:

Firefox starts immediately consume lot of cpu and memory until its crash or windows kills the process

Expected results:

Load fine without leak like in version 79 on windows 10.

Comment 1

[Alin Ilea, Desktop QA]

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0

Hi,

I have managed to reproduce the issue on latest Nightly 84.0a1 (2020-11-013) using Windows 10. The issue is not reproducible in Beta or Release.
Further, I will move this over to a component so developers can take a look over it. If this is not the correct component please feel free to change it to an appropriate one.
Regression range:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=7c5bd84d0851a0cf5aaededce8e9a701a28cd009&tochange=049d66db392d8ac74b262ab22a2aab62231fad5d
Possibly regressed by bug 1676538.

Thanks for your input.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Alin, bug 1676538 only impacts R600 GPUs which I would be surprised if you're using. Can you try to get a memory report from about:memory when the leak happens?

Comment 3

[Jeff Muizelaar [:jrmuizel]]

ivo, can you attach the graphics section of your about:support?

Comment 4

[Alin Ilea, Desktop QA]

Attachment: memory-report.json.gz

Jeff, please let me know if this will help you. Maybe another bug from the pushlog is the regressor. Thanks for looking at this.

Comment 5

[ivo]

Attachment: Copy of about:support Graphics section

Comment 6

[ivo]

Hi all,

I have attached the requested about:support Graphics section. It is from my PC, we were able to reproduce it also on different machines with different hardware.

Thank you for looking into this issue!

Comment 7

[Jeff Muizelaar [:jrmuizel]]

16,041.78 MB (99.63%) ── heap-unclassified in the gpu process.

Comment 8

[Jeff Muizelaar [:jrmuizel]]

Here's a stack of us trying to allocate 14GB

   0 libsystem_malloc.dylib malloc_zone_realloc
   1 libsystem_malloc.dylib realloc
   2 XUL alloc::alloc::realloc::h7aaf360305be3ee7 /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/alloc.rs:111
   3 XUL alloc::alloc::Global::grow_impl::h13edd6efd02988a4 /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/alloc.rs:186
   4 XUL _$LT$alloc..alloc..Global$u20$as$u20$core..alloc..AllocRef$GT$::grow::h399cc0e38b94bdaf /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/alloc.rs:238
   5 XUL alloc::raw_vec::finish_grow::ha161f14e8d3b5191 /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:491
   6 XUL alloc::raw_vec::RawVec$LT$T$C$A$GT$::grow_amortized::hec8c99f703f32ffc /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:427
   7 XUL alloc::raw_vec::RawVec$LT$T$C$A$GT$::try_reserve::h493e21fc268f3257 /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:316
   8 XUL alloc::raw_vec::RawVec$LT$T$C$A$GT$::reserve::h18852be8d5e1bf75 /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:310
   9 XUL alloc::vec::Vec$LT$T$GT$::push::hc9d45b266b84ea14 /Users/jrmuizel/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec.rs:1210
  10 XUL webrender::api_resources::ApiResources::create_blob_scene_builder_requests::_$u7b$$u7b$closure$u7d$$u7d$::hbfe87606a0ef2e9f /Users/jrmuizel/source/gecko-inbound/gfx/wr/webrender/src/api_resources.rs:282
  11 XUL webrender::image_tiling::for_each_tile_in_range::h3c547c6ac87287e0 /Users/jrmuizel/source/gecko-inbound/gfx/wr/webrender/src/image_tiling.rs:586
  12 XUL webrender::api_resources::ApiResources::create_blob_scene_builder_requests::h6a91fd94a2604228 /Users/jrmuizel/source/gecko-inbound/gfx/wr/webrender/src/api_resources.rs:263
  13 XUL webrender::api_resources::ApiResources::update::h7a1dbc98c7eae9c0 /Users/jrmuizel/source/gecko-inbound/gfx/wr/webrender/src/api_resources.rs:167
  14 XUL webrender::render_api::RenderApi::send_transaction::h2fc49925e29a6568 /Users/jrmuizel/source/gecko-inbound/gfx/wr/webrender/src/render_api.rs:1285
  15 XUL wr_api_send_transaction /Users/jrmuizel/source/gecko-inbound/gfx/webrender_bindings/src/bindings.rs:2094
  16 XUL mozilla::layers::WebRenderBridgeParent::SetDisplayList(mozilla::gfx::RectTyped<mozilla::LayoutDevicePixel, float> const&, mozilla::ipc::ByteBuf&&, mozilla::wr::BuiltDisplayListDescriptor const&, nsTArray<mozilla::layers::OpUpdateResource> const&, nsTArray<mozilla::layers::RefCountedShmem> const&, nsTArray<mozilla::ipc::Shmem> const&, mozilla::TimeStamp const&, mozilla::wr::TransactionBuilder&, mozilla::wr::Epoch, bool, bool) /Users/jrmuizel/source/gecko-inbound/gfx/layers/wr/WebRenderBridgeParent.cpp:1083
  17 XUL mozilla::layers::WebRenderBridgeParent::ProcessDisplayListData(mozilla::layers::DisplayListData&, mozilla::wr::Epoch, mozilla::TimeStamp const&, bool, bool) /Users/jrmuizel/source/gecko-inbound/gfx/layers/wr/WebRenderBridgeParent.cpp:1118
  18 XUL mozilla::layers::WebRenderBridgeParent::RecvSetDisplayList(mozilla::layers::DisplayListData&&, nsTArray<mozilla::layers::OpDestroy>&&, unsigned long long const&, mozilla::layers::BaseTransactionId<mozilla::layers::TransactionIdType> const&, bool const&, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, nsTString<char> const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::CompositionPayload>&&) /Users/jrmuizel/source/gecko-inbound/gfx/layers/wr/WebRenderBridgeParent.cpp:1165
  19 XUL mozilla::layers::PWebRenderBridgeParent::OnMessageReceived(IPC::Message const&) /Users/jrmuizel/source/gecko-inbound/obj-nomalloc/ipc/ipdl/PWebRenderBridgeParent.cpp:395
  20 XUL mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&) /Users/jrmuizel/source/gecko-inbound/obj-nomalloc/ipc/ipdl/PCompositorManagerParent.cpp:197
  21 XUL mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) /Users/jrmuizel/source/gecko-inbound/ipc/glue/MessageChannel.cpp:2150
  22 XUL mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) /Users/jrmuizel/source/gecko-inbound/ipc/glue/MessageChannel.cpp:2074
  23 XUL mozilla::ipc::MessageChannel::MessageTask::Run() /Users/jrmuizel/source/gecko-inbound/ipc/glue/MessageChannel.cpp:1953
  24 XUL nsThread::ProcessNextEvent(bool, bool*) /Users/jrmuizel/source/gecko-inbound/xpcom/threads/nsThread.cpp:1197

Comment 9

[Jeff Muizelaar [:jrmuizel]]

With the following patch I get tile counts of at least: 23582610 so it seems like something is going very wrong:

diff --git a/gfx/wr/webrender/src/api_resources.rs b/gfx/wr/webrender/src/api_resources.rs
index 0a48858fc422e..bf43fe9840498 100644
--- a/gfx/wr/webrender/src/api_resources.rs
+++ b/gfx/wr/webrender/src/api_resources.rs
@@ -240,4 +240,5 @@ impl ApiResources {
 
         let mut blob_request_params = Vec::new();
+        let mut count = 0;
         for key in keys {
             let template = self.blob_image_templates.get_mut(key).unwrap();
@@ -262,4 +263,8 @@ impl ApiResources {
 
             for_each_tile_in_range(&tiles, |tile| {
+                if count % 100 == 0 {
+                    println!("tile count: {}", count);
+                }
+                count += 1;
                 let still_valid = template.valid_tiles_after_bounds_change
                     .map(|valid_tiles| valid_tiles.contains(tile))

Comment 10

[Jeff Muizelaar [:jrmuizel]]

Alin, can you double check the regression window? I can reproduce the problem on older builds as well.

Comment 11

[Jeff Muizelaar [:jrmuizel]]

I'm guessing this was caused by bug 1555356

Comment 12

[Jeff Muizelaar [:jrmuizel]]

Yeah, I got https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e27a62f854b29efd5719040bb53227b65c3e954a&tochange=a86dcd6715c9e52fdd8e4a937b5e94ea2b9a5fcb as the regression window

Comment 13

[Rares Doghi, Desktop QA]

Hi Jeff, I re-did the mozregression for this issue and here is the Pushlog from the First Bad build:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=f340bbb582d16c083344b1370bd4e4d4caf5fb17&tochange=a86dcd6715c9e52fdd8e4a937b5e94ea2b9a5fcb

It Seems that "Bug 1555356 - Make images inside of SVGs active" might be the cause of it.

I have also update the flags.

Comment 14

[Jeff Muizelaar [:jrmuizel]]

The test case has lots of images, so we're probably splitting it into too many different blobs.

Comment 15

[Jeff Muizelaar [:jrmuizel]]

In the call we discussed only making large images active. That would probably solve this.

Instead of calling SVGGeometryFrame::CreateWebRenderCommands() in nsDisplaySVGGeometry::ShouldBeActive() we could add SVGGeometryFrame::ShouldBeActive() that we call first.

Comment 16

[Jeff Muizelaar [:jrmuizel]]

Further investigation reveals a tile range of: Rect(15139x15139 at (0, 0)) which suggests that we're trying to create 229189321 tiles which seems like too many

Comment 17

[Jeff Muizelaar [:jrmuizel]]

Yeah, our visible rect is 7750969x7750969 which feels too high.

Comment 18

[Robert Longson [:longsonr]]

bug 1684625 which basically disables web-render for SVG images so this bug is sort of fixed but it will come back if the svg-images pref is turned on again.

Comment 19

[Mayank Bansal]

Cant repro anymore with active SVG re-enabled. Should this be closed?

Comment 20

[Nicolas Silva [:nical]]

Let's close it.