1139928 - High memory usage in ImageLib with IMDb video panorama demo

Status: NEW

(Core :: Graphics: ImageLib, defect, P3)

Description

[Mark Straver]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0
Build ID: 20150303030230

Steps to reproduce:

1. Visit http://ie.microsoft.com/testdrive/Graphics/VideoPanorama/Default.html
2. Mouseover to scroll the panorama left/right


Actual results:

Video thumbnails flicker and the FPS displayed drops dramatically


Expected results:

Smooth FPS and no flickering

Comment 1

[Mark Straver]

I've noticed the same thing happens with the tab bar! especially the first time scrolling tabs horizontally seems to be very susceptible to this problem, causing it to flicker. Repeat scrolling doesn't seem to be impacted as much and hard to see.
So, likely this affects all horizontal pixel scrolling.

Happens on nightly and -release.

Graphics section of a:s

Graphics
Adapter Description	AMD Radeon HD 6800 Series
Adapter Drivers	aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64
Adapter RAM	1024
ClearType Parameters	Gamma: 2200 Pixel Structure: R ClearType Level: 0 Enhanced Contrast: 200
Device ID	0x6738
Direct2D Enabled	true
DirectWrite Enabled	true (6.2.9200.16492)
Driver Date	11-20-2014
Driver Version	14.501.1003.0
GPU #2 Active	false
GPU Accelerated Windows	1/1 Direct3D 11 (OMTC)
Subsys ID	174b174b
Vendor ID	0x1002
WebGL Renderer	Google Inc. -- ANGLE (AMD Radeon HD 6800 Series Direct3D9Ex vs_3_0 ps_3_0)
windowLayerManagerRemote	true
AzureCanvasBackend	direct2d
AzureContentBackend	direct2d
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0

Comment 2

[Mark Straver]

As an aside, testing FF40, I noticed that this makes memory consumption grow out of bounds in the tabs process. Scrolling from one end to the other and back again, memory usage grows to over 1.1GB just for that page.

All of it goes into imagelib-surface-cache by the looks of it. I would imagine that that memory usage might be directly related.

From about:memory:

    0.03 MB ── canvas-2d-pixels
    0.00 MB ── d3d11-shared-textures
    0.00 MB ── d3d9-shared-texture
    0.00 MB ── d3d9-shared-textures
    0.00 MB ── d3d9-surface-image
    0.00 MB ── gfx-d2d-surface-cache
    4.00 MB ── gfx-d2d-surface-vram
    0.00 MB ── gfx-d2d-vram-draw-target
    0.00 MB ── gfx-d2d-vram-source-surface
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-tiles-waste
          0 ── ghost-windows
   36.36 MB ── gpu-committed
   21.96 MB ── gpu-dedicated
    9.10 MB ── gpu-shared
1,032.77 MB ── heap-allocated
      1,112 ── heap-chunks
    1.00 MB ── heap-chunksize
1,040.58 MB ── heap-committed
1,112.00 MB ── heap-mapped
      0.75% ── heap-overhead-ratio
    0.00 MB ── imagelib-surface-cache-estimated-locked
1,023.75 MB ── imagelib-surface-cache-estimated-total
    0.32 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,124.42 MB ── private
1,137.59 MB ── resident
1,399.50 MB ── vsize
1,805.94 MB ── vsize-max-contiguous

Comment 3

[Mark Straver]

Also, it seems Firefox uses a completely ridiculous amount of memory for each image:

│  │  │  ├───98.71 MB (12.50%) -- image(http://ie.microsoft.com/testdrive/Graphics/VideoPanorama/thumbnails/Twilight_HTML5.jpg)
│  │  │  │   ├──98.66 MB (12.49%) ── decoded-heap
│  │  │  │   └───0.05 MB (00.01%) ── source

100MB for an image that, decoded, would use 1.6MB in decoded size at 853x480x4 (32bpp)...?

Comment 4

[Justin [:JW_SoftvisionQA]]

I can reproduce this bug on Nightly 46.0a1

Name 	Firefox
Version 	46.0a1
Build ID 	20151222030207
User Agent 	Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:46.0) Gecko/20100101 Firefox/46.0

Also on Firefox 43.0.2:

Name 	Firefox
Version 	43.0.1
Build ID 	20151216175450
User Agent 	Mozilla/5.0 (Windows NT 6.3; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0

I will mark this bug as new.

Comment 5

[u279076]

(In reply to Justin - QA from comment #4)
> I can reproduce this bug on Nightly 46.0a1

Can you provide steps to reproduce? The original link doesn't point to the actual demo anymore.

Comment 6

[Justin [:JW_SoftvisionQA]]

The link worked for me 5 min ago but now when I click it, it does not.

Comment 7

[Justin [:JW_SoftvisionQA]]

Anthony try http://www.flightarcade.com/

Comment 8

[u279076]

(In reply to Justin - QA from comment #7)
> Anthony try http://www.flightarcade.com/

The demo works fine for me, no flickering or performance issues. Can you provide a screencast to illustrate the issue? Also, please provide a copy of about:support for the system you're testing.

Comment 9

[Mark Straver]

Try the archive.org copy. Microsoft has removed older demos that this was one of.

https://web.archive.org/web/20150117144156/http://ie.microsoft.com/testdrive/Graphics/VideoPanorama/Default.html

Comment 10

[Mark Straver]

I've just tested on nightly, it maxes out all my cores and starts eating memory like there's no tomorrow when scrolling left and right. ouch.

Comment 11

[u279076]

(In reply to Mark Straver from comment #10)
> I've just tested on nightly, it maxes out all my cores and starts eating
> memory like there's no tomorrow when scrolling left and right. ouch.

Thanks, I'm not seeing any issue locally; memory usage is stable and cpu usage peaks at 22%. Could you provide an updated copy of about:support and about:memory? I suspect this isn't a graphics bug at all and might belong in ImageLib.

Comment 12

[Mark Straver]

I ran on a fresh profile to minimize interference, e10s enabled.

I think you may be right that it belongs in ImageLib since the memory issue seems to be centered around:

1,286.64 MB ── gpu-committed
1,023.99 MB ── imagelib-surface-cache-estimated-total

I'll attach a screenshot of my system resource meter clearly showing all 4 cores maxing out when scrolling, as well as a current about:memory measurement (after scrolling) and about:support output.

Comment 13

[Mark Straver]

Attachment: System resources during demo

Comment 14

[Mark Straver]

Attachment: about:memory after scrolling

Comment 15

[Mark Straver]

Attachment: aboutsupport.txt

Comment 16

[Mark Straver]

Of note, the actual flickering I initially included in this bug report has gone away somewhere, but this resource usage is still off the charts.

Comment 17

[u279076]

(In reply to Mark Straver from comment #16)
> Of note, the actual flickering I initially included in this bug report has
> gone away somewhere, but this resource usage is still off the charts.

Okay, so let's re-purpose this bug report for the memory issue and move it over to ImageLib.

Comment 18

[Mark Straver]

Thanks, although the CPU usage is just as limiting. Mind you, I have a pretty fast processor and it gets completely maxed out (4 cores of 3.4 GHz) and the resulting load does visibly impair the demo. Probably the same cause, but still something to note.

Comment 19

[u279076]

It'd be useful if you could capture a profile with the built-in profiler:
https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler

Comment 20

[Mark Straver]

Not sure if I did this correctly.
https://cleopatra.io/#report=b511690670fc5dd783a8814f0eb8768247f9d2f7

Comment 21

[u279076]

(In reply to Mark Straver from comment #20)
> Not sure if I did this correctly.
> https://cleopatra.io/#report=b511690670fc5dd783a8814f0eb8768247f9d2f7

Thanks, we'll need to wait for a developer to take a look at that profile.

Comment 22

[Timothy Nikkel (:tnikkel)]

(In reply to Mark Straver from comment #9)
> Try the archive.org copy. Microsoft has removed older demos that this was
> one of.
> 
> https://web.archive.org/web/20150117144156/http://ie.microsoft.com/testdrive/
> Graphics/VideoPanorama/Default.html

Testing this site I can see memory use go up. Like comment 3 it's image memory. The reason it's using so much memory is that there is more than one decoded copy of the image around, each one with a different size.

We should add some logic so that we don't bother decoding to a new size, but just scale the existing size during image size animations.

Comment 23

[Seth Fowler [:seth] [:s2h]]

(In reply to Timothy Nikkel (:tn) from comment #22)
> Testing this site I can see memory use go up. Like comment 3 it's image
> memory. The reason it's using so much memory is that there is more than one
> decoded copy of the image around, each one with a different size.
> 
> We should add some logic so that we don't bother decoding to a new size, but
> just scale the existing size during image size animations.

This is a known issue. If we layerize the image, it already works that way, but in this case the image seems to be getting scaled by animating the containing div's width and height and we're unable to layerize it.

I have a patch floating around somewhere to fix this that I need to rebase and upload. The algorithm is roughly as follows:

(1) Initialize a timestamp value to be infinitely in the past and a counter to 0.

(2) When RasterImage calls SurfaceCache::LookupBestMatch() and gets a MatchType of SUBSTITUTE_BECAUSE_PENDING, (this indicates that we have *some* decoded surface for the image, but it's of the wrong size) check the timestamp.

  (2a) If the timestamp's greater than X seconds in the past, set the timestamp to the current time and set the counter to 1.

  (2b) If the timestamp's within the last X seconds, increment the counter.

(3) When RasterImage::LookupFrame() gets called, check the counter.

  (3a) If the counter is greater than 0 and the timestamp is greater than X seconds in the past, set the counter to 0.

  (3b) If the counter is greater than Y and the timestamp is within the last X seconds, set the timestamp to the current time and replace the normal lookup logic with the following behavior: look up the exact size the caller requested using SurfaceCache::Lookup() (so no substitution allowed), and if that lookup fails, proceed with the normal lookup logic after first setting the size we're looking for to the intrinsic size of the image.

Hopefully it's clear how that all works. Basically if we have to substitute surfaces more than Y times within an X second window (indicating that we keep getting requests for new sizes), we will only return a non-intrinsic-size surface if we already decoded it. If we don't have a surface for a certain size decoded already, we will fall back to the intrinsic size, so we'll never trigger a decode for any size other than the intrinsic size from that point forward. We'll stay in that state until we manage to go more than X seconds without RasterImage::LookupFrame() being called for that image; when we do hit that point, we'll return to our normal behavior.

Comment 24

[Mark Straver]

So, IIUC, the high CPU usage I see is because the source image gets decoded over and over? Why is this necessary, to begin with? Shouldn't the source image be decoded once - and only once! - to its original (intrinsic) size, stored in its raw decoded format, and then resized as-needed for the page element? IMHO decoding it more than once when a decoded version is already available is a total waste and just throwing away performance fnar.