Closed Bug 1133275 Opened 9 years ago Closed 1 year ago

Glitched colors/solid black in regions of the page being repainted, possibly due to VRAM exhaustion

Categories

(Core :: Graphics, defect)

Product:
Core
Component:
Graphics
Version:
37 Branch
Platform:
x86_64
Windows 8.1
Type:
defect

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: kael, Unassigned)

Details

(Whiteboard: gfx-noted)

Attachments

(2 files)

Glitch in effect on Youtube
524.90 KB, image/png
Details
Screenshot of compositor glitch on Nightly
836.91 KB, image/jpeg
Details 
Geforce GTX 970 x2 (latest driver - 347.52), Win8.1, Aurora 37.0a2 (2015-02-12)

Sometimes after I've had a browser session going for a while, certain pages will get into a hosed state where they no longer repaint correctly. So far I've seen this on pages using WebGL (Bungie's online 'profile' pages for Destiny player characters) and pages using HTML5 video (Youtube).

In this hosed state, partial repaints (like when mousing over things, or when content changes) result in incorrect colors or solid black being painted inside the repaint region, or areas outside the repaint region turning black.

In both cases, the high-framerate content (WebGL viewport containing a textured model, HTML5 video that's playing) is totally fine and doesn't seem to glitch; it's just that while it's going everything else in the page is messed up.

Closing the tab seems to make everything recover, and every other tab in the browser is still fine.

The first time I saw this was a couple weeks ago; I don't have a reliable repro yet.

According to about:memory, Firefox is using a ton of VRAM, so maybe this is caused by demanding pages driving things up to the point where surfaces are being evicted/lost or something?

1,748.92 MB ── gpu-committed
1,095.20 MB ── gpu-dedicated
  152.81 MB ── gpu-shared

Here's my about:support related to gfx. (No gfx prefs have been changed)

Graphics
--------

Adapter Description: NVIDIA GeForce GTX 970
Adapter Drivers: nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um
Adapter RAM: 4095
Device ID: 0x13c2
Direct2D Enabled: true
DirectWrite Enabled: true (6.3.9600.17111)
Driver Date: 2-5-2015
Driver Version: 9.18.13.4752
GPU #2 Active: false
GPU Accelerated Windows: 2/2 Direct3D 11 (OMTC)
Subsys ID: 39753842
Vendor ID: 0x10de
WebGL Renderer: Google Inc. -- ANGLE (NVIDIA GeForce GTX 970 Direct3D11 vs_5_0 ps_5_0)
windowLayerManagerRemote: true
AzureCanvasBackend: direct2d 1.1
AzureContentBackend: direct2d 1.1
AzureFallbackCanvasBackend: cairo
AzureSkiaAccelerated: 0
Thinking back, in one case this started happening while I was playing a youtube video and even the *video* became black, at which point the browser totally hung and I had to terminate the process.

I checked in procexp and the committed/dedicated/system GPU memory statistics here are accurate; Firefox really is using that much vram. Ouch.
I think this is something fundamental about how Firefox uses VRAM on my configuration, and that performance/behavior degrades once it's using too much. Some experiments:

-- fresh browser instance --

  339.59 MB ── d3d11-shared-textures
    0.00 MB ── d3d9-shared-texture
    0.00 MB ── d3d9-shared-textures
    0.00 MB ── d3d9-surface-image
    0.00 MB ── gfx-d2d-surface-cache
    4.00 MB ── gfx-d2d-surface-vram
    0.00 MB ── gfx-d2d-vram-draw-target
    0.00 MB ── gfx-d2d-vram-source-surface
    0.05 MB ── gfx-surface-win32
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-tiles-waste
          0 ── ghost-windows
1,097.39 MB ── gpu-committed
  951.97 MB ── gpu-dedicated
   19.52 MB ── gpu-shared
  279.70 MB ── heap-allocated
        371 ── heap-chunks
    1.00 MB ── heap-chunksize
  286.35 MB ── heap-committed
  371.00 MB ── heap-mapped
      2.37% ── heap-overhead-ratio
          0 ── host-object-urls
   11.45 MB ── imagelib-surface-cache-estimated-locked
  141.24 MB ── imagelib-surface-cache-estimated-total
    1.77 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
  561.81 MB ── private
  584.54 MB ── resident
1,963.21 MB ── vsize
1,746.19 MB ── vsize-max-contiguous

-- fresh, gc/cc/minimize --


  339.59 MB ── d3d11-shared-textures
    0.00 MB ── d3d9-shared-texture
    0.00 MB ── d3d9-shared-textures
    0.00 MB ── d3d9-surface-image
    0.00 MB ── gfx-d2d-surface-cache
    4.00 MB ── gfx-d2d-surface-vram
    0.00 MB ── gfx-d2d-vram-draw-target
    0.00 MB ── gfx-d2d-vram-source-surface
    0.05 MB ── gfx-surface-win32
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-tiles-waste
          0 ── ghost-windows
  969.27 MB ── gpu-committed
  814.18 MB ── gpu-dedicated
   16.75 MB ── gpu-shared
  209.88 MB ── heap-allocated
        364 ── heap-chunks
    1.00 MB ── heap-chunksize
  216.81 MB ── heap-committed
  364.00 MB ── heap-mapped
      3.30% ── heap-overhead-ratio
          0 ── host-object-urls
   11.45 MB ── imagelib-surface-cache-estimated-locked
   11.45 MB ── imagelib-surface-cache-estimated-total
    1.77 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
  488.07 MB ── private
  508.32 MB ── resident
1,883.28 MB ── vsize
1,805.94 MB ── vsize-max-contiguous

-- fresh, clean profile --

  383.68 MB ── gpu-committed
  380.63 MB ── gpu-dedicated
    9.22 MB ── gpu-shared

-- fresh, clean profile after opening ~12 copies of the 'new tab' page --

  528.67 MB ── gpu-committed
  575.57 MB ── gpu-dedicated
   25.30 MB ── gpu-shared

-- fresh, clean profile after opening many windows --

3,571.06 MB ── gpu-committed
3,416.63 MB ── gpu-dedicated
   50.91 MB ── gpu-shared


So a fresh copy of the browser isn't actually doing much better VRAM-wise, and gc/cc/minimize don't put much of a dent in the usage.
Spawning a fresh copy of the browser on a clean profile looks a lot better; a few hundred megs of VRAM. GC/CC/minimize don't do *anything* there so i didn't include them.

Opening copies of the new tab page eats up more VRAM than I'd expect. This is probably part of why my main profile uses so much VRAM - I think I've got maybe 50 tabs open? This is despite the fact that most of those tabs aren't loaded, since they're from a previous session (they load once I click them.)

The many windows test is terrifying. I hit ctrl-N a dozen times or so (I actually did this by accident when I was trying to open tabs), and shortly the windows that are opening start glitching out (solid black regions, etc) - eventually one of them was just solid white and I couldn't get it to paint no matter what I did.

Interesting side note: Even in this 'bad state' where I'm clearly using too much VRAM, the windows I already had open seem to be working fine. Only the new one that's solid white is hosed.

In these low video memory scenarios I would have expected FF to fall back onto software rasterization for new windows. I think it used to.
Summary: Glitched colors/solid black in regions of the page being repainted → Glitched colors/solid black in regions of the page being repainted, possibly due to VRAM exhaustion
I can reproduce the extreme VRAM usage in Firefox 35 as well. Opening enough windows causes painting to permanently break in all the windows, and even if I close enough to bring the usage back down, the existing windows never paint correctly again.
Is it reasonable that vram/address space exhaustion could cause these symptoms or should I file a separate bug for that? It seems to coincide with it but it's possible that's a coincidence. The browser certainly seems to behave this way when it can't allocate surfaces, and I'm pretty close to running out of address space sometimes too.

Main profile after being open for a day or so

    0.01 MB ── canvas-2d-pixels
  241.12 MB ── d3d11-shared-textures
    0.00 MB ── d3d9-shared-texture
    0.00 MB ── d3d9-shared-textures
    0.00 MB ── d3d9-surface-image
    0.00 MB ── gfx-d2d-surface-cache
    4.00 MB ── gfx-d2d-surface-vram
    0.00 MB ── gfx-d2d-vram-draw-target
    0.00 MB ── gfx-d2d-vram-source-surface
    5.73 MB ── gfx-surface-win32
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-tiles-waste
          0 ── ghost-windows
1,522.98 MB ── gpu-committed
1,091.32 MB ── gpu-dedicated
  180.03 MB ── gpu-shared
  540.18 MB ── heap-allocated
        629 ── heap-chunks
    1.00 MB ── heap-chunksize
  549.54 MB ── heap-committed
  629.00 MB ── heap-mapped
      1.73% ── heap-overhead-ratio
          0 ── host-object-urls
   10.10 MB ── imagelib-surface-cache-estimated-locked
  124.41 MB ── imagelib-surface-cache-estimated-total
    3.39 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,311.26 MB ── private
1,329.45 MB ── resident
2,967.11 MB ── vsize
  395.63 MB ── vsize-max-contiguous

After GC/CC/Minimize

    0.01 MB ── canvas-2d-pixels
  231.63 MB ── d3d11-shared-textures
    0.00 MB ── d3d9-shared-texture
    0.00 MB ── d3d9-shared-textures
    0.00 MB ── d3d9-surface-image
    0.00 MB ── gfx-d2d-surface-cache
    4.00 MB ── gfx-d2d-surface-vram
    0.00 MB ── gfx-d2d-vram-draw-target
    0.00 MB ── gfx-d2d-vram-source-surface
    6.94 MB ── gfx-surface-win32
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-tiles-waste
          0 ── ghost-windows
1,449.41 MB ── gpu-committed
  968.75 MB ── gpu-dedicated
  129.52 MB ── gpu-shared
  522.57 MB ── heap-allocated
        631 ── heap-chunks
    1.00 MB ── heap-chunksize
  532.55 MB ── heap-committed
  631.00 MB ── heap-mapped
      1.90% ── heap-overhead-ratio
          0 ── host-object-urls
   10.08 MB ── imagelib-surface-cache-estimated-locked
   10.08 MB ── imagelib-surface-cache-estimated-total
    3.39 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,258.19 MB ── private
1,230.86 MB ── resident
2,914.55 MB ── vsize
  395.63 MB ── vsize-max-contiguous
Looks more like it happens when address space is near the limit. Repro'd it again on youtube.

1,467.55 MB ── gpu-committed
1,055.02 MB ── gpu-dedicated
  230.10 MB ── gpu-shared
  609.22 MB ── heap-allocated
        855 ── heap-chunks
    1.00 MB ── heap-chunksize
  620.87 MB ── heap-committed
  855.00 MB ── heap-mapped
      1.91% ── heap-overhead-ratio
          1 ── host-object-urls
   84.36 MB ── imagelib-surface-cache-estimated-locked
  127.78 MB ── imagelib-surface-cache-estimated-total
    5.20 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,673.32 MB ── private
1,702.34 MB ── resident
3,605.24 MB ── vsize
Some fixes have recently landed on Nightly that should help with this. Can you reproduce in Nightly?
Flags: needinfo?(kg)
I'll run this profile in Nightly for a few days and let you know.
Looks promising already:

Fresh instance 37.0a2 (2015-02-20)

  848.45 MB ── gpu-committed
  736.62 MB ── gpu-dedicated
   12.77 MB ── gpu-shared
  172.94 MB ── heap-allocated
        179 ── heap-chunks
    1.00 MB ── heap-chunksize
  175.06 MB ── heap-committed
  179.00 MB ── heap-mapped
      1.22% ── heap-overhead-ratio
          0 ── host-object-urls
    3.45 MB ── imagelib-surface-cache-estimated-locked
   24.86 MB ── imagelib-surface-cache-estimated-total
    1.07 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
  310.09 MB ── private
  338.29 MB ── resident
1,537.75 MB ── vsize
1,805.94 MB ── vsize-max-contiguous

Fresh instance 38.0a1 (2015-02-20)

  472.57 MB ── gpu-committed
  496.80 MB ── gpu-dedicated
   16.37 MB ── gpu-shared
  236.60 MB ── heap-allocated
        245 ── heap-chunks
    1.00 MB ── heap-chunksize
  239.22 MB ── heap-committed
  245.00 MB ── heap-mapped
      1.10% ── heap-overhead-ratio
          0 ── host-object-urls
    3.35 MB ── imagelib-surface-cache-estimated-locked
   33.56 MB ── imagelib-surface-cache-estimated-total
    1.06 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
  408.30 MB ── private
  439.47 MB ── resident
1,427.90 MB ── vsize
1,805.94 MB ── vsize-max-contiguous
Flags: needinfo?(kg)
Reproduced crazy compositor glitching again on this page:
https://www.bungie.net/en/Profile/254/9323990

1,782.16 MB ── gpu-committed
1,303.97 MB ── gpu-dedicated
  206.53 MB ── gpu-shared
  659.50 MB ── heap-allocated
        891 ── heap-chunks
    1.00 MB ── heap-chunksize
  670.62 MB ── heap-committed
  891.00 MB ── heap-mapped
      1.68% ── heap-overhead-ratio
          0 ── host-object-urls
   14.13 MB ── imagelib-surface-cache-estimated-locked
   69.16 MB ── imagelib-surface-cache-estimated-total
    5.21 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,428.61 MB ── private
1,428.34 MB ── resident
3,183.24 MB ── vsize

The glitching is constant and seems to be triggered because the page is repainting webgl content (and some animated dom content too I think).

This might be a different bug but it seems really similar.
Opening YT in this state doesn't seem to reproduce glitching on the youtube page, but it does make other pages stop compositing right. Maybe YT is a different bug.
The compositor glitching on the Bungie site subsides temporarily if I halve the width of the window with Win-Left. Growing the window back to fullscreen makes it glitch like crazy again. Restarting fixes it too.
Happened without any HTML5 video / webgl content open this time.


1,382.73 MB ── gpu-committed
  947.16 MB ── gpu-dedicated
  236.92 MB ── gpu-shared
1,075.12 MB ── heap-allocated
      1,192 ── heap-chunks
    1.00 MB ── heap-chunksize
1,085.02 MB ── heap-committed
1,192.00 MB ── heap-mapped
      0.92% ── heap-overhead-ratio
          0 ── host-object-urls
   23.82 MB ── imagelib-surface-cache-estimated-locked
   46.40 MB ── imagelib-surface-cache-estimated-total
    7.31 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,911.04 MB ── private
1,479.05 MB ── resident
3,509.96 MB ── vsize
   60.00 MB ── vsize-max-contiguous

vsize-max-contiguous of 60mb seems bad :-)
Wow, already hit it again. I had a paused youtube video in one tab and opened Google Maps in another.

1,838.29 MB ── gpu-committed
1,232.10 MB ── gpu-dedicated
  218.82 MB ── gpu-shared
  910.78 MB ── heap-allocated
        956 ── heap-chunks
    1.00 MB ── heap-chunksize
  920.81 MB ── heap-committed
  956.00 MB ── heap-mapped
      1.10% ── heap-overhead-ratio
          1 ── host-object-urls
    3.08 MB ── imagelib-surface-cache-estimated-locked
  158.62 MB ── imagelib-surface-cache-estimated-total
    2.64 MB ── js-main-runtime-temporary-peak
          0 ── low-commit-space-events
1,756.83 MB ── private
1,668.59 MB ── resident
3,429.53 MB ── vsize
  171.63 MB ── vsize-max-contiguous

Google maps was turning black and then showing a 'something went wrong, please reload the page' message at the top. Reloads choked with this same compositor glitching & solid black pretty promptly.

Is there a way I can try to get accounting of these gpu resources on a per-page basis? Explicit heap is never more than around 800mb, it seems likely that most of this is the gpu resources eating up address space.
On 64-bit builds I don't get the black compositor glitches, so I can keep a session going as long as I want. GPU commit still builds up over time though, I opened maps up a few moments ago and checked about:memory out of curiosity:

      3,613.51 MB ── gpu-committed
      2,277.82 MB ── gpu-dedicated

Opening that same maps page in an empty profile only used about 180MB of gpu memory at the same window size, so it's not maps specifically. Restarting this profile drops me to around 1600mb of GPU commit, which is still not great. I think I'm just getting demolished by GPU memory usage from my # of tabs, and then in 32-bit builds that blows away my address space and things fall over.

On these 64-bit builds I don't actually see the heap get out of control. It builds up and can get somewhat large but the individual page usage is pretty reasonable. The only thing remotely resembling a leak I see there is 100-200mb image data from a long-running session that doesn't go away after GC/CC/minimize.

I definitely think the accumulated GPU commit is not right here.
I realized I never tested my theory that my 4k monitor is why I hit a ceiling here. It is definitely part of the problem. I tested this by moving my two browser windows from maximized on the 4k panel to maximized on the 1080p panel, then restarting.

fresh, 1080p
        114.97 MB ── d3d11-shared-textures
          0.00 MB ── d3d9-shared-texture
          0.00 MB ── d3d9-shared-textures
          0.00 MB ── d3d9-surface-image
          0.00 MB ── gfx-d2d-surface-cache
          4.00 MB ── gfx-d2d-surface-vram
          0.00 MB ── gfx-d2d-vram-draw-target
          0.00 MB ── gfx-d2d-vram-source-surface
          0.04 MB ── gfx-surface-win32
          0.00 MB ── gfx-textures
          0.00 MB ── gfx-tiles-waste
                0 ── ghost-windows
        386.45 MB ── gpu-committed
        333.17 MB ── gpu-dedicated
         17.61 MB ── gpu-shared
        396.91 MB ── heap-allocated
              417 ── heap-chunks
          1.00 MB ── heap-chunksize
        403.33 MB ── heap-committed
        417.00 MB ── heap-mapped
            1.61% ── heap-overhead-ratio
                0 ── host-object-urls
          6.51 MB ── imagelib-surface-cache-estimated-locked
         48.32 MB ── imagelib-surface-cache-estimated-total
          1.25 MB ── js-main-runtime-temporary-peak
        647.04 MB ── private
        631.12 MB ── resident
      1,528.89 MB ── vsize

then I dragged the two windows back to the 4k monitor:
fresh, 1080p -> drag to 4k monitor

        246.43 MB ── d3d11-shared-textures
          0.00 MB ── d3d9-shared-texture
          0.00 MB ── d3d9-shared-textures
          0.00 MB ── d3d9-surface-image
          0.00 MB ── gfx-d2d-surface-cache
          4.00 MB ── gfx-d2d-surface-vram
          0.00 MB ── gfx-d2d-vram-draw-target
          0.00 MB ── gfx-d2d-vram-source-surface
          0.04 MB ── gfx-surface-win32
          0.00 MB ── gfx-textures
          0.00 MB ── gfx-tiles-waste
                0 ── ghost-windows
      1,228.44 MB ── gpu-committed
        733.87 MB ── gpu-dedicated
         20.99 MB ── gpu-shared
        516.80 MB ── heap-allocated
              636 ── heap-chunks
          1.00 MB ── heap-chunksize
        528.38 MB ── heap-committed
        636.00 MB ── heap-mapped
            2.24% ── heap-overhead-ratio
                0 ── host-object-urls
          7.20 MB ── imagelib-surface-cache-estimated-locked
        142.49 MB ── imagelib-surface-cache-estimated-total
          3.16 MB ── js-main-runtime-temporary-peak
        877.28 MB ── private
        855.54 MB ── resident
      2,186.39 MB ── vsize

then restarted on the 4k monitor:
fresh, 4k

        437.55 MB ── d3d11-shared-textures
          0.00 MB ── d3d9-shared-texture
          0.00 MB ── d3d9-shared-textures
          0.00 MB ── d3d9-surface-image
          0.00 MB ── gfx-d2d-surface-cache
          4.00 MB ── gfx-d2d-surface-vram
          0.00 MB ── gfx-d2d-vram-draw-target
          0.00 MB ── gfx-d2d-vram-source-surface
          0.05 MB ── gfx-surface-win32
          0.00 MB ── gfx-textures
          0.00 MB ── gfx-tiles-waste
                0 ── ghost-windows
      1,207.83 MB ── gpu-committed
      1,028.68 MB ── gpu-dedicated
         20.63 MB ── gpu-shared
        366.59 MB ── heap-allocated
              453 ── heap-chunks
          1.00 MB ── heap-chunksize
        376.03 MB ── heap-committed
        453.00 MB ── heap-mapped
            2.57% ── heap-overhead-ratio
                0 ── host-object-urls
          6.89 MB ── imagelib-surface-cache-estimated-locked
         83.28 MB ── imagelib-surface-cache-estimated-total
          2.58 MB ── js-main-runtime-temporary-peak
        664.62 MB ── private
        646.87 MB ── resident
      2,279.89 MB ── vsize

Out of curiosity I checked my high-dpi (but not 4k) laptop running 32-bit developer edition and the gpu memory usage is pretty low (~300mb) with a dozen tabs open. Even opening a dozen more tabs didn't make it climb that much. I'm not sure if that means anything - it's got a different vendor's GPU, so this could be something about my drivers or my configuration?
Whiteboard: gfx-noted
Severity: normal → S3

Reporter, are you still experiencing graphical glitches when Firefox uses large amounts of VRAM?

Flags: needinfo?(kg)

Nope

Status: NEW → RESOLVED
Closed: 1 year ago
Flags: needinfo?(kg)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: