1305298 - SEGV in sk_memset32 after GraphicsCriticalError

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect, P5)

Description

[Atte Kettunen]

Attachment: minimized-repro-file.hml

Tested on: 

OS: Ubuntu 16.04

Firefox: ASAN-build from https://ftp.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-linux64-asan/1470837316/

Note: Reproducing this bug takes some time. Open the attached repro-file in Firefox ASAN-build and let it run for some time. On my laptop it took ~30s for the crash to trigger. The repro-file may also cause the computer to go unresponsive.

I attached a minimized repro-file. The original repro-file cause SEGV with multiple different stack-traces, after minimizer the repro-file is more stable. 


ASAN-trace:

Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Failed 2 buffer db=0 dw=0 for 6606092, 2667421, 14696, 14697 (t=10.3998) [GFX1-]: Failed 2 buffer db=0 dw=0 for 6606092, 2667421, 14696, 14697
Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Failed 2 buffer db=0 dw=0 for 6606092, 2667421, 14696, 14697 (t=10.3998) |[C1][GFX1-]: Failed 2 buffer db=0 dw=0 for 6606092, 2667421, 14696, 14697 (t=14.751) [GFX1-]: Failed 2 buffer db=0 dw=0 for 6606092, 2667421, 14696, 14697
ASAN:DEADLYSIGNAL
=================================================================
==8184==ERROR: AddressSanitizer: SEGV on unknown address 0x7efc927f3000 (pc 0x7efe0d2df19a bp 0x7fff2eb23150 sp 0x7fff2eb23140 T0)
    #0 0x7efe0d2df199 in sk_memset32 /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkUtils.h:31:19
    #1 0x7efe0d2df199 in D32_Src_BitmapXferProc(void*, unsigned long, unsigned int) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkDraw.cpp:149
    #2 0x7efe0d2c9a62 in CallBitmapXferProc /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkDraw.cpp:253:9
    #3 0x7efe0d2c9a62 in SkDraw::drawPaint(SkPaint const&) const /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkDraw.cpp:284
    #4 0x7efe0d01224a in SkCanvas::internalDrawPaint(SkPaint const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:2049:9
    #5 0x7efe0d011cde in SkCanvas::onDrawPaint(SkPaint const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:2042:5
    #6 0x7efe0d02309c in drawPaint /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:1886:5
    #7 0x7efe0d02309c in SkCanvas::drawColor(unsigned int, SkXfermode::Mode) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/src/core/SkCanvas.cpp:2799
    #8 0x7efe0572081a in clear /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/skia/skia/include/core/SkCanvas.h:603:9
    #9 0x7efe0572081a in mozilla::gfx::DrawTargetSkia::ClearRect(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, float> const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/2d/DrawTargetSkia.cpp:1692
    #10 0x7efe059ff0df in mozilla::layers::RotatedContentBuffer::BorrowDrawTargetForPainting(mozilla::layers::RotatedContentBuffer::PaintState&, mozilla::layers::RotatedContentBuffer::DrawIterator*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/layers/RotatedBuffer.cpp:779:7
    #11 0x7efe05af2d1f in mozilla::layers::ClientPaintedLayer::PaintThebes() /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/layers/client/ClientPaintedLayer.cpp:81:31
    #12 0x7efe05af39b1 in mozilla::layers::ClientPaintedLayer::RenderLayerWithReadback(mozilla::layers::ReadbackProcessor*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/layers/client/ClientPaintedLayer.cpp:148:3
    #13 0x7efe05b0bd50 in mozilla::layers::ClientContainerLayer::RenderLayer() /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/layers/client/ClientContainerLayer.h:62:7
    #14 0x7efe05b0bd50 in mozilla::layers::ClientContainerLayer::RenderLayer() /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/layers/client/ClientContainerLayer.h:62:7
    #15 0x7efe05b0bd50 in mozilla::layers::ClientContainerLayer::RenderLayer() /builds/slave/m-cen-l64-asan-000000000000000/build/src/gfx/layers/client/ClientContainerLayer.h:62:7
.
.
.

Comment 1

[Atte Kettunen]

Attachment: original-repro-file.html

Adding the original repro-file so that the fix can be verified also with it.

Comment 2

[Atte Kettunen]

FWIW, I tried several runs with the original repro-file. 

1) "Crash Annotation GraphicsCriticalErro" doesn't appear on every crash
2) First common frame between different runs is mozilla::layers::ClientPaintedLayer::RenderLayerWithReadback  from there on the stack is pretty much the same.

Comment 3

[Lee Salzman [:lsalzman]]

This is caused by our usage of Shmem, most prominently in E10s. We allocate Shmems that are overcommitted to the file system. When we finally try to write to enough of it, the Shmem can possibly fill up the entire /dev/shm file system area, in which case a SIGBUS gets signaled when no more space can be allocated for to back the memory page we are writing to.

So, this is just a shmem/file system out-of-memory issue. It is not actually related to Skia at all.

I am looking into a possible "workaround" to signal the OOM at a more convenient place rather than at random places in our code when writing to shmems.

Comment 4

[Lee Salzman [:lsalzman]]

Jeff Muizelaar had previously been working on a fix to preallocate shmems in the file system, so that we don't display overcommit behavior anymore. This was in bug 1245239.

The problem is, however, that we seemingly actually depend on this overcommit behavior to work around the small size of our /dev/shm shared memory file system on our test machines. So we are seemingly only using part of the shmems that we are allocating.

I am not sure there is a nice workaround for this. We may be stuck with the SIGBUS on OOM behavior. It's not particularly harmful, though, just annoying.

Comment 5

[Lee Salzman [:lsalzman]]

Marking this as low priority/P5 for now. If someone has a reason why this needs to be resolved more urgently, we can discuss and possibly pick it back up then.

Comment 6

[Daniel Veditz [:dveditz]]

Is this a dupe of bug 1245239 then, or at least a specific demonstration of it?

We don't need to hide OOM bugs.

Comment 7

[Lee Salzman [:lsalzman]]

This would be a demonstration of bug 1245239. Not sure whether we should dup it, as this is about a specific test-case for that demonstration...

Comment 8

[Jeff Muizelaar [:jrmuizel]]

The best way out of this may be to wait for kernel upgrades to the test machines. Once they are using new enough kernels we can switch to memfd_create which will not have this problem.

Comment 9

[Darkspirit]

(In reply to Atte Kettunen from comment #0)
> Created attachment 8794598 [details]
> minimized-repro-file.hml
> 
> The repro-file may also cause the computer to go unresponsive.

With webrender+webrendest, I my computer freezes completely after watching long youtube videos or a long sesion, I opened bug 1377120 for it. Now, I only have webrender enabled and got bp-55b71758-8497-4bdf-8440-7493b0170701, can't determine if it's and IPC, Decoder, gfx or webrender problem.

But I found this bug.

I just wanted to let you know that your minimized-repo-file.html (attachment 8794598 [details]) still works and I have GBs of free memory, swap and ssd space.

Nightly 56 Build 20170630100234 @ Debian Testing x64 (Radeon RX 480, Linux 4.9.0-3-amd64, all updates installed).
Attachment 8794598 [details] reliably causes a tab crash with [@ sk_memset32 ], bp-67618efe-cb53-4d51-ad11-029480170701 for example.

Comment 10

[Darkspirit]

Supplement: I opened about 5 tabs with that attachment, saw some letters and my desktop freezed. My web radio stream in VLC repeated in a loop. I only got one bp-3d3b75f9-69e1-4eca-b308-82fa70170701. After 20 seconds I pressed the reset button.
(In bug 1377120 my music normally keeps playing while the desktop, except the mouse, is frozen. So this bug here seems to be a different one than mine.)

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.