1061693 - Among sessions attempting D3D11 layers, graphics-related crash signatures are abnormally prevalent

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect)

Description

[Benoit Jacob [:bjacob] (mostly away)]

Graphics-related crash signatures aren't normally so prevalent, but if we filter for users who at least attempt D3D11 layers (that is, at the moment, unreleased Firefox versions on Windows), then the prevalence of graphics-related signatures becomes much higher, which indicates unusual graphics-related crashiness there.

Here are the top 50 crash signatures at the moment. Will file bugs, blocking this one, for some of the graphicsy ones.

zcat 20140901-pub-crashdata.csv.gz | grep D3D11 | cut -f 1 -d '  ' | sort | uniq -c | sort -rn | head -n 50
    976 OOM | small
    148 mozilla::layers::CompositorD3D11::DrawQuad(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::EffectChain const&, float, mozilla::gfx::Matrix4x4 const&)
    128 mozalloc_abort(char const* const) | NS_DebugBreak | nsDebugImpl::Abort(char const*, int)
     97 F1398665248_____________________________
     64 IUnknown::QueryInterface<IDXGISurface>(IDXGISurface**)
     40 mozilla::layers::BasicCompositor::BeginFrame(nsIntRegion const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const*, mozilla::gfx::Matrix const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::RectTyped<mozilla::gfx::...
     29 EnterBaseline
     26 mozilla::layers::ImageContainer::CreateImage(mozilla::ImageFormat)
     25 _VEC_memcpy | mozilla::net::CacheFileChunk::OnDataRead(mozilla::net::CacheFileHandle*, char*, tag_nsresult)
     20 js::types::ConstraintTypeSet::sweep(JS::Zone*, bool*)
     19 mozilla::net::nsHttpTransaction::OnTransportStatus(nsITransport*, tag_nsresult, unsigned __int64)
     19 CDevice::ID3D10Device1_CopyResource_(ID3D10Device1*, ID3D10Resource*, ID3D10Resource*)
     19 abort | abort | __android_log_assert | stagefright::MPEG4Source::start(stagefright::MetaData*)
     17 CInputContext::GetSelection(unsigned long, unsigned long, unsigned long, TF_SELECTION*, unsigned long*)
     16 OOM | unknown | js::CrashAtUnhandlableOOM(char const*) | js::Nursery::moveToTenured(js::gc::MinorCollectionTracer*, JSObject*)
     15 CInputContext::_NotifyEndEdit()
     14 ScanTypeObject
     14 mozilla::net::nsHttpChannel::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult)
     14 mozilla::net::Http2Session::CleanupStream(mozilla::net::Http2Stream*, tag_nsresult, mozilla::net::Http2Session::errorType)
     12 _moz_cairo_win32_surface_create_with_d3dsurface9
     12 F_835252764_________________________________________________________
     12 CPrivAlloc::operator delete(void*)
     12 abort
     11 OOM | large | mozalloc_abort(char const* const) | mozalloc_handle_oom(unsigned int) | moz_xmalloc | gfxAlphaBoxBlur::Init(gfxRect const&, nsIntSize const&, nsIntSize const&, gfxRect const*, gfxRect const*)
     11 nsWindow::SetCursor(nsCursor)
     11 mozilla::gfx::DrawTargetDual::DrawTargetDual(mozilla::gfx::DrawTarget*, mozilla::gfx::DrawTarget*)
     11 mozilla::dom::Console::Console(nsPIDOMWindow*)
     11 mozalloc_abort(char const* const) | NS_DebugBreak | mozilla::ipc::MessageChannel::DebugAbort(char const*, int, char const*, char const*, bool) | mozilla::ipc::MessageChannel::Send(IPC::Message*, IPC::Message*)
     11 mfmp4srcsnk.dll@0x7e22f
     11 js::RemapWrapper(JSContext*, JSObject*, JSObject*)
     11 CCliModalLoop::CCliModalLoop(unsigned long, unsigned long, unsigned long, int)
     10 PushMarkStack
     10 js::gc::GCRuntime::beginMarkPhase(JS::gcreason::Reason)
     10 GraphWalker<ScanBlackVisitor>::DoWalk(nsDeque&)
      9 ScanShape
      9 js::GCMarker::processMarkStackTop(js::SliceBudget&)
      9 EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER
      9 @0x0 | aticfx32.dll@0x10c99
      8 nsXPCWrappedJS::Release()
      8 nsTypeAheadFind::GetSearchContainers(nsISupports*, nsISelectionController*, bool, bool, nsIPresShell**, nsPresContext**)
      8 nsFrame::DestroyFrom(nsIFrame*)
      8 nsEditorSpellCheck::nsEditorSpellCheck()
      8 mozilla::net::nsHttpChannel::OnDoneReadingPartialCacheEntry(bool*)
      8 mozilla::gfx::DrawTargetDual::SetTransform(mozilla::gfx::Matrix const&)
      8 MarkInternal<js::PropertyName>
      8 js::UncheckedUnwrap(JSObject*, bool, unsigned int*)
      8 HASH_Update
      8 arena_dalloc | js::ctypes::CData::Finalize
      8 @0x0 | aticfx32.dll@0x10d09
      7 nsView::Destroy()

Comment 1

[Robert Kaiser]

A number of those show up in the top 10 overall topcrashes in 33+ (i.e. where OMTC for Windows is enabled).

Comment 2

[Bas Schouten (:bas.schouten)]

Just for the record, since we never blacklist D3D11 (yet). The 'users that have at least tried d3d11', should be 'everyone on Windows7+' :)

Comment 3

[Benoit Jacob [:bjacob] (mostly away)]

... and on pre-release channels. yeah. What we're seeing here is that graphics is now a prevalent source of crashiness, and it didn't use to be; looking at specific crashes, roughly half of them seem directly related to OMTC and/or D3D11.

Comment 4

[Robert Kaiser]

We know that OMTC on Windows is a major source of instability on 33 and higher still (on 32 Beta and now release I think we're good, we disabled OMTC there while still on Aurora).
We will be monitoring this very closely once the first 33 Beta is out and we might need to disable OMTC mid-beta-cycle if no improvements are made within the next ~2 weeks.

Comment 5

[Milan Sreckovic [:milan] (needinfo for best results)]

Yes, that sounds like the right approach.  As Bas points out, we now have a much larger audience hitting DX11 than we had hitting DX10, so chances are there will be some combination of bug fixes and blacklisting that would need to happen in the next couple of weeks.

Comment 6

[Tracy Walker [:tracy]]

Asking here instead of in all the blocking bugs. Is any progress being made toward improving stability of OMTC such that we'd be able to keep OMTC enabled?

Comment 7

[Nicolas Silva [:nical]]

(In reply to [:tracy] Tracy Walker - QA Mentor from comment #6)
> Asking here instead of in all the blocking bugs. Is any progress being made
> toward improving stability of OMTC such that we'd be able to keep OMTC
> enabled?

Yes. Bas and I are looking at them, most likely a good chunk of the issues will be going into the driver blacklist.

Comment 8

[Sylvestre Ledru [:Sylvestre]]

Nical, can you share some info wrt the driver blacklist? Ready or not, ETA, etc.
Thanks

Comment 9

[Nicolas Silva [:nical]]

While we expected to have to blacklist things, most of the "mysterious" bugs have been solved now, and we don't soo driver-specific issues (except intel being memory hungry, but we can't blacklist all of the intel gpus).

Comment 10

[Benjamin Smedberg]

If we discover that all the Intel drivers have VM hogging issues that we've discovered in the past, we could certainly consider blacklisting all of them. In any case are you saying that there is nothing more to track in this bug, or that there is still work that we need to accomplish for FF33 to blacklist some cards?

Comment 11

[Nicolas Silva [:nical]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #10)
> If we discover that all the Intel drivers have VM hogging issues that we've
> discovered in the past, we could certainly consider blacklisting all of
> them.

Intel drivers tend to consume twice the amount of texture memory they should which is sad, especially now that we are double buffering these textures in places where we used to have single buffering, but we should not blacklist all intel drivers over that, at least not now. There are other ways to get OOM crashes down to an acceptable level without having to lose the benefits of d2d (especially for that large a population of users).

> In any case are you saying that there is nothing more to track in this
> bug, or that there is still work that we need to accomplish for FF33 to
> blacklist some cards?

This bug tracks the unusual crashiness related to D3D11+OMTC so let's keep it open until we are satisfied with the stability on beta.

Comment 12

[Nicolas Silva [:nical]]

Actually, nevermind my last comment, there will actually be a little bit of blacklisting in bug 1060736, but for rendering issues rather than memory-related ones.

Comment 13

[Sylvestre Ledru [:Sylvestre]]

We are blacklisting enough. Stop tracking.