Closed Bug 1410829 Opened 2 years ago Closed 2 years ago

Assertion failure: [GFX1]: Texture deallocated too late during shutdown, at gfx/2d/Logging.h:518

Categories

(Core :: Audio/Video: MediaStreamGraph, defect, P2)

defect

Tracking

()

RESOLVED FIXED
mozilla58
Tracking Status
firefox-esr52 --- unaffected
firefox56 --- unaffected
firefox57 --- unaffected
firefox58 --- fixed

People

(Reporter: aryx, Assigned: pehrsons)

References

(Blocks 1 open bug)

Details

(Keywords: assertion, intermittent-failure, Whiteboard: [stockwell fixed:product])

Attachments

(3 files)

+++ This bug was initially created as a clone of Bug #1347625 +++

https://treeherder.mozilla.org/logviewer.html#?job_id=138816775&repo=autoland

[task 2017-10-23T08:43:13.301Z] 08:43:13     INFO - GECKO(1485) | [GFX1]: Texture deallocated too late during shutdown
[task 2017-10-23T08:43:13.302Z] 08:43:13     INFO - GECKO(1485) | Assertion failure: [GFX1]: Texture deallocated too late during shutdown, at /builds/worker/workspace/build/src/gfx/2d/Logging.h:520
[task 2017-10-23T08:43:48.002Z] 08:43:48     INFO - GECKO(1485) | #01: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::~Log [gfx/2d/Logging.h:524]
[task 2017-10-23T08:43:48.002Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.002Z] 08:43:48     INFO - GECKO(1485) | #02: mozilla::layers::DeallocateTextureClient [gfx/layers/client/TextureClient.cpp:330]
[task 2017-10-23T08:43:48.003Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.004Z] 08:43:48     INFO - GECKO(1485) | #03: mozilla::layers::TextureClient::Destroy [gfx/layers/client/TextureClient.cpp:430]
[task 2017-10-23T08:43:48.004Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.004Z] 08:43:48     INFO - GECKO(1485) | #04: mozilla::layers::TextureClient::~TextureClient [gfx/layers/client/TextureClient.cpp:610]
[task 2017-10-23T08:43:48.005Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.005Z] 08:43:48     INFO - GECKO(1485) | #05: mozilla::layers::TextureClient::~TextureClient [gfx/layers/client/TextureClient.cpp:617]
[task 2017-10-23T08:43:48.005Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.006Z] 08:43:48     INFO - GECKO(1485) | #06: mozilla::AtomicRefCountedWithFinalize<mozilla::layers::TextureClient>::Release [gfx/layers/AtomicRefCountedWithFinalize.h:144]
[task 2017-10-23T08:43:48.007Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.007Z] 08:43:48     INFO - GECKO(1485) | #07: PLDHashTable::~PLDHashTable [xpcom/ds/PLDHashTable.cpp:327]
[task 2017-10-23T08:43:48.008Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.008Z] 08:43:48     INFO - GECKO(1485) | #08: mozilla::layers::SourceSurfaceImage::~SourceSurfaceImage [xpcom/ds/nsTHashtable.h:391]
[task 2017-10-23T08:43:48.009Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.009Z] 08:43:48     INFO - GECKO(1485) | #09: mozilla::layers::SourceSurfaceImage::~SourceSurfaceImage [gfx/layers/ImageContainer.h:935]
[task 2017-10-23T08:43:48.009Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.010Z] 08:43:48     INFO - GECKO(1485) | #10: mozilla::layers::Image::Release [gfx/layers/ImageContainer.h:199]
[task 2017-10-23T08:43:48.011Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.012Z] 08:43:48     INFO - GECKO(1485) | #11: mozilla::dom::OutputStreamDriver::StreamListener::~StreamListener [dom/media/CanvasCaptureMediaStream.cpp:74]
[task 2017-10-23T08:43:48.013Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.015Z] 08:43:48     INFO - GECKO(1485) | #12: mozilla::dom::OutputStreamDriver::StreamListener::~StreamListener [dom/media/CanvasCaptureMediaStream.cpp:74]
[task 2017-10-23T08:43:48.017Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.019Z] 08:43:48     INFO - GECKO(1485) | #13: mozilla::MediaStreamListener::Release [dom/media/MediaStreamListener.h:48]
[task 2017-10-23T08:43:48.020Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.021Z] 08:43:48     INFO - GECKO(1485) | #14: mozilla::MediaStream::RemoveAllListenersImpl [dom/media/MediaStreamGraph.cpp:2077]
[task 2017-10-23T08:43:48.023Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.024Z] 08:43:48     INFO - GECKO(1485) | #15: mozilla::MediaStream::Destroy()::Message::Run()
[task 2017-10-23T08:43:48.025Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.026Z] 08:43:48     INFO - GECKO(1485) | #16: mozilla::MediaStreamGraphImpl::AppendMessage [dom/media/MediaStreamGraph.cpp:1881]
[task 2017-10-23T08:43:48.027Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.029Z] 08:43:48     INFO - GECKO(1485) | #17: mozilla::MediaStream::Destroy [dom/media/MediaStreamGraph.cpp:2114]
[task 2017-10-23T08:43:48.030Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.031Z] 08:43:48     INFO - GECKO(1485) | #18: mozilla::DOMMediaStream::Destroy [dom/media/DOMMediaStream.cpp:498]
[task 2017-10-23T08:43:48.033Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.035Z] 08:43:48     INFO - GECKO(1485) | #19: mozilla::DOMMediaStream::cycleCollection::Unlink [dom/media/DOMMediaStream.cpp:382]
[task 2017-10-23T08:43:48.036Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.038Z] 08:43:48     INFO - GECKO(1485) | #20: mozilla::dom::CanvasCaptureMediaStream::cycleCollection::Unlink [mfbt/RefPtr.h:63]
[task 2017-10-23T08:43:48.040Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.042Z] 08:43:48     INFO - GECKO(1485) | #21: nsCycleCollector::CollectWhite [xpcom/base/nsCycleCollector.cpp:3403]
[task 2017-10-23T08:43:48.044Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.045Z] 08:43:48     INFO - GECKO(1485) | #22: nsCycleCollector::Collect [xpcom/base/nsCycleCollector.cpp:3769]
[task 2017-10-23T08:43:48.047Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.048Z] 08:43:48     INFO - GECKO(1485) | #23: nsCycleCollector::ShutdownCollect [xpcom/base/nsCycleCollector.cpp:3687]
[task 2017-10-23T08:43:48.050Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.052Z] 08:43:48     INFO - GECKO(1485) | #24: nsCycleCollector_shutdown [mfbt/RefPtr.h:63]
[task 2017-10-23T08:43:48.053Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.054Z] 08:43:48     INFO - GECKO(1485) | #25: mozilla::ShutdownXPCOM [xpcom/build/XPCOMInit.cpp:987]
[task 2017-10-23T08:43:48.056Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.057Z] 08:43:48     INFO - GECKO(1485) | #26: XRE_TermEmbedding [toolkit/xre/nsEmbedFunctions.cpp:225]
[task 2017-10-23T08:43:48.059Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.060Z] 08:43:48     INFO - GECKO(1485) | #27: mozilla::ipc::ScopedXREEmbed::Stop [ipc/glue/ScopedXREEmbed.cpp:109]
[task 2017-10-23T08:43:48.061Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.063Z] 08:43:48     INFO - GECKO(1485) | #28: XRE_InitChildProcess [toolkit/xre/nsEmbedFunctions.cpp:708]
[task 2017-10-23T08:43:48.064Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.093Z] 08:43:48     INFO - GECKO(1485) | #29: content_process_main [ipc/contentproc/plugin-container.cpp:65]
[task 2017-10-23T08:43:48.094Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.095Z] 08:43:48     INFO - GECKO(1485) | #30: main [browser/app/nsBrowserApp.cpp:283]
[task 2017-10-23T08:43:48.095Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.096Z] 08:43:48     INFO - GECKO(1485) | #31: libc.so.6 + 0x20830
[task 2017-10-23T08:43:48.096Z] 08:43:48     INFO - 
[task 2017-10-23T08:43:48.097Z] 08:43:48     INFO - GECKO(1485) | #32: _start
(In reply to Sebastian Hengst [:aryx][:archaeopteryx] (needinfo on intermittent or backout) from comment #0)
> [task 2017-10-23T08:43:48.038Z] 08:43:48     INFO - GECKO(1485) | #20:
> mozilla::dom::CanvasCaptureMediaStream::cycleCollection::Unlink
> [mfbt/RefPtr.h:63]

My hunch is this is bug 1407542.
Depends on: 1407542
It's related in the sense that they're both about cycle collecting DOMMediaStreams, yes, but I don't think bug 1407542 will be a definite fix for this. Other classes can hold references to DOMMediaStream (though I admit it's unclear if there would be cycles from those).

This has probably been rare because it would normally get cycle collected before reaching shutdown. Maybe the chunking on try changed.

We could have MediaStreamGraph signal all MediaStreamListeners that shutdown is occurring. MSG should go through shutdown, [1], before graphics, [2], which goes through shutdown before the cycle collector, [3]. Destructing CanvasCaptureMediaStream's OutputStreamDriver::mImage during unlink (so [3] when this fails) is what causes issues.


One day we get rid of VideoChunk durations and push straight to the SourceMediaStream (i.e., get rid of that mImage), and this will be a non-problem.

[1] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/xpcom/build/XPCOMInit.cpp#876
[2] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/xpcom/build/XPCOMInit.cpp#885
[3] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/xpcom/build/XPCOMInit.cpp#985
Assignee: nobody → apehrson
Component: Canvas: WebGL → Audio/Video: MediaStreamGraph
No longer depends on: 1407542
Rank: 19
Priority: -- → P2
Hmm, The MediaStreamListener is notified of removal from the graph *after* the graph has cleared its shutdown blocker. That sounds wrong.
(In reply to Andreas Pehrson [:pehrsons] from comment #3)
> Hmm, The MediaStreamListener is notified of removal from the graph *after*
> the graph has cleared its shutdown blocker. That sounds wrong.

Because it was only removed on stream destruction (so on cyclecollector shutdown in the worst case). I'm fixing that.
Comment on attachment 8922761 [details]
Bug 1410829 - Signal listeners removed on shutdown so they can clean up.

https://reviewboard.mozilla.org/r/193866/#review199090

::: dom/media/MediaStreamGraph.cpp:2098
(Diff revision 1)
> +  }
> +  mTrackListeners.Clear();
> +
> +  if (SourceMediaStream* source = AsSourceStream()) {
> +    source->RemoveAllDirectListeners();
> +  }

Can you update the commit message to explain this? I think I understand, but it's a bit late, so I kind of doubt I'm getting things right.

Are we just being explicit here to signal to all the  track, direct and stream (that was always the case) listeners that they are being removed, so they can clear their gfx resources references?

In the past, we used the fact that the different object's dtor were called, so we wouldn't leak, but we also wouln't release the texture properly, right?
Attachment #8922761 - Flags: review?(padenot)
Comment on attachment 8922762 [details]
Bug 1410829 - Clean up streams and listeners before unblocking shutdown.

https://reviewboard.mozilla.org/r/193868/#review199094
Attachment #8922762 - Flags: review?(padenot) → review+
Comment on attachment 8922761 [details]
Bug 1410829 - Signal listeners removed on shutdown so they can clean up.

https://reviewboard.mozilla.org/r/193866/#review199096

This looks like it's correct, but see below. r+ with a longer/clearer commit message if I understood it right, r- otherwise.
Attachment #8922761 - Flags: review+
Could you elaborate on "In the past, we used the fact that the different object's dtor were called"? E.g., what would "different object" be?

What we now get is the shutdown sequence: [media, gfx, cycleCollector]

dtors are called on cycleCollector shutdown when this fails, because that releases images after gfx shutdown and we hit this hard assert. We need to move this cleanup to media shutdown, so that's basically what the patches do.
Flags: needinfo?(padenot)
Whiteboard: [stockwell needswork]
Comment on attachment 8922763 [details]
Bug 1410829 - Clear out mImage on media shutdown.

https://reviewboard.mozilla.org/r/193870/#review200854

lgtm.
Attachment #8922763 - Flags: review?(jib) → review+
Pushed by pehrsons@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/db0fa983b90f
Signal listeners removed on shutdown so they can clean up. r=padenot
https://hg.mozilla.org/integration/autoland/rev/d9257f9b2296
Clean up streams and listeners before unblocking shutdown. r=padenot
https://hg.mozilla.org/integration/autoland/rev/0253bbb60b2f
Clear out mImage on media shutdown. r=jib
Clearing NI.
Flags: needinfo?(padenot)
Whiteboard: [stockwell needswork] → [stockwell fixed:product]
Depends on: 1357466
Depends on: 1415556
Regressions: 1583463
You need to log in before you can comment on or make changes to this bug.