1267752 - crash in shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | nsThread::ProcessNextEvent | NS_ProcessNextEvent | mozilla::MediaShutdownManager::Shutdown

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P1)

Description

[Marco Castelluccio [:marco]]

This bug was filed from the Socorro interface and is 
report bp-11565cd6-fd26-4e5f-bfa6-b7f5e2160425.
=============================================================

Filing this bug because there is no associated report to that signature yet.

This shutdown hang is #26 top crasher (with 2696 crashes) on Firefox 45 and #32 top crasher (with 508 crashes) on Firefox 46.

There are a few reports for Firefox 47 (6), no reports for Firefox 48.

Looking at the report comments, looks like this is often associated with YouTube or videos on Facebook.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

Chris: (from the metadata:) AsyncPluginShutdownStates 	

{clearkey:{11931400:"S=CloseActive"},Primetime Content Decryption Module provided by Adobe Systems, Incorporated:{b9dd800:"S=CloseActive"},gmpopenh264:{c519800:"S=CloseActive"},-:{-:"012345=Async shutdown complete"}}

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Randell, the AsyncPluginShutdownStates metadata you cite shows that the GMP shutdown completed successfully; so it's unlikely that it's the thing blocking the overall media shutdown.
(But I admit I'm not familiar with MediaShutdownManager, so it's possible I'm missing something.)

If we cannot easily reproduce the issue and find the cause of these hangs, we could implement a similar scheme as with GMP shutdown, to track a finer-grained progress of MediaDecoders shutdown.

Comment 3

[JW Wang [:jwwang] [:jw_wang]]

The crash stacks are quite different from each other. We should file individual bugs to trace them respectively.

Comment 4

[JW Wang [:jwwang] [:jw_wang]]

https://crash-stats.mozilla.com/report/index/621c109e-ea9b-4d66-a979-cafec2160420
Bug 1267930 filed.

Comment 5

[JW Wang [:jwwang] [:jw_wang]]

https://crash-stats.mozilla.com/report/index/936734a4-0018-499f-b9f0-e4d012160426
Bug 1267931 filed.

Comment 6

[Chris Pearce [:cpearce (Not reading bugmail)]]

jwwang is looking into these.

Comment 7

[Alastor Wu [:alwu]]

I think these crashes [1] are related with bug1135562.
They're both stuck when the audioses.dll was calling into rpcrt4.dll.

[1] 
https://crash-stats.mozilla.com/report/index/b1460f4f-b1e2-484c-9441-d6c242160520
https://crash-stats.mozilla.com/report/index/11184b96-ae08-493d-93ac-8703e2160520

Comment 8

[Alastor Wu [:alwu]]

https://crash-stats.mozilla.com/report/index/38b3fd81-44ff-4454-9a8e-1e1812160519
Bug 1274479 filed.

Comment 9

[Alastor Wu [:alwu]]

https://crash-stats.mozilla.com/report/index/f95ec9f4-b92a-45d0-b946-06a102160515
Bug 1274498 filed.

Comment 10

[Ting-Yu Chou [:ting] (away)]

Saw another one on Nightly: https://crash-stats.mozilla.com/report/index/f832a3fd-a47b-4cf2-ae78-c55552160524, thread 58 is with unresolved symbols in msmpeg2vdec.dll, not sure if this belongs to any bug you have filed.

Comment 11

[JW Wang [:jwwang] [:jw_wang]]

(In reply to Ting-Yu Chou [:ting] from comment #10)
> Saw another one on Nightly:
> https://crash-stats.mozilla.com/report/index/f832a3fd-a47b-4cf2-ae78-
> c55552160524, thread 58 is with unresolved symbols in msmpeg2vdec.dll, not
> sure if this belongs to any bug you have filed.

Maybe this is bug 1254389.

Comment 12

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

(In reply to JW Wang [:jwwang] from comment #11)
> (In reply to Ting-Yu Chou [:ting] from comment #10)
> > Saw another one on Nightly:
> > https://crash-stats.mozilla.com/report/index/f832a3fd-a47b-4cf2-ae78-
> > c55552160524, thread 58 is with unresolved symbols in msmpeg2vdec.dll, not
> > sure if this belongs to any bug you have filed.
> 
> Maybe this is bug 1254389.

The crash doesn't happen in msmpeg2vdec.dll itself, and not even in the same thread, so I don't believe it is the same issue.

And as far as I know there are no symbols available for msmpeg2[av]dec.dll, so having unresolved symbols is (unfortunately) normal.

Comment 13

[Tzuhao Kuo [:kaku]]

I revisit 50 reports happened at 5/25, here is the brief statistics:

Bug 1274498 : 17/50
Bug 1274513 : 17/50
Bug 1274479 :  6/50
Bug 1267930 :  4/50 (This should already be solved.)

The others are not recognizable or the hanging function is no long in the current code base.

I will update the related bugs respectively.

Comment 14

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Probably bug 1267933.

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | PR_WaitCondVar | nsThread::ProcessNextEvent | NS_ProcessNextEvent | mozilla::MediaShutdownManager::Shutdown':
 - nightly (version 51): 0 crashes from 2016-08-01.
 - aurora  (version 50): 0 crashes from 2016-08-01.
 - beta    (version 49): 0 crashes from 2016-08-02.
 - release (version 48): 1109 crashes from 2016-07-25.
 - esr     (version 45): 848 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly       0       0       0
 - aurora        0       0       0
 - beta          0       0       0
 - release     241     588     280
 - esr          55     120      93

Affected platform: Windows

Crash rank on the last 7 days:
             Browser   Content   Plugin
 - nightly
 - aurora
 - beta
 - release
 - esr