1625449 - nsDOMDataChannel::Send can hang content processes

Status: NEW

(Core :: WebRTC: Networking, defect, P3)

Description

[Florian Quèze [:florian]]

Here's a profile where nsDOMDataChannel::Send blocked the main thread of a content process for 251s (which is probably the time it took me to capture the profile after the process stopped responding) https://perfht.ml/2UsHp21

I had to kill the content process.

This happened while watching a video on https://www.6play.fr/, on a Windows machine on Firefox release. I had seen a similar hang twice on my nightly on Mac while watching a video on https://www.france.tv/france-2/

I don't know why these video playing websites need a datachannel.

The stack visible in the profile is:

SleepConditionVariableCS
sctp_os_timer_stop(sctp_callout*)
sctp_chunk_output(sctp_inpcb*, sctp_tcb*, int, int)
sctp_lower_sosend(socket*, sockaddr*, uio*, mbuf*, mbuf*, int, sctp_sndrcvinfo*)
usrsctp_sendv(socket*, void const*, unsigned long long, sockaddr*, int, void*, int, unsigned int, int)
mozilla::DataChannelConnection::SendMsgInternal(mozilla::OutgoingMsg&, unsigned long long*)
mozilla::DataChannelConnection::SendMsgInternalOrBuffer(nsTArray<nsAutoPtr<mozilla::BufferedOutgoingMsg> >&, mozilla::OutgoingMsg&, bool&, unsigned long long*)
mozilla::DataChannelConnection::SendDataMsgInternalOrBuffer(mozilla::DataChannel&, unsigned char const*, unsigned long long, unsigned int)
mozilla::DataChannelConnection::SendDataMsgCommon(unsigned short, nsTSubstring<char> const&, bool)
mozilla::DataChannel::SendBinaryMsg(nsTSubstring<char> const&, mozilla::ErrorResult&)
nsDOMDataChannel::Send(mozilla::dom::TypedArray<unsigned char,&JS::UnwrapArrayBufferMaybeShared,&JS::GetArrayBufferMaybeSharedData,&JS::GetArrayBufferMaybeSharedLengthAndData,&JS::NewArrayBuffer> const&, mozilla::ErrorResult&)
mozilla::dom::RTCDataChannel_Binding::send(JSContext*, JS::Handle<JSObject *>, void*, JSJitMethodCallArgs const&)
0x1dbabac65f6
js::RunScript
EventHandlerNonNull
EventDispatcher::Dispatch
Worker runnable
XRE_InitChildProcess
(root)

Comment 1

[Dan Minor [:dminor]]

There is what looks to be a deadlock in sctp we're looking at in Bug 1620871. There is also a deadlock under investigation upstream, see Bug 1613889. I'm not familiar enough with the sctp code to say whether these are all the same problem or not.

Comment 2

[Byron Campen [:bwc]]

This is the stack from here: https://bugzilla.mozilla.org/show_bug.cgi?id=1620871#c36

Any idea what might be happening here?

Comment 3

[Lennart Grahl [:lgrahl]]

This looks weird. Any ideas, Michael?

Comment 4

[Michael Tüxen]

There was a use-after free bug related to the timer code. This was reported by Mozilla. Peter provided a fix for it and the corresponding crash was gone, as reported by Mozilla. However, Google reported, that the fix introduced a deadlock. This is being worked on in https://github.com/sctplab/usrsctp/pull/417. So I guess that this is just an incarnation of the deadlock Google was finding...

Is there a way to reproduce this issue?

Comment 5

[Lennart Grahl [:lgrahl]]

I haven't encountered this in more than a year and it potentially depends on how the data channels are being used. Perhaps the reporter knows of a way to reproduce.

Comment 6

[Florian Quèze [:florian]]

(In reply to Lennart Grahl [:lgrahl] from comment #5)

I haven't encountered this in more than a year and it potentially depends on how the data channels are being used. Perhaps the reporter knows of a way to reproduce.

I don't know anything more than: "This happened while watching a video on https://www.6play.fr/, on a Windows machine on Firefox release. I had seen a similar hang twice on my nightly on Mac while watching a video on https://www.france.tv/france-2/", sorry.

Comment 7

[Florian Quèze [:florian]]

I saw this content process hang again yesterday on a Nightly from June 5th, on OS X: https://share.firefox.dev/37YxRlr