845088 - Assertion failure and crash on shutdown: ((bool)(__builtin_expect(!!(!NS_FAILED_impl(rv)), 1))), at /media/webrtc/signaling/../../../media/mtransport/runnable_utils.h:48 [@ mozilla::MediaPipeline::PipelineTransport::SendRtcpPacket]

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Signaling, defect, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The assertion as logged on bug 835851 we also hit with another stack as given below. I can see this while running our Mochitests with the patch from bug 837458 applied. I haven't tested yet without it. It's enough to run the basic audio peerconnection test and we crash on shutdown:

Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x0

Thread 19 (crashed)
0  XUL!mozilla::RUN_ON_THREAD [runnable_utils.h:8c92d4651a44 : 48 + 0x0]
1  XUL!mozilla::MediaPipeline::PipelineTransport::SendRtcpPacket(void const*, int) [MediaPipeline.cpp:8c92d4651a44 : 576 + 0x11]
rip = 0x000000010306910c   rsp = 0x0000000126fa03c0
rbp = 0x0000000126fa03f0
Found by: stack scanning
2  XUL!mozilla::WebrtcAudioConduit::SendRTCPPacket(int, void const*, int) [AudioConduit.cpp:8c92d4651a44 : 672 + 0xa]
rip = 0x0000000103035c02   rsp = 0x0000000126fa0400
rbp = 0x0000000126fa0430
Found by: stack scanning
3  XUL!webrtc::voe::Channel::SendRTCPPacket(int, void const*, int) [channel.cc:8c92d4651a44 : 340 + 0xe]
rip = 0x0000000102fa2a66   rsp = 0x0000000126fa0440
rbp = 0x0000000126fa0480
Found by: stack scanning
4  XUL!webrtc::RTCPSender::SendRTCP(unsigned int, int, unsigned short const*, bool, unsigned long long) [rtcp_sender.cc:8c92d4651a44 : 1926 + 0x15]
rip = 0x0000000102f7dc33   rsp = 0x0000000126fa0490
rbp = 0x0000000126fa0b20
Found by: stack scanning
5  XUL!cat6_prob + 0x111a1
rip = 0x0000000103884f50   rsp = 0x0000000126fa0560
rbp = 0x0000000126fa0b20
Found by: stack scanning
6  XUL!cat6_prob + 0x11518

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Same crash happens without the patch based on 122888:06935f2db267.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This reliable crashes for me on shutdown. Something in the last two weeks should have regressed this.

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

On linux (debug) this doesn't crash for me.  Also, how are you running it?  I ran it as part of the full test, and directly by name (but then it doesn't shut down on it's own, you can to close the window)>

Comment 4

[Randell Jesup [:jesup] (needinfo me)]

Note: this is NOT bug 835851.  They're both assertion failures (love those signatures!), but in very different places.

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

This is      rv = thread->IsOnCurrentThread(&on); MOZ_ASSERT(NS_SUCCEEDED(rv));

That can only reasonable happen if the STS is shut down:
    nsCOMPtr<nsIThread> thread = GetThreadSafely();
    NS_ENSURE_TRUE(thread, NS_ERROR_NOT_INITIALIZED);

Which means RTCP is being sent after shutdown has gotten a fair ways.

Whimboo: can you get an NSPR log (debug build) and a real crash stack (gdb)?

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

Very likely the same cause as bug 844710 (which shows in this log that RTCP didn't shut down after basicAudio ran):

https://tbpl.mozilla.org/php/getParsedLog.php?id=20056050&full=1&branch=mozilla-inbound

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I run the basic peerconnection tests in order only. I will get a stack for you later once I have finished the update of the pc framework for mochitests.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I'm not able to reproduce it in the network I'm in right now. It might be related to my network at home. I will have to re-test tomorrow morning. If I still see the crash I will hand you the full log.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I tried to reproduce this bug a couple of times with a variation of tests but I don't see it anymore.

Comment 10

[Maire Reavy [:mreavy]]

Per Comment 9, I'm closing this.  Please reopen if you see it again.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It's not fixed given that I was using the same build. I will reopen if it happens again and I can provide more information.