Closed Bug 1293457 Opened 5 years ago Closed 3 years ago

Intermittent WebRtcIceGatherTest.TestStunTcpServerTrickle | Value of: res

Categories

(Core :: WebRTC: Networking, defect, P5)

defect

Tracking

()

RESOLVED INCOMPLETE
Tracking Status
firefox50 --- unaffected
firefox51 --- affected

People

(Reporter: intermittent-bug-filer, Assigned: drno)

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Rank: 25
Component: WebRTC → WebRTC: Networking
Priority: -- → P2
Neat: a SEGV on shutdown, because we try to fire a gathering callback on shutdown

20:24:20     INFO -  ASAN:DEADLYSIGNAL
20:24:20     INFO -  =================================================================
20:24:20     INFO -  ==1138==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7f970fc51808 bp 0x7f9705922130 sp 0x7f9705921fe0 T6)
20:24:21     INFO -      #0 0x7f970fc51807 in RefPtr /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:91:25
20:24:21     INFO -      #1 0x7f970fc51807 in ctx /home/worker/workspace/build/src/media/mtransport/nricectxhandler.h:29
20:24:21     INFO -      #2 0x7f970fc51807 in (anonymous namespace)::IceTestPeer::GatheringStateChange(mozilla::NrIceCtx*, mozilla::NrIceCtx::GatheringState) /home/worker/workspace/build/src/media/mtransport/test/ice_unittest.cpp:1016
20:24:21     INFO -      #3 0x7f9711cfce4b in operator() /home/worker/workspace/build/src/media/mtransport/sigslot.h:2420:5
20:24:21     INFO -      #4 0x7f9711cfce4b in mozilla::NrIceCtx::SetGatheringState(mozilla::NrIceCtx::GatheringState) /home/worker/workspace/build/src/media/mtransport/nricectx.cpp:989
20:24:21     INFO -      #5 0x7f9717cfe12c in nr_ice_gather_finished_cb /home/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_ctx.c:550:9
20:24:21     INFO -      #6 0x7f9717ce84b9 in nr_ice_candidate_destroy /home/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_candidate.c:316:7
20:24:21     INFO -      #7 0x7f9717cf0a95 in nr_ice_component_destroy /home/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_component.c:162:7
20:24:21     INFO -      #8 0x7f9717d000e7 in nr_ice_media_stream_destroy /home/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_media_stream.c:102:7
20:24:21     INFO -      #9 0x7f9717cff0bf in nr_ice_remove_media_stream /home/worker/workspace/build/src/media/mtransport/third_party/nICEr/src/ice/ice_ctx.c:796:10
20:24:21     INFO -      #10 0x7f9711d083ef in mozilla::NrIceMediaStream::Close() /home/worker/workspace/build/src/media/mtransport/nricemediastream.cpp:602:13
20:24:21     INFO -      #11 0x7f9711cf954a in mozilla::NrIceCtx::~NrIceCtx() /home/worker/workspace/build/src/media/mtransport/nricectx.cpp:676:7
20:24:21     INFO -      #12 0x7f9711cf9bed in mozilla::NrIceCtx::~NrIceCtx() /home/worker/workspace/build/src/media/mtransport/nricectx.cpp:672:23
20:24:21     INFO -      #13 0x7f970fc54a74 in Release /home/worker/workspace/build/src/media/mtransport/nricectx.h:333:3
20:24:21     INFO -      #14 0x7f970fc54a74 in Release /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:40
20:24:21     INFO -      #15 0x7f970fc54a74 in Release /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:387
20:24:21     INFO -      #16 0x7f970fc54a74 in ~RefPtr /home/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:78
20:24:21     INFO -      #17 0x7f970fc54a74 in ~NrIceCtxHandler /home/worker/workspace/build/src/media/mtransport/nricectxhandler.h:44
Looks like we could set the |done_cb| in nr_ice_candidate_destroy(). That should avoid the SEGV. But I'm not sure if that would not have other side effects.

The actual test failure looks like a simple gathering timeout. Probably around the TCP server reflexive gathering. But no idea why.
(In reply to Nils Ohlmeier [:drno] from comment #2)
> The actual test failure looks like a simple gathering timeout. Probably
> around the TCP server reflexive gathering. But no idea why.

Apparently the reason is that Twillio's STUN server no longer responds to TCP connections.
Comment on attachment 8783419 [details]
Bug 1293457: stop printing gathering results on shutdown.

https://reviewboard.mozilla.org/r/73216/#review71114

::: media/mtransport/test/ice_unittest.cpp:1009
(Diff revision 1)
>    }
>  
>    // Handle events
>    void GatheringStateChange(NrIceCtx* ctx,
>                              NrIceCtx::GatheringState state) {
> -    (void)ctx;
> +    if (shutting_down_) {

Why remote this line?
Comment on attachment 8783419 [details]
Bug 1293457: stop printing gathering results on shutdown.

https://reviewboard.mozilla.org/r/73216/#review71118

One small question, otherwise it looks good to me.
Attachment #8783419 - Flags: review?(mfroman) → review+
Comment on attachment 8783419 [details]
Bug 1293457: stop printing gathering results on shutdown.

https://reviewboard.mozilla.org/r/73216/#review71114

> Why remote this line?

Because it compiles without it :-)
Looks like a left over from old times to me. So I thought I would clean it up...
Pushed by drno@ohlmeier.org:
https://hg.mozilla.org/integration/autoland/rev/389bcc7eb849
stop printing gathering results on shutdown. r=mjf
https://hg.mozilla.org/mozilla-central/rev/389bcc7eb849
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla51
This is still happening.
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1293457
Assignee: nobody → drno
Status: RESOLVED → REOPENED
Flags: needinfo?(drno)
Resolution: FIXED → ---
Target Milestone: mozilla51 → ---
Okay, so the first patch only fixed the seg fault on shutdown, which was triggered by the actual intermittent: for some unknown reason gathering sometimes does not finish with the fake STUN server an TCP.
Flags: needinfo?(drno)
Mass change P2->P3 to align with new Mozilla triage process.
Priority: P2 → P3
Bulk priority update of open intermittent test failure bugs. 

P3 => P5

https://bugzilla.mozilla.org/show_bug.cgi?id=1381960
Priority: P3 → P5
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: REOPENED → RESOLVED
Closed: 5 years ago3 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.