1341354 - error_get_my_stack leaks from WebRTC calling NSSTrustDomain_Create

Status: NEW

(NSS :: Libraries, defect, P3)

Description

[Andrew McCreight [:mccr8]]

Bug 1187421 fixed a number of NSS error_get_my_stack leaks, but there are still some remaining LSan leaks, in mda2 and mda3, coming from when peer connection initializes NSS:

Direct leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x4b2f84 in calloc /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:66:3
    #1 0x7f56d04eff21 in error_get_my_stack /home/worker/workspace/build/src/security/nss/lib/base/error.c:110:17
    #2 0x7f56d04f0278 in nss_ClearErrorStack /home/worker/workspace/build/src/security/nss/lib/base/error.c:246:23
    #3 0x7f56d04eea28 in NSSArena_Create /home/worker/workspace/build/src/security/nss/lib/base/arena.c:329:5
    #4 0x7f56d0512d4e in NSSTrustDomain_Create /home/worker/workspace/build/src/security/nss/lib/pki/trustdomain.c:33:13
    #5 0x7f56d0503b69 in STAN_LoadDefaultNSS3TrustDomain /home/worker/workspace/build/src/security/nss/lib/pki/pki3hack.c:117:10
    #6 0x7f56d04ebd20 in nss_Init /home/worker/workspace/build/src/security/nss/lib/nss/nssinit.c:705:13
    #7 0x7f56d04ecf7a in NSS_NoDB_Init /home/worker/workspace/build/src/security/nss/lib/nss/nssinit.c:950:12
    #8 0x7f56c52ae460 in InitNSSInContent /home/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:202:7
    #9 0x7f56c52ae460 in mozilla::PeerConnectionImpl::Initialize(mozilla::dom::PeerConnectionObserver&, nsGlobalWindow*, mozilla::PeerConnectionConfiguration const&, nsISupports*) /home/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:596
    #10 0x7f56c52b2e75 in mozilla::PeerConnectionImpl::Initialize(mozilla::dom::PeerConnectionObserver&, nsGlobalWindow&, mozilla::dom::RTCConfiguration const&, nsISupports*, mozilla::ErrorResult&) /home/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:732:9
    #11 0x7f56be8aa2d0 in mozilla::dom::PeerConnectionImplBinding::initialize(JSContext*, JS::Handle<JSObject*>, mozilla::PeerConnectionImpl*, JSJitMethodCallArgs const&) /home/worker/workspace/build/src/obj-firefox/dom/bindings/PeerConnectionImplBinding.cpp:81:3

If this can't be easily fixed, it would be good to change the LSan whitelist to maybe NSSTrustDomain_Create so that it is more specific.

I don't know if this is the fault of WebRTC or NSS.

Here's the try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d177f0889c53&selectedJob=78831451

Comment 1

[Andrew McCreight [:mccr8]]

There are three instances of this leak. Two are in dom/media/tests/mochitest/ (I guess this directory is split across two runs of MDA?) and one is in dom/media/tests/mochitest/identity/. The latter only has a few tests so it would be the place to start looking.