Closed
Bug 1019934
Opened 10 years ago
Closed 10 years ago
Intermittent 408431-1.html | Exited with code 1 during test run (after "AddressSanitizer: heap-use-after-free PeerConnectionImpl.cpp:1832 IsClosed")
Categories
(Core :: WebRTC, defect)
Tracking
()
RESOLVED
WORKSFORME
Tracking | Status | |
---|---|---|
firefox30 | --- | unaffected |
firefox31 | --- | unaffected |
firefox32 | --- | fixed |
firefox33 | --- | affected |
firefox-esr24 | --- | unaffected |
People
(Reporter: RyanVM, Unassigned)
References
Details
(Keywords: crash, intermittent-failure)
Attachments
(2 files)
9.63 KB,
text/plain
|
Details | |
801 bytes,
patch
|
michal
:
review+
|
Details | Diff | Splinter Review |
This started on May 30, but was getting mis-starred under bug 1017068. https://tbpl.mozilla.org/php/getParsedLog.php?id=40980021&tree=Mozilla-Central Ubuntu ASAN VM 12.04 x64 mozilla-central opt test crashtest on 2014-06-03 13:23:28 PDT for push 298b39b50ff7 slave: tst-linux64-spot-1033 13:28:31 INFO - ==1725==ERROR: AddressSanitizer: heap-use-after-free on address 0x6130002dc890 at pc 0x7fafa762851f bp 0x7fff41bf9670 sp 0x7fff41bf9668 13:28:31 INFO - READ of size 4 at 0x6130002dc890 thread T0 13:28:31 INFO - #0 0x7fafa762851e (/builds/slave/test/build/application/firefox/libxul.so+0x24d551e) 13:28:31 INFO - #1 0x7fafa764e430 (/builds/slave/test/build/application/firefox/libxul.so+0x24fb430) 13:28:31 INFO - #2 0x7fafa6474c95 (/builds/slave/test/build/application/firefox/libxul.so+0x1321c95) 13:28:31 INFO - #3 0x7fafa6334bfa (/builds/slave/test/build/application/firefox/libxul.so+0x11e1bfa) 13:28:31 INFO - #4 0x7fafa6c81989 (/builds/slave/test/build/application/firefox/libxul.so+0x1b2e989) 13:28:31 INFO - #5 0x7fafa6c2b810 (/builds/slave/test/build/application/firefox/libxul.so+0x1ad8810) 13:28:31 INFO - #6 0x7fafa8fdd537 (/builds/slave/test/build/application/firefox/libxul.so+0x3e8a537) 13:28:31 INFO - #7 0x7fafabf03508 (/builds/slave/test/build/application/firefox/libxul.so+0x6db0508) 13:28:31 INFO - #8 0x7fafabd72403 (/builds/slave/test/build/application/firefox/libxul.so+0x6c1f403) 13:28:31 INFO - #9 0x7fafabd732e3 (/builds/slave/test/build/application/firefox/libxul.so+0x6c202e3) 13:28:31 INFO - #10 0x7fafabd7412d (/builds/slave/test/build/application/firefox/libxul.so+0x6c2112d) 13:28:31 INFO - #11 0x48a2c7 (/builds/slave/test/build/application/firefox/firefox+0x48a2c7) 13:28:31 INFO - #12 0x7fafb4f2276c (/lib/x86_64-linux-gnu/libc.so.6+0x2176c) 13:28:31 INFO - #13 0x48972c (/builds/slave/test/build/application/firefox/firefox+0x48972c) 13:28:31 INFO - 0x6130002dc890 is located 144 bytes inside of 384-byte region [0x6130002dc800,0x6130002dc980) 13:28:31 INFO - freed by thread T0 here: 13:28:31 INFO - #0 0x471b41 (/builds/slave/test/build/application/firefox/firefox+0x471b41) 13:28:31 INFO - #1 0x7fafa761890c (/builds/slave/test/build/application/firefox/libxul.so+0x24c590c) 13:28:31 INFO - #2 0x7fafa637d509 (/builds/slave/test/build/application/firefox/libxul.so+0x122a509) 13:28:31 INFO - #3 0x7fafa637e0d9 (/builds/slave/test/build/application/firefox/libxul.so+0x122b0d9) 13:28:31 INFO - #4 0x7fafa637c1c9 (/builds/slave/test/build/application/firefox/libxul.so+0x12291c9) 13:28:31 INFO - previously allocated by thread T0 here: 13:28:31 INFO - #0 0x471d41 (/builds/slave/test/build/application/firefox/firefox+0x471d41) 13:28:31 INFO - #1 0x7fafb109cbed (/builds/slave/test/build/application/firefox/libmozalloc.so+0x1bed) 13:28:31 INFO - #2 0x7fafa76403e9 (/builds/slave/test/build/application/firefox/libxul.so+0x24ed3e9) 13:28:31 INFO - #3 0x7fafa871a5a4 (/builds/slave/test/build/application/firefox/libxul.so+0x35c75a4) 13:28:31 INFO - #4 0x7fafa921c538 (/builds/slave/test/build/application/firefox/libxul.so+0x40c9538) 13:28:31 INFO - SUMMARY: AddressSanitizer: heap-use-after-free ??:0 ?? 13:28:31 INFO - Shadow bytes around the buggy address: 13:28:31 INFO - 0x0c26800538c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c26800538d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c26800538e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c26800538f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c2680053900: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 13:28:31 INFO - =>0x0c2680053910: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd 13:28:31 INFO - 0x0c2680053920: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 13:28:31 INFO - 0x0c2680053930: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c2680053940: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c2680053950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - 0x0c2680053960: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 13:28:31 INFO - Shadow byte legend (one shadow byte represents 8 application bytes): 13:28:31 INFO - Addressable: 00 13:28:31 INFO - Partially addressable: 01 02 03 04 05 06 07 13:28:31 INFO - Heap left redzone: fa 13:28:31 INFO - Heap right redzone: fb 13:28:31 INFO - Freed heap region: fd 13:28:31 INFO - Stack left redzone: f1 13:28:31 INFO - Stack mid redzone: f2 13:28:31 INFO - Stack right redzone: f3 13:28:31 INFO - Stack partial redzone: f4 13:28:31 INFO - Stack after return: f5 13:28:31 INFO - Stack use after scope: f8 13:28:31 INFO - Global redzone: f9 13:28:31 INFO - Global init order: f6 13:28:31 INFO - Poisoned by user: f7 13:28:31 INFO - Contiguous container OOB:fc 13:28:31 INFO - ASan internal: fe 13:28:31 INFO - ==1725==ABORTING 13:28:32 INFO - TEST-INFO | Main app process: killed by SIGHUP 13:28:32 WARNING - TEST-UNEXPECTED-FAIL | file:///builds/slave/test/build/tests/reftest/tests/dom/src/offline/crashtests/408431-1.html | Exited with code 1 during test run 13:28:32 INFO - INFO | automation.py | Application ran for: 0:02:12.725001 13:28:32 INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpEkaGO9pidlog 13:28:32 INFO - WARNING | leakcheck | refcount logging is off, so leaks can't be detected! 13:28:32 INFO - REFTEST INFO | runreftest.py | Running tests: end. 13:28:32 ERROR - Return code: 1
Reporter | ||
Comment 1•10 years ago
|
||
https://tbpl.mozilla.org/php/getParsedLog.php?id=40980557&tree=Mozilla-Inbound We also have bug 1019533 for a recent timeout in this test.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 3•10 years ago
|
||
Stacks are bogus, which is bad.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 8•10 years ago
|
||
Here's the manually symbolized trace, hope that helps until we fix the symbolizer bug.
![]() |
||
Comment 10•10 years ago
|
||
wrong-comment |
This seems like duplicate of bug 971980. But here we have more info. I will duplicate after checking on this bug. Thanks!
![]() |
||
Comment 11•10 years ago
|
||
(In reply to Honza Bambas (:mayhemer) from comment #10) > This seems like duplicate of bug 971980. But here we have more info. I > will duplicate after checking on this bug. Thanks! Err... overlook. Not related... This is new.
![]() |
||
Comment 12•10 years ago
|
||
OK, more related to bug 1011771 landed on 2014-05-28. However, the stack trace looks broken, doesn't make much sense, so hard to say for sure...
Blocks: 1011771
![]() |
||
Comment 13•10 years ago
|
||
When looking at the stack trace at bug 1020584 I am more sure this is related to bug 1011771. Seems like there still is a problem. Michal, would you agree?
Flags: needinfo?(michal.novotny)
Comment 14•10 years ago
|
||
I don't see any similarity with bug 1011771 here. There is not much information in the stack, but it seems to me that it is more related to bug 971980.
Flags: needinfo?(michal.novotny)
![]() |
||
Comment 15•10 years ago
|
||
(In reply to Michal Novotny (:michal) from comment #14) > I don't see any similarity with bug 1011771 here. There is not much > information in the stack, but it seems to me that it is more related to bug > 971980. That was my first thought too (overlook), but look at the stack at [1] mainly at: freed by thread T0 here: #0 0x471b41 in __interceptor_free _asan_rtl_ #1 0x7fe135896c40 in Release /builds/slave/m-in-l64-asan-0000000000000000/build/netwerk/cache2/CacheFileChunk.cpp:77 Clearly double-delete of a CacheFileChunk object, the code we have touched in bug 1011771. [1] https://bug1020584.bugzilla.mozilla.org/attachment.cgi?id=8434480
![]() |
||
Comment 16•10 years ago
|
||
Ah!!! I had once the same mistake... I realize now. We must not access mRefCnt in Release() after DispatchRelease() call since sometimes the object can already be freed... Sorry Michal, your patch v2 was correct with return a local copy of the ref counter in Release(). Will provide a patch - one line.
Assignee: nobody → honzab.moz
Status: NEW → ASSIGNED
![]() |
||
Comment 17•10 years ago
|
||
Attachment #8434838 -
Flags: review?(michal.novotny)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Updated•10 years ago
|
Attachment #8434838 -
Flags: review?(michal.novotny) → review+
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Reporter | ||
Comment 26•10 years ago
|
||
(In reply to TBPL Robot from comment #25) This is on a run that definitely postdates comment 23 :(. Decoder, can we symbolize this one to see if anything looks different?
Flags: needinfo?(choller)
Keywords: leave-open
Comment 27•10 years ago
|
||
Here you go: ==1744==ERROR: AddressSanitizer: heap-use-after-free on address 0x613000284990 at pc 0x7fd6718c963f bp 0x7fff2d6e9950 sp 0x7fff2d6e9948 READ of size 4 at 0x613000284990 thread T0 #0 0x7fd6718c963e in CheckApiState build/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1832 #1 0x7fd6718ef550 in Run build/media/webrtc/signaling/../../../media/mtransport/runnable_utils_generated.h:122 #2 0x7fd670712765 in ProcessNextEvent build/xpcom/threads/nsThread.cpp:766 #3 0x7fd6705d225a in NS_ProcessNextEvent build/xpcom/glue/nsThreadUtils.cpp:263 #4 0x7fd670f206d9 in Run build/ipc/glue/MessagePump.cpp:95 #5 0x7fd670eca550 in RunInternal build/ipc/chromium/src/base/message_loop.cc:229 #6 0x7fd673267df7 in Run build/widget/xpwidgets/nsBaseAppShell.cpp:164 #7 0x7fd67619f618 in Run build/toolkit/components/startup/nsAppStartup.cpp:278 #8 0x7fd67600e503 in XRE_mainRun build/toolkit/xre/nsAppRunner.cpp:4012 #9 0x7fd67600f3e6 in XRE_main build/toolkit/xre/nsAppRunner.cpp:4083 #10 0x7fd67601023d in XRE_main build/toolkit/xre/nsAppRunner.cpp:4297 #11 0x48a2c7 in do_main build/browser/app/nsBrowserApp.cpp:282 #12 0x7fd67f1be76c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226 #13 0x48972c in _start ??:0 0x613000284990 is located 144 bytes inside of 384-byte region [0x613000284900,0x613000284a80) freed by thread T0 here: #0 0x471b41 in __interceptor_free _asan_rtl_ #1 0x7fd6718b9a2c in Release build/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:467 #2 0x7fd67061ad49 in ReleaseSliceNow build/xpcom/base/CycleCollectedJSRuntime.cpp:1066 #3 0x7fd67061b919 in ReleaseNow build/xpcom/base/CycleCollectedJSRuntime.cpp:1138 #4 0x7fd670619829 in OnGC build/xpcom/base/CycleCollectedJSRuntime.cpp:1238 previously allocated by thread T0 here: #0 0x471d41 in __interceptor_malloc _asan_rtl_ #1 0x7fd67b338bed in moz_xmalloc build/memory/mozalloc/mozalloc.cpp:52 #2 0x7fd6718e1509 in operator new build/obj-firefox/media/webrtc/signaling/signaling_ecc/../../../../dist/include/mozilla/mozalloc.h:201 #3 0x7fd6729af474 in _constructor build/obj-firefox/dom/bindings/./PeerConnectionImplBinding.cpp:1179 #4 0x7fd6734a5b78 in construct build/js/xpconnect/wrappers/XrayWrapper.cpp:1629
Flags: needinfo?(choller)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Reporter | ||
Comment 31•10 years ago
|
||
(In reply to Christian Holler (:decoder) from comment #27) > Here you go: Interesting, that looks more like bug 1018372. Glandium landed the fix for the ASAN issues (just waiting on merge to m-c), so we can wait at this point to see how things change once we start getting useful stacks and adjust this bug accordingly.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Updated•10 years ago
|
Summary: Intermittent 408431-1.html | Exited with code 1 during test run after "AddressSanitizer: heap-use-after-free ??:0 ??" error → Intermittent 408431-1.html | Exited with code 1 during test run (after "AddressSanitizer: heap-use-after-free PeerConnectionImpl.cpp:1832 IsClosed")
Updated•10 years ago
|
Component: Networking: Cache → WebRTC
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
![]() |
||
Updated•10 years ago
|
Assignee: nobody → honzab.moz
Status: NEW → ASSIGNED
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Reporter | ||
Updated•10 years ago
|
status-firefox30:
--- → unaffected
status-firefox31:
--- → unaffected
status-firefox32:
--- → affected
status-firefox-esr24:
--- → unaffected
Keywords: leave-open
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
![]() |
||
Updated•10 years ago
|
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 48•10 years ago
|
||
Fixed by backout.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WORKSFORME
Comment hidden (Legacy TBPL/Treeherder Robot) |
Updated•10 years ago
|
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Updated•10 years ago
|
status-firefox33:
--- → affected
Comment 50•10 years ago
|
||
Bill, it looks like maybe those WebRTC UAFs that started happening after your patch are back...
Flags: needinfo?(wmccloskey)
Updated•10 years ago
|
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Flags: needinfo?(wmccloskey)
Resolution: --- → WORKSFORME
You need to log in
before you can comment on or make changes to this bug.
Description
•