Closed Bug 1477903 Opened 7 years ago Closed 7 years ago

AddressSanitizer: global-buffer-overflow in nonrd_use_partition

Categories

(Core :: Audio/Video, defect)

Product:
Core
Component:
Audio/Video
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED INVALID
Tracking Flags:
Tracking Status
firefox-esr52 --- wontfix
firefox-esr60 - wontfix
firefox61 --- wontfix
firefox62 - wontfix
firefox63 - wontfix

People

(Reporter: glandium, Unassigned)

Details

(Keywords: csectype-bounds, sec-high)

This is an ASAN report I got on try with ASAN + LTO on mac, while running mochitest-media. The report doesn't happen without LTO. (I was chasing some other crash, hoping that ASAN might show something interesting, but it turns out it's unrelated to what I'm chasing). 20:23:06 ERROR - GECKO(873) | ==874==ERROR: AddressSanitizer: global-buffer-overflow on address 0x0001200838bf at pc 0x00011acc5edd bp 0x00016769cb90 sp 0x00016769cb88 20:23:06 INFO - GECKO(873) | READ of size 1 at 0x0001200838bf thread T2483 20:23:06 INFO - GECKO(873) | ==874==WARNING: failed to fork (errno 1) 20:23:06 INFO - GECKO(873) | ==874==WARNING: failed to fork (errno 1) 20:23:06 INFO - GECKO(873) | ==874==WARNING: failed to fork (errno 1) 20:23:06 INFO - GECKO(873) | ==874==WARNING: failed to fork (errno 1) 20:23:06 INFO - GECKO(873) | ==874==WARNING: failed to fork (errno 1) 20:23:06 INFO - GECKO(873) | ==874==WARNING: Failed to use and restart external symbolizer! 20:23:06 INFO - GECKO(873) | #0 0x11acc5edc in nonrd_use_partition (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xafdfedc) 20:23:06 INFO - GECKO(873) | #1 0x11acc507e in nonrd_use_partition (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xafdf07e) 20:23:06 INFO - GECKO(873) | #2 0x11acb608b in vp9_encode_tile (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xafd008b) 20:23:06 INFO - GECKO(873) | #3 0x11ad14e94 in enc_worker_hook (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb02ee94) 20:23:06 INFO - GECKO(873) | #4 0x11aed198e in execute (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb1eb98e) 20:23:06 INFO - GECKO(873) | #5 0x11ad137c6 in vp9_encode_tiles_mt (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb02d7c6) 20:23:06 INFO - GECKO(873) | #6 0x11acbb074 in encode_frame_internal (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xafd5074) 20:23:06 INFO - GECKO(873) | #7 0x11acb80b7 in vp9_encode_frame (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xafd20b7) 20:23:06 INFO - GECKO(873) | #8 0x11ad0bbc2 in encode_frame_to_data_rate (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb025bc2) 20:23:06 INFO - GECKO(873) | #9 0x11ad0239a in vp9_get_compressed_data (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb01c39a) 20:23:06 INFO - GECKO(873) | #10 0x11ae13256 in encoder_encode (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb12d256) 20:23:06 INFO - GECKO(873) | #11 0x11ae2f350 in vpx_codec_encode (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xb149350) 20:23:06 INFO - GECKO(873) | #12 0x11cfee7a7 in webrtc::VP9EncoderImpl::Encode(webrtc::VideoFrame const&, webrtc::CodecSpecificInfo const*, std::__1::vector<webrtc::FrameType, std::__1::allocator<webrtc::FrameType> > const*) (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xd3087a7) 20:23:06 INFO - GECKO(873) | #13 0x11cf595f7 in webrtc::VCMGenericEncoder::Encode(webrtc::VideoFrame const&, webrtc::CodecSpecificInfo const*, std::__1::vector<webrtc::FrameType, std::__1::allocator<webrtc::FrameType> > const&) (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xd2735f7) 20:23:06 INFO - GECKO(873) | #14 0x11cfaa29e in webrtc::vcm::VideoSender::AddVideoFrame(webrtc::VideoFrame const&, webrtc::CodecSpecificInfo const*) (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xd2c429e) 20:23:06 INFO - GECKO(873) | #15 0x11d05fee4 in webrtc::ViEEncoder::EncodeVideoFrame(webrtc::VideoFrame const&, long long) (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xd379ee4) 20:23:06 INFO - GECKO(873) | #16 0x11d066f89 in webrtc::ViEEncoder::EncodeTask::Run() (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xd380f89) 20:23:06 INFO - GECKO(873) | #17 0x11cc2fe04 in rtc::TaskQueue::TaskContext::RunTask(void*) (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xcf49e04) 20:23:06 INFO - GECKO(873) | #18 0x10e3cb723 in asan_dispatch_call_block_and_release (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/libclang_rt.asan_osx_dynamic.dylib:x86_64+0x55723) 20:23:06 INFO - GECKO(873) | #19 0x7fff8f48dc12 in _dispatch_client_callout (/usr/lib/system/libdispatch.dylib:x86_64+0x1c12) 20:23:06 INFO - GECKO(873) | #20 0x7fff8f491364 in _dispatch_queue_drain (/usr/lib/system/libdispatch.dylib:x86_64+0x5364) 20:23:06 INFO - GECKO(873) | #21 0x7fff8f492ecb in _dispatch_queue_invoke (/usr/lib/system/libdispatch.dylib:x86_64+0x6ecb) 20:23:06 INFO - GECKO(873) | #22 0x7fff8f4906b6 in _dispatch_root_queue_drain (/usr/lib/system/libdispatch.dylib:x86_64+0x46b6) 20:23:06 INFO - GECKO(873) | #23 0x7fff8f49efe3 in _dispatch_worker_thread3 (/usr/lib/system/libdispatch.dylib:x86_64+0x12fe3) 20:23:06 INFO - GECKO(873) | #24 0x7fff8e212a9c in _pthread_wqthread (/usr/lib/system/libsystem_pthread.dylib:x86_64+0x3a9c) 20:23:06 INFO - GECKO(873) | #25 0x7fff8e2103dc in start_wqthread (/usr/lib/system/libsystem_pthread.dylib:x86_64+0x13dc) 20:23:06 INFO - GECKO(873) | 0x0001200838bf is located 33 bytes to the left of global variable 'tx_size_wide_unit' defined in '/builds/worker/workspace/build/src/third_party/aom/av1/common/common_data.h:985:18' (0x1200838e0) of size 56 20:23:06 INFO - GECKO(873) | 0x0001200838bf is located 7 bytes to the right of global variable 'tx_size_high_unit' defined in '/builds/worker/workspace/build/src/third_party/aom/av1/common/common_data.h:1010:18' (0x120083880) of size 56 20:23:06 INFO - GECKO(873) | SUMMARY: AddressSanitizer: global-buffer-overflow (/Users/cltbld/tasks/task_1532054723/build/application/Firefox Nightly.app/Contents/MacOS/XUL:x86_64+0xafdfedc) in nonrd_use_partition 20:23:06 INFO - GECKO(873) | Shadow bytes around the buggy address: 20:23:06 INFO - GECKO(873) | 0x1000240106c0: 00 06 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 f9 f9 20:23:06 INFO - GECKO(873) | 0x1000240106d0: f9 f9 f9 f9 00 00 00 00 00 04 f9 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | 0x1000240106e0: 00 00 06 f9 f9 f9 f9 f9 00 00 06 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | 0x1000240106f0: 00 00 f9 f9 f9 f9 f9 f9 00 05 06 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | 0x100024010700: 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | =>0x100024010710: 00 00 00 00 00 00 00[f9]f9 f9 f9 f9 00 00 00 00 20:23:06 INFO - GECKO(873) | 0x100024010720: 00 00 00 f9 f9 f9 f9 f9 00 00 06 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | 0x100024010730: 00 00 06 f9 f9 f9 f9 f9 00 00 06 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | 0x100024010740: 00 06 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 f9 20:23:06 INFO - GECKO(873) | 0x100024010750: f9 f9 f9 f9 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 20:23:06 INFO - GECKO(873) | 0x100024010760: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 00 00 00 00 20:23:06 INFO - GECKO(873) | Shadow byte legend (one shadow byte represents 8 application bytes): 20:23:06 INFO - GECKO(873) | Addressable: 00 20:23:06 INFO - GECKO(873) | Partially addressable: 01 02 03 04 05 06 07 20:23:06 INFO - GECKO(873) | Heap left redzone: fa 20:23:06 INFO - GECKO(873) | Freed heap region: fd 20:23:06 INFO - GECKO(873) | Stack left redzone: f1 20:23:06 INFO - GECKO(873) | Stack mid redzone: f2 20:23:06 INFO - GECKO(873) | Stack right redzone: f3 20:23:06 INFO - GECKO(873) | Stack after return: f5 20:23:06 INFO - GECKO(873) | Stack use after scope: f8 20:23:06 INFO - GECKO(873) | Global redzone: f9 20:23:06 INFO - GECKO(873) | Global init order: f6 20:23:06 INFO - GECKO(873) | Poisoned by user: f7 20:23:06 INFO - GECKO(873) | Container overflow: fc 20:23:06 INFO - GECKO(873) | Array cookie: ac 20:23:06 INFO - GECKO(873) | Intra object redzone: bb 20:23:06 INFO - GECKO(873) | ASan internal: fe 20:23:06 INFO - GECKO(873) | Left alloca redzone: ca 20:23:06 INFO - GECKO(873) | Right alloca redzone: cb Unfortunately, things are not hooked up such that we'd get line numbers directly, but I was able to correlate with crash reporter symbols on that build, and the crash is happening on this line of code: https://hg.mozilla.org/try/file/9ca81e107e6bb3176ffe75a63544c11dd06eefa6/media/libvpx/libvpx/vp9/common/vp9_onyxc_int.h#l427 and the calling frame is on: https://hg.mozilla.org/try/file/9ca81e107e6bb3176ffe75a63544c11dd06eefa6/media/libvpx/libvpx/vp9/encoder/vp9_encodeframe.c#l3790 Looking a few lines above that, we can see a `subsize = get_subsize(bsize, PARTITION_SPLIT)`, which, looking at the corresponding lookup tables, can return a value of BLOCK_INVALID. That value does overflow the size of mi_width_log2_lookup. Not having any clue about how this code works, I can't say what the right fix would be, but adding `bsize < BLOCK_INVALID` as a condition on https://hg.mozilla.org/try/file/9ca81e107e6bb3176ffe75a63544c11dd06eefa6/media/libvpx/libvpx/vp9/encoder/vp9_encodeframe.c#l3720 solved it for me, and I'm now back to my original crash, without an ASAN report. I haven't looked at any of the other partition_plane_context callers to see if any of them might pass an invalid bsize, and likewise for update_partition_context, which appears a few lines above partition_plane_context and is also indexing lookup tables without bound checks. Introducing bound checks to these functions would be a defensive thing to do, but the error would need to be propagated properly.
Nils: who manages vpx code these days?
Group: core-security → media-core-security
Flags: needinfo?(drno)
Flags: needinfo?(drno)
Thomas, what do you think about this one here? Does glandium's fix look sane to you? Or should we try to patch this differently?
Flags: needinfo?(tdaede)
The fix is OK, but it indicates that there is another bug that is likely affecting encoder quality. It would also break output bitstreams (both before and after path) but there is a separate codepath for bitstream writing. I'll need to take a look at that to see if it has the same error. Also, our libvpx is out of date - though I checked master and I suspect it is vulnerable to this same issue.
Flags: needinfo?(tdaede)
(In reply to Thomas Daede [:TD-Linux] from comment #3) > The fix is OK, but it indicates that there is another bug that is likely > affecting encoder quality. It would also break output bitstreams (both > before and after path) but there is a separate codepath for bitstream > writing. I'll need to take a look at that to see if it has the same error. > > Also, our libvpx is out of date - though I checked master and I suspect it > is vulnerable to this same issue. Thanks for checking. Indeed the latest code on github appears unchanged to me in the relevant areas. We do have a ticket for updating to libvpx 1.7.0. I guess we should have a patch which we can uplift into all relevant branches of Firefox. And then we should update libvpx to a revision which includes the fix upstream.
Setting affected status based on looking at the code on the branches.
status-firefox61: --- → affected
status-firefox62: --- → affected
status-firefox63: --- → affected
status-firefox-esr52: --- → affected
status-firefox-esr60: --- → affected
tracking-firefox62: --- → ?
tracking-firefox63: --- → ?
A couple notes: - as mentioned in the last paragraph in comment 0, there are other code paths that could possibly lead to the same kind of buffer overflow, but I haven't looked at them. I only looked at the one code path that was reported to me, and the mentioned "fix" only addresses that one. - I only filed this as a security bug because it's the safer option, but I also didn't assess whether that's actually a security issue. For one, I don't even know if web content can trigger vp9 encoding. And ASAN doesn't catch anything without LTO also enabled, which is actually weird. Like, the code path doesn't happen when the build is not LTO'ed? O_o
(In reply to Mike Hommey [:glandium] from comment #6) > - I only filed this as a security bug because it's the safer option, but I > also didn't assess whether that's actually a security issue. For one, I > don't even know if web content can trigger vp9 encoding. And ASAN doesn't > catch anything without LTO also enabled, which is actually weird. Like, the > code path doesn't happen when the build is not LTO'ed? O_o Definitely safer to keep it as a sec bug for now. If VP9 enconding is being used is under the control of the JS code which does setup a WebRTC connection. Now controlling what exactly gets encoded is probably a little harder if you take the input from a camera. But since we can capture from a canvas I think it's feasible for an attacker to create content specifically aimed at triggering problems in video encoders.
FWIW, I updated my tree yesterday and pushing mozilla-central from then with LTO enabled, and this time, I got a probably related crash in vp9_encodeframe.c without ASAN: 19:59:33 INFO - Thread 64 (crashed) 19:59:33 INFO - 0 XUL!nonrd_use_partition [vp9_encodeframe.c:260ba1e47323877f8ac4cdcfdff3511a402b1d0d : 3717 + 0x0] 19:59:33 INFO - rax = 0x0000000000000000 rdx = 0x0000000000000000 19:59:33 INFO - rcx = 0x0000000000000000 rbx = 0x000000012e94b860 19:59:33 INFO - rsi = 0x00000001302e1020 rdi = 0x0000000130202020 19:59:33 INFO - rbp = 0x000000013159aa50 rsp = 0x000000013159a9d0 19:59:33 INFO - r8 = 0x000000013159adc0 r9 = 0x0000000000000002 19:59:33 INFO - r10 = 0x0000000000000003 r11 = 0x0000000131807fe0 19:59:33 INFO - r12 = 0x00000000000000ff r13 = 0x0000000000000001 19:59:33 INFO - r14 = 0x0000000000000001 r15 = 0x0000000000000002 19:59:33 INFO - rip = 0x0000000113574b43 19:59:33 INFO - Found by: given as instruction pointer in context 19:59:33 INFO - 1 XUL!nonrd_use_partition [vp9_encodeframe.c:260ba1e47323877f8ac4cdcfdff3511a402b1d0d : 3795 + 0x2c] 19:59:33 INFO - rbp = 0x000000013159ab00 rsp = 0x000000013159aa60 19:59:33 INFO - rip = 0x0000000113574cb0 19:59:33 INFO - Found by: previous frame's frame pointer 19:59:33 INFO - 2 XUL!nonrd_use_partition [vp9_encodeframe.c:260ba1e47323877f8ac4cdcfdff3511a402b1d0d : 3792 + 0x30] 19:59:33 INFO - rbp = 0x000000013159abb0 rsp = 0x000000013159ab10 19:59:33 INFO - rip = 0x0000000113574c6c 19:59:33 INFO - Found by: previous frame's frame pointer 19:59:33 INFO - 3 XUL!nonrd_use_partition [vp9_encodeframe.c:260ba1e47323877f8ac4cdcfdff3511a402b1d0d : 3790 + 0x33] 19:59:33 INFO - rbp = 0x000000013159ac60 rsp = 0x000000013159abc0 19:59:33 INFO - rip = 0x0000000113574c29 19:59:33 INFO - Found by: previous frame's frame pointer 19:59:33 INFO - 4 XUL!vp9_encode_tile [vp9_encodeframe.c:260ba1e47323877f8ac4cdcfdff3511a402b1d0d : 3860 + 0x39] 19:59:33 INFO - rbp = 0x000000013159ae90 rsp = 0x000000013159ac70 19:59:33 INFO - rip = 0x000000011356ec0c
I think that shows the fix mentioned in comment 0 is not at the right location (this was with that applied). I guess nonrd_use_partition shouldn't recurse if subsize is BLOCK_INVALID?
Tracking for 62/63, looks like you are all actively working on this so I'm tracking to make sure to follow up in a week or so.
tracking-firefox62: ? → +
tracking-firefox63: ? → +
I tried making nonrd_use_partition for PARTITION_SPLIT not recurse if subsize is BLOCK_INVALID, but I still got a crash. And now that I rebased, I'm now getting a crash in write_modes_sb in vp9_bitstream.c, and it looks like it has the same lookup table overflow problem. I'm starting to think maybe the "solution" would be to extend those lookup tables to avoid an off by one on BLOCK_INVALID...
What's the easiest way for me to reproduce this individual issue? I can take a look at doing a more involved fix. Yeah, the existence of BLOCK_INVALID was a mistake in this codebase. Not a huge fan of the extending lookup tables, but you can change the BLOCK_INVALID value to a much smaller one so that the tables only become one larger.
BLOCK_INVALID *is* BLOCK_SIZES, so the tables would only become one larger. Adding ac_add_options --enable-lto to browser/config/mozconfigs/macosx64/nightly is enough to enjoy. Then mach try fuzzy -q test-macosx64/opt-mochitest-media-e10s
Ah whoops, was looking at the libaom codebase where it's 255. (BTW this same bug is likely reproducible in libaom as well)
It still bugs me that this doesn't happen without LTO. ASAN should catch this on its own, but it doesn't happen unless LTO is enabled as well. I'm not excluding that there is some miscompilation involved here, and that those unbounded accesses are actually fine. In fact, for nonrd_use_partition, all non-recursive calls are with BLOCK_64X64, and all recursive calls are using get_subsize(bsize, PARTITION_SPLIT). So the first recursion would get subsize_lookup[PARTITION_SPLIT][BLOCK_64X64], which is BLOCK_32X32. From there, it would recurse using subsize_lookup[PARTITION_SPLIT][BLOCK_32X32], which is BLOCK_16X16. and so on. I suspect the overflow is actually a red herring, and something else is corrupted that leads to the lookup tables being misused.
Actually, to enable LTO, just (re)apply the patch from bug 1473786 (otherwise you'll have compare_mozconfig errors). I've tried adding a MOZ_RELEASE_ASSERT(bsize < BLOCK_INVALID) at the beginning of write_modes_sb (since this is now where I get the crash). The thing is that it is only hit when LTO is enabled.
So, there clearly is something wrong going on. I was able to get some information from when the subsize ends up being BLOCK_INVALID, and the values passed to get_subsize are 12 for bsize (BLOCK_64X64) and 3 for partition (PARTITION_SPLIT). The value as defined in subsize_lookup should be BLOCK_32X32, yet, somehow it got BLOCK_INVALID. Digging deeper, but it looks like there is no actual security issue here, "just" a miscompilation or mislinkage with LTO.
Found the problem, and it's definitely a compiler or linker problem. The function reads from subsize_lookup as expected, but the problem is that subsize_lookup doesn't contain the right data! The data it's looking at is: 5712640 00030609 0c0fff02 05080b0e ff010407 ................ 5712650 0a0dff00 0306090c ff020508 0b0eff02 ................ 5712660 05080b0e ff010407 0a0dff01 04070a0d ................ 5712670 ffff1113 15ffffff 101214ff 00000000 ................ And if you look attentively, this content corresponds to... aom's subsize_lookup (instead of vp9's). Which actually shouldn't be happening because aom's is static.
This is clearly a ld64 bug (the mac linker). This doesn't happen with ld.bfd or lld, so this presumably doesn't affect the LTO'ed windows builds, but those don't work for mac. A small testcase is as follows: $ cat <<EOF > a.c static const char foo[] = "foo"; char get_foo(int i) { return foo[i]; } EOF $ cat <<EOF > b.c const char foo_[] = "bar"; char get_foo2(int i) { return foo_[i]; } EOF $ /builds/worker/workspace/build/src/clang/bin/clang -target x86_64-apple-darwin11 -B /builds/worker/workspace/build/src/cctools/bin -isysroot /builds/worker/workspace/build/src/MacOSX10.11.sdk -flto=thin -o libfoo.dylib -shared -fPIC [ab].c $ /builds/worker/workspace/build/src/clang/bin/llvm-objdump -s libfoo.dylib libfoo.dylib: file format Mach-O 64-bit x86-64 Contents of section __text: 0f70 554889e5 488d052d 00000089 7dfc4863 UH..H..-....}.Hc 0f80 4dfc0fbe 04085dc3 90909090 90909090 M.....]......... 0f90 554889e5 488d050d 00000089 7dfc4863 UH..H.......}.Hc 0fa0 4dfc0fbe 04085dc3 M.....]. Contents of section __const: 0fa8 62617200 bar. Contents of section __unwind_info: 0fac 01000000 1c000000 00000000 1c000000 ................ 0fbc 00000000 1c000000 02000000 700f0000 ............p... 0fcc 34000000 34000000 a90f0000 00000000 4...4........... 0fdc 34000000 03000000 0c000100 10000100 4............... 0fec 00000000 00000001 ........ See how there is no data section containing the "foo" string, because it was "merged" with the "bar" string. Interestingly, this does *not* happen with full lto: $ /builds/worker/workspace/build/src/clang/bin/clang -target x86_64-apple-darwin11 -B /builds/worker/workspace/build/src/cctools/bin -isysroot /builds/worker/workspace/build/src/MacOSX10.11.sdk -flto -o libfoo.dylib -shared -fPIC [ab].c $ /builds/worker/workspace/build/src/clang/bin/llvm-objdump -s libfoo.dylib libfoo.dylib: file format Mach-O 64-bit x86-64 Contents of section __text: 0f70 554889e5 488d0531 00000089 7dfc4863 UH..H..1....}.Hc 0f80 4dfc0fbe 04085dc3 0f1f8400 00000000 M.....]......... 0f90 554889e5 488d050d 00000089 7dfc4863 UH..H.......}.Hc 0fa0 4dfc0fbe 04085dc3 M.....]. Contents of section __const: 0fa8 62617200 bar. Contents of section __cstring: 0fac 666f6f00 foo. Contents of section __unwind_info: 0fb0 01000000 1c000000 00000000 1c000000 ................ 0fc0 00000000 1c000000 02000000 700f0000 ............p... 0fd0 34000000 34000000 a90f0000 00000000 4...4........... 0fe0 34000000 03000000 0c000100 10000100 4............... 0ff0 00000000 00000001 ........
So all in all, I think none of the versions of Firefox are actually affected here, that there isn't a bug in vp9, although maybe vp9 could try do with some defensive tests. I don't know if there are ways to craft a stream that would trigger some bad behavior. I don't know if the VP9_COMP data that is used to select partitions and subsizes comes from user data or not.
So I don't have the workspace set up like you do, but this at least doesn't repro, likely due to being newer versions: > mozs-MacBook-Pro:tmp jgilbert$ clang -target x86_64-apple-darwin11 -isysroot /Applications/Xcode.app//Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -flto=thin -o libfoo.dylib -shared -fPIC [ab].c > > mozs-MacBook-Pro:tmp jgilbert$ objdump -s libfoo.dylib > > libfoo.dylib: file format Mach-O 64-bit x86-64 > > Contents of section __text: > 0f70 554889e5 488d0531 00000089 7dfc4863 UH..H..1....}.Hc > 0f80 4dfc0fbe 04085dc3 90909090 90909090 M.....]......... > 0f90 554889e5 488d050d 00000089 7dfc4863 UH..H.......}.Hc > 0fa0 4dfc0fbe 04085dc3 M.....]. > Contents of section __const: > 0fa8 62617200 bar. > Contents of section __cstring: > 0fac 666f6f00 foo. > Contents of section __unwind_info: > 0fb0 01000000 1c000000 00000000 1c000000 ................ > 0fc0 00000000 1c000000 02000000 700f0000 ............p... > 0fd0 34000000 34000000 a90f0000 00000000 4...4........... > 0fe0 34000000 03000000 0c000100 10000100 4............... > 0ff0 00000000 00000001 ........ > > mozs-MacBook-Pro:tmp jgilbert$ clang --version > Apple LLVM version 9.1.0 (clang-902.0.39.2) > Target: x86_64-apple-darwin17.6.0 > Thread model: posix > InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin > > mozs-MacBook-Pro:tmp jgilbert$ ld -v > @(#)PROGRAM:ld PROJECT:ld64-351.8 > configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS) > LTO support using: LLVM version 9.1.0, (clang-902.0.39.2) (static support for 21, runtime is 21) > TAPI support using: Apple TAPI version 9.1.0 (tapi-902.0.9) I'm unclear what -B achieves or might change here, but the rest seems reasonable.
That's exactly the kind of information I was looking for. I can reproduce with Xcode 8, and that's the last version where ld64 was open sourced. Meaning, Apple fixed the issue. I'll try poking around and see if we can get the fix from them.
IIUC, this isn't something we really need to be tracking for Firefox?
Flags: needinfo?(mh+mozilla)
I think we can actually close this. This is a toolchain problem that manifests itself as unsafe access to memory. There might be some unsafety lurking in the code, but the problem I filed this for actually doesn't exist with a non-buggy toolchain. It also wouldn't happen with bug 1477888 fixed.
Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(mh+mozilla)
Resolution: --- → INVALID
Group: media-core-security
You need to log in before you can comment on or make changes to this bug.