1532867 - UBSan: left shift of negative value in [@ mozilla::BitWriter::WriteBits]

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Tyson Smith [:tsmith]]

Found in m-c commit 78601cacfe69

This was trigger while watching a video stream on twitch.tv

This was build with undefined behavior sanitizer checks enabled via mozconfig.
ac_add_options --enable-undefined-sanitizer="shift"

src/dom/media/BitWriter.cpp:40:36: runtime error: left shift of negative value -1
    #0 0x7f54040d9dc3 in mozilla::BitWriter::WriteBits(unsigned long, unsigned long) src/dom/media/BitWriter.cpp:40:36
    #1 0x7f54046b1517 in WriteU8 src/objdir-ff-ubsan/dist/include/BitWriter.h:20:34
    #2 0x7f54046b1517 in mozilla::H264::CreateExtraData(unsigned char, unsigned char, unsigned char, mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> const&) src/dom/media/platforms/agnostic/bytestreams/H264.cpp:1041
    #3 0x7f54049260b7 in mozilla::MP4Decoder::GetTracksInfo(mozilla::MediaContainerType const&, mozilla::MediaResult&) src/dom/media/mp4/MP4Decoder.cpp:133:11
    #4 0x7f54049247f0 in mozilla::MP4Decoder::IsSupportedType(mozilla::MediaContainerType const&, mozilla::DecoderDoctorDiagnostics*) src/dom/media/mp4/MP4Decoder.cpp:153:17
    #5 0x7f5404059d45 in CanHandleCodecsType src/dom/media/DecoderTraits.cpp:109:9
    #6 0x7f5404059d45 in mozilla::CanHandleMediaType(mozilla::MediaContainerType const&, mozilla::DecoderDoctorDiagnostics*) src/dom/media/DecoderTraits.cpp:160
    #7 0x7f5404578cf4 in mozilla::dom::MediaSource::IsTypeSupported(nsTSubstring<char16_t> const&, mozilla::DecoderDoctorDiagnostics*) src/dom/media/mediasource/MediaSource.cpp:103:7
    #8 0x7f540457db4d in mozilla::dom::MediaSource::IsTypeSupported(mozilla::dom::GlobalObject const&, nsTSubstring<char16_t> const&) src/dom/media/mediasource/MediaSource.cpp:369:17
    #9 0x7f540226358e in mozilla::dom::MediaSource_Binding::isTypeSupported(JSContext*, unsigned int, JS::Value*) src/objdir-ff-ubsan/dom/bindings/MediaSourceBinding.cpp:770:15
    #10 0x7f54090b2c2d in CallJSNative src/js/src/vm/Interpreter.cpp:440:13
    #11 0x7f54090b2c2d in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) src/js/src/vm/Interpreter.cpp:532
    #12 0x7f54090b404d in InternalCall(JSContext*, js::AnyInvokeArgs const&) src/js/src/vm/Interpreter.cpp:587:10
    #13 0x7f540908b1e5 in CallFromStack src/js/src/vm/Interpreter.cpp:591:10
    #14 0x7f540908b1e5 in Interpret(JSContext*, js::RunState&) src/js/src/vm/Interpreter.cpp:3055
    #15 0x7f54090745a0 in js::RunScript(JSContext*, js::RunState&) src/js/src/vm/Interpreter.cpp:420:10
    #16 0x7f54090b2d34 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) src/js/src/vm/Interpreter.cpp:560:13
    #17 0x7f54090b404d in InternalCall(JSContext*, js::AnyInvokeArgs const&) src/js/src/vm/Interpreter.cpp:587:10
    #18 0x7f54090b433a in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) src/js/src/vm/Interpreter.cpp:603:8
    #19 0x7f5409f80587 in js::jit::InvokeFunction(JSContext*, JS::Handle<JSObject*>, bool, bool, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>) src/js/src/jit/VMFunctions.cpp:232:10
    #20 0x16ff98d3853c  (<unknown module>)

Comment 1

[Michael Froman [:mjf]]

Jean-Yves, any thoughts on who should take a look at this?

Comment 2

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1532867 - Force unsigned arithmetic. r?gerald

"For negative a, the behavior of a << b is undefined." https://en.cppreference.com/w/cpp/language/operator_arithmetic

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/be38681e8287
Force unsigned arithmetic. r=gerald

Comment 4

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/be38681e8287