UBSan: signed integer overflow in [@ ConditionDimension]

RESOLVED FIXED in Firefox 68

Status

()

defect
P2
normal
Rank:
15
RESOLVED FIXED
4 months ago
3 months ago

People

(Reporter: tsmith, Assigned: jya)

Tracking

(Blocks 2 bugs, {csectype-undefined, testcase})

unspecified
mozilla68
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite ?

Firefox Tracking Flags

(firefox67 wontfix, firefox68 fixed)

Details

Attachments

(2 attachments)

Posted video testcase.mp4

Found in m-c commit af29567ecdba

This was build with undefined behavior sanitizer checks enabled via mozconfig.
ac_add_options --enable-undefined-sanitizer="enum,signed-integer-overflow"

src/dom/media/platforms/agnostic/bytestreams/H264.cpp:338:59: runtime error: 2.14748e+09 is outside the range of representable values of type 'int'
    #0 0x7f3778ebb089 in ConditionDimension src/dom/media/platforms/agnostic/bytestreams/H264.cpp:338:59
    #1 0x7f3778ebb089 in mozilla::H264::DecodeSPS(mozilla::MediaByteBuffer const*, mozilla::SPSData&) src/dom/media/platforms/agnostic/bytestreams/H264.cpp:502
    #2 0x7f3778ebd312 in mozilla::H264::ExtractExtraData(mozilla::MediaRawData const*) src/dom/media/platforms/agnostic/bytestreams/H264.cpp:858:12
    #3 0x7f3778eb313b in mozilla::H264ChangeMonitor::CheckForChange(mozilla::MediaRawData*) src/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:55:30
    #4 0x7f3778e9f037 in mozilla::MediaChangeMonitor::CheckForChange(mozilla::MediaRawData*) src/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:554:36
    #5 0x7f3778ea5a1a in operator() src/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:270:22
    #6 0x7f3778ea5a1a in mozilla::detail::ProxyFunctionRunnable<mozilla::MediaChangeMonitor::Decode(mozilla::MediaRawData*)::$_1, mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true> >::Run() src/objdir-ff-ubsan/dist/include/mozilla/MozPromise.h:1419
    #7 0x7f3772f4957b in mozilla::TaskQueue::Runner::Run() src/xpcom/threads/TaskQueue.cpp:199:12
    #8 0x7f3772f7c107 in nsThreadPool::Run() src/xpcom/threads/nsThreadPool.cpp:241:14
    #9 0x7f3772f7cb5c in non-virtual thunk to nsThreadPool::Run() src/xpcom/threads/nsThreadPool.cpp
    #10 0x7f3772f72822 in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1179:14
    #11 0x7f3772f7866d in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:482:10
    #12 0x7f377407b64a in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:303:20
    #13 0x7f3773f56287 in RunInternal src/ipc/chromium/src/base/message_loop.cc:315:10
    #14 0x7f3773f56287 in RunHandler src/ipc/chromium/src/base/message_loop.cc:308
    #15 0x7f3773f56287 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:290
    #16 0x7f3772f6c710 in nsThread::ThreadFunc(void*) src/xpcom/threads/nsThread.cpp:454:11
    #17 0x7f379204230e in _pt_root src/nsprpub/pr/src/pthreads/ptthread.c:201:5
Flags: in-testsuite?

jya, this appears like your code. Can you take a look?

Rank: 15
Flags: needinfo?(jyavenard)
Priority: -- → P2
Assignee: nobody → jyavenard
Flags: needinfo?(jyavenard)

float(INT32_MAX); gets compiled into 2.14748365E+9 using clang, which is slightly bigger than INT32_MAX, as such 2.14748365E+9 <= INT32_MAX will return true (as INT32_MAX gets converted to a float)

Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bcb48454bcae
Get around int32_t to float rounding error. r=bryce
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.