1152740 - h264dec can cause division by zero

Status: RESOLVED WONTFIX

(Core :: Audio/Video: GMP, defect)

Description

[Hanno Boeck]

Attachment: openh264-divzero.diff

In certain circumstances the h264dec tool can trigger a division by zero in line 296 of h264dec.cpp. As the variable dElapsed is a double it won't crash. To see this one has to compile with undefined behaviour sanitizer.

This should reproduce:
make CC=clang CXX=clang++ CFLAGS="-fsanitize=undefined -fpic" LDFLAGS="-fsanitize=undefined -lpthread"
dd if=/dev/zero bs=1 count=1 of=zero
UBSAN_OPTIONS="print_stacktrace=1" ./h264dec zero out

It's not really causing any issues, I'm just reporting it because it could confuse people looking for bugs with ubsan. Attached a patch (probably it could be done in some nicer way).

Comment 1

[Dan Minor [:dminor]]

Thank you for the report and patch. Since this affects a command line tool not used in Firefox, if the issue still exists, it should be reported upstream: https://github.com/cisco/openh264.