If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Undefined behavior caused by out-of-range shift in secmod_mkCipherFlags and secmod_mkSlotFlags

NEW
Unassigned

Status

NSS
Libraries
5 years ago
5 years ago

People

(Reporter: kinetik, Unassigned)

Tracking

(Blocks: 1 bug)

3.13.6
x86_64
Linux

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
Building mozilla-central with Clang 3.2 on x86_64 with -fcatch-undefined-behavior results in a binary that crashes in secmod_mkCipherFlags and secmod_mkSlotFlags due to out-of-range left shifts:

703 secmod_mkSlotFlags(unsigned long defaultFlags)
...
706     int i,j;
...
708     for (i=0; i < sizeof(defaultFlags)*8; i++) {
709 	if (defaultFlags & (1<<i)) {

sizeof(defaultFlags) * 8 = 64 on LP64, loop terminates when i == 32.  The same pattern occurs twice in secmod_mkCipherFlags:

http://mxr.mozilla.org/mozilla-central/source/security/nss/lib/softoken/pk11pars.h#667
http://mxr.mozilla.org/mozilla-central/source/security/nss/lib/softoken/pk11pars.h#686

Also, presumably the second loop should be using sizeof(ssl1).
You need to log in before you can comment on or make changes to this bug.