Closed Bug 1309159 Opened 8 years ago Closed 8 years ago

Fix multiple UBSan warnings in rijndael.c

Categories

(NSS :: Test, defect)

defect
Not set
normal

Tracking

(firefox-esr45 wontfix)

RESOLVED FIXED
Tracking Status
firefox-esr45 --- wontfix

People

(Reporter: ttaubert, Assigned: ttaubert)

References

Details

(Keywords: sec-low)

We have a few unaligned pointer warnings that are easily fixed, but there's also: rijndael.c:422:15: runtime error: left shift of 254 by 24 places cannot be represented in type 'int' SUMMARY: AddressSanitizer: undefined-behavior rijndael.c:422:15 in [...] The problem seems to be that SBOX() returns a PRUint8, so in SUBBYTE(): > (SBOX((w >> 24) & 0xff) << 24) throws the warning. The left-hand operator is promoted to "int" if we don't specify PRUint32.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → 3.28
Group: crypto-core-security → core-security-release
Mistakenly thought we landed 3.28.1 on the ESR-45 branch
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.