AES-GCM produces incorrect authentication tags on some platforms for data >= 512 MB due to 32-bit integer overflow in length computation
Categories
(NSS :: Libraries, defect, P2)
Tracking
(firefox-esr115150+ fixed, firefox-esr140148+ fixed, firefox147 wontfix, firefox148+ fixed, firefox149+ fixed)
People
(Reporter: clayvv, Assigned: jschanck)
References
Details
(Keywords: csectype-intoverflow, reporter-external, sec-moderate, Whiteboard: [client-bounty-form][adv-main148+][adv-esr140.8+][adv-esr115.35+])
Attachments
(2 files, 1 external file)
Firefox WebCrypto AES-GCM Bug Report
Summary
WebCrypto AES-GCM produces incorrect authentication tags for data >= 512 MB due to 32-bit integer overflow in length computation
Affected Versions
- Tested on: Firefox (current version as of January 2026)
- Component: Core :: Security: PSM (NSS)
Description
Firefox's WebCrypto implementation of AES-GCM produces incorrect authentication tags when encrypting or decrypting data >= 512 MB (536,870,912 bytes). This causes interoperability failures with all other AES-GCM implementations including Chrome, OpenSSL, Java (SunJCE and BouncyCastle), and Node.js.
The bug appears to be a 32-bit integer overflow in the GCM length block computation. The GCM specification (NIST SP 800-38D) requires 64-bit length fields, but Firefox/NSS appears to use 32-bit arithmetic. When the ciphertext length in bits exceeds 2^32 (which occurs at 512 MB = 2^32 bits / 8), the length wraps around, producing an incorrect GHASH and thus an incorrect authentication tag.
Steps to Reproduce
- Open the attached
firefox-gcm-bug-test.htmlin Firefox - Click "Run Tests (10MB - 1024MB)"
- Compare the computed tags with the reference values shown on the page
- Repeat in Chrome for comparison
Expected Results
Firefox should produce identical tags to Chrome and server-side implementations at all sizes:
| Size | Expected Tag (Chrome/OpenSSL/Java) |
|---|---|
| 10 MB | E0AB33498A9A54A93796DEDC62763441 |
| 50 MB | 1A213F7F4F53ADF640B03A4E99B859AE |
| 100 MB | 9D6D3B30C437D33FCA1C0FFBA7EDF3C1 |
| 200 MB | 7F4DFED7961C3C5359AC9C3C93409708 |
| 512 MB | 513254EA22DA5846ABD0BD873BD53F2B |
| 1024 MB | A85365DB36D9D1B5298F7E6C98EB9839 |
Actual Results
Firefox matches expected values for < 512 MB but diverges at >= 512 MB:
| Size | Firefox Tag | Matches Expected? |
|---|---|---|
| 10 MB | E0AB33498A9A54A93796DEDC62763441 |
✓ YES |
| 50 MB | 1A213F7F4F53ADF640B03A4E99B859AE |
✓ YES |
| 100 MB | 9D6D3B30C437D33FCA1C0FFBA7EDF3C1 |
✓ YES |
| 200 MB | 7F4DFED7961C3C5359AC9C3C93409708 |
✓ YES |
| 512 MB | DA4BA5991D2051E467401216A907A3D4 |
✗ NO ← Diverges here |
| 1024 MB | 7CA0873D492DC2F0B0AE214FBD4EA1C6 |
✗ NO |
Chrome produces correct tags at ALL sizes, confirming this is Firefox-specific.
Impact
Summary
- Files >= 512 MB encrypted by Firefox cannot be decrypted by any other implementation
- Files >= 512 MB encrypted by other implementations cannot be decrypted by Firefox
- Firefox-to-Firefox works (both use same buggy computation)
- This breaks interoperability for any application using WebCrypto AES-GCM with large files
Real-world consequences for applications using WebCrypto AES-GCM
-
AES-GCM MAC verification cannot be reliably used for files >= 512 MB
- Applications cannot know which browser uploaded a file
- If ANY chance exists that a file was uploaded by Firefox, MAC verification must be skipped
- This effectively defeats the authentication guarantee of AES-GCM for large files
-
Interoperability is broken bidirectionally
- Files >= 512 MB encrypted by Firefox cannot be verified by Chrome, server-side code, or any standards-compliant implementation
- Files >= 512 MB encrypted by other implementations cannot be verified by Firefox
-
Data loss risk when this bug is fixed
- Files currently uploaded by Firefox with size >= 512 MB have incorrect MAC tags stored
- Once Mozilla fixes this bug, those files will become unverifiable/undownloadable by Firefox
- Applications must implement fallback decryption (e.g., AES-CTR) to maintain access to legacy Firefox-uploaded files
-
Security implications
- Applications are forced to either:
- (a) Skip MAC verification for large files (losing authentication guarantee), or
- (b) Block large file uploads entirely, or
- (c) Block Firefox users from uploading large files
- None of these options are acceptable for production applications
- Applications are forced to either:
Technical Analysis
The GCM authentication tag is computed using GHASH over:
- Additional Authenticated Data (AAD)
- Ciphertext
- A length block containing:
[len(AAD) in bits || len(ciphertext) in bits]
The length block must use 64-bit big-endian integers per NIST SP 800-38D.
At 512 MB:
- Ciphertext = 536,870,912 bytes
- Length in bits = 4,294,967,296 = 2^32 exactly
- If using 32-bit arithmetic, this wraps to 0
This explains why Firefox diverges at exactly the 512 MB boundary.
Test Parameters
All tests use identical parameters:
| Parameter | Value |
|---|---|
| Algorithm | AES-256-GCM |
| Key (hex) | F7F698D851DE2EB343CAB95BC3D498E4A6E665DB8D93CE278E595371B4B66594 |
| IV (hex) | 4A0F0A225AE4FB34555FC51C |
| AAD | 2026-01-09T19:23:00.000Z (UTF-8 encoded) |
| Plaintext | Zero-filled buffer of specified size |
| Tag length | 128 bits |
Attachments
firefox-gcm-bug-test.html- Self-contained test page that can be run in any browser
References
- NIST SP 800-38D: Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC
- Section 7.1 specifies 64-bit length fields in the GHASH computation
| Reporter | ||
Comment 1•10 months ago
|
||
I did not mention above, but the tests I did with the attached HTML were only with Chrome and Firefox on a Mac running Sequoia 15.7.3
| Reporter | ||
Comment 2•10 months ago
|
||
I believe the problem is in security/nss/lib/freebl/gcm.c on line 360
Buggy code:
ghash->cLen += (len * PR_BITS_PER_BYTE);
The parameter len is declared as unsigned int (32-bit) in the function signature:
SECStatus gcmHash_Update(gcmHashContext *ghash, const unsigned char *buf, unsigned int len)
When len >= 536,870,912 bytes (512 MB), the multiplication len * 8 overflows
the 32-bit unsigned int (2^32 = 4,294,967,296) before being added to the 64-bit cLen.
This causes the GHASH length block to contain an incorrect bit count, producing
an invalid authentication tag for any data >= 512 MB.
If you agree, the fastest fix with minimal changes would be to cast len to uint64_t before the multiplication:
ghash->cLen += ((uint64_t)len * PR_BITS_PER_BYTE);
A more comprehensive fix that would maintain API consistentency might be to change the function signature of gcmHash_Update so that len is declared as a uint64_t.
Updated•10 months ago
|
Updated•10 months ago
|
| Assignee | ||
Comment 3•10 months ago
|
||
Thank you for this detailed report!
This primarily affects arm64. We use a different GHASH implementation on Intel and PPC.
| Assignee | ||
Comment 4•10 months ago
|
||
| Reporter | ||
Comment 5•10 months ago
|
||
@:jschanck - Once the fix has been made though, any files >=512MB in size previously uploaded via Firefox will not pass the MAC verification even if being downloaded with Firefox, right?
This is particularly important for me as I am a Technical Leader at Cisco on the Webex team and this will have a significant impact on us as we allow files of up to 2GB in size to be attached to messages and support using a browser (https://web.webex.com).
As this potentially impacts any web service that does AES-GCM encryption of large file uploads (including in the FedRAMP environment), a MAC check of these large "legacy" files cannot relied upon and skipping the MAC check is not suitable an option either.
| Assignee | ||
Comment 6•10 months ago
|
||
| Assignee | ||
Comment 7•10 months ago
•
|
||
Yes, although I'll add that it's already the case that a file that was encrypted on an arm64 build of Firefox will fail to decrypt on an x64 build of Firefox (and vice versa).
I agree that skipping the MAC check is not a suitable option. Fortunately, using a correct implementation, you can easily compute that tag that you would expect from an implementation with this bug. I'll attach some sample code based on your example. Note that I used Sonnet 4.5 to generate this. I only done a cursory review.
Comment 8•10 months ago
|
||
We are shipping 147 today so that's a 'wontfix' for practical reasons, but we should try to get a fix in 148 and the corresponding ESR-140. I doubt this is worth fixing on ESR-115 for the dwindling number of stranded Win7 users.
Although integer overflows can lead to security bugs (e.g. overflowing unexpectedly small buffers), this seems to be a correctness/interoperability problem rather than a security issue.
| Assignee | ||
Comment 9•10 months ago
•
|
||
Contrary to what I said in our meeting yesterday, Dan, there is a security issue here. I missed this earlier because the variable is called cLen, but the integer overflow affects the calculation of the length of the associated data as well.
Here's why that matters. If we view GHASH(H, a, c) as a polynomial in H, then the trailing coefficient is supposed to be len(a) || len(c). This bug causes NSS to use len(a) mod 2^32 || len(c) mod 2^32 instead. The security proof for GHASH (page 14 of https://eprint.iacr.org/2004/193.pdf) uses the fact that the tuple (a, c) can be extracted unambiguously from the polynomial GHASH(H, a, c) (it can because the lengths of a and c are encoded in the trailing coefficient). Reducing the lengths mod 2^32 introduces ambiguity: you can move a 512 MB prefix of c to a suffix of a and vice versa.
The upshot is that if an attacker observes a GCM ciphertext (iv, a, c, tag) where a is multiple of 16 bytes in length and c is more than 512 MB in length, then he can write down a second valid ciphertext (iv, a || c0, c1, tag) where c = c0 || c1 and c0 is (some multiple of) 512 MB. In crypto jargon the fact that this new ciphertext decrypts without an authentication failure is a violation of the "INT-CTXT" property of the AEAD scheme.
If the bug had only affected len(c) there would be no issue, as len(a) || len(c) mod 2^32 is still enough information to extract (a, c) from GHASH(H, a, c).
There may not be a real world security impact from this bug (the forged ciphertext will decrypt to a "random" plaintext). But we can't write this off as "just" an interop bug.
Updated•10 months ago
|
Comment 10•9 months ago
|
||
:jschanck, there are only a couple of weeks left in beta for Fx148.
What's the plan for landing this and adding uplift requests for the affected versions?
Comment 11•9 months ago
|
||
Pushed by jschanck@mozilla.com:
https://hg.mozilla.org/projects/nss/rev/245385e16fa6
avoid integer overflow in platform-independent ghash. r=nss-reviewers,nkulatova
Comment 12•9 months ago
|
||
Updated•9 months ago
|
Updated•9 months ago
|
Comment 13•9 months ago
|
||
:jschanck, the last Fx148 beta builds, so we'll need uplift requests before then
Updated•9 months ago
|
Comment 14•9 months ago
|
||
https://bugzilla.mozilla.org/show_bug.cgi?id=2016211 We have this for beta, if I understood you correctly
Comment 15•9 months ago
|
||
Thanks, Anna. Is the fix already in firefox-main?
The patch under bug 2016211 will need an uplift request so the relman can review and land it.
https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift
Also, :dveditz flagged that the fix is also needed in esr140 and maybe esr115.
Updated•9 months ago
|
| Assignee | ||
Updated•9 months ago
|
Updated•9 months ago
|
Comment 17•9 months ago
|
||
ESR140 builds on Monday 2026-02-16.
Anna, could you provide an uplift for ESR140 later today?
Comment 18•9 months ago
|
||
| Reporter | ||
Comment 20•9 months ago
|
||
(In reply to John Schanck [:jschanck] from comment #7)
Yes, although I'll add that it's already the case that a file that was encrypted on an arm64 build of Firefox will fail to decrypt on an x64 build of Firefox (and vice versa).
I agree that skipping the MAC check is not a suitable option. Fortunately, using a correct implementation, you can easily compute that tag that you would expect from an implementation with this bug. I'll attach some sample code based on your example. Note that I used Sonnet 4.5 to generate this. I only done a cursory review.
Just realized that as a companion to this fix, won't Firefox across all platforms need to have both a correct and an "incorrect" method of MAC check on decryption? Otherwise Intel and PPC builds will continue to not be able to decrypt >=512MB files uploaded by arm64 builds prior to 148 and arm64 builds >=148 will not be able to decrypt >= 512MB files uploaded by arm64 builds <148.
Updated•9 months ago
|
Updated•9 months ago
|
| Assignee | ||
Comment 21•9 months ago
|
||
(In reply to Clay Ver Valen from comment #20)
Just realized that as a companion to this fix, won't Firefox across all platforms need to have both a correct and an "incorrect" method of MAC check on decryption? Otherwise Intel and PPC builds will continue to not be able to decrypt >=512MB files uploaded by arm64 builds prior to 148 and arm64 builds >=148 will not be able to decrypt >= 512MB files uploaded by arm64 builds <148.
I'm not expecting significant breakage from this, and it's possible to work around the bug from the calling code. Here's an version of the code I posted in comment 6 that handles long associated data inputs as well. There's a comment in the fixupGCMTag function below that explains why it works.
function gfmulFull(X, Y) {
const Z = new Uint8Array(16);
const V = new Uint8Array(Y);
for (let i = 0; i < 128; i++) {
const byteIndex = Math.floor(i / 8);
const bitIndex = 7 - (i % 8);
const bitSet = (X[byteIndex] >> bitIndex) & 1;
if (bitSet) {
for (let j = 0; j < 16; j++) {
Z[j] ^= V[j];
}
}
const lsb = V[15] & 1;
for (let j = 15; j > 0; j--) {
V[j] = (V[j] >> 1) | ((V[j-1] & 1) << 7);
}
V[0] >>= 1;
if (lsb) {
V[0] ^= 0xE1;
}
}
return Z;
}
async function computeH(cek) {
// Compute H = E(K, 0) - the GHASH key
const key = await crypto.subtle.importKey(
'raw', cek, { name: 'AES-CTR' }, false, ['encrypt']
);
const zeroBlock = new Uint8Array(16);
const encrypted = await crypto.subtle.encrypt(
{ name: 'AES-CTR', counter: zeroBlock, length: 128 },
key,
new Uint8Array(16)
);
return new Uint8Array(encrypted);
}
function fixupGCMTag(tag, aLenBytes, cLenBytes, H) {
const aLenBits = aLenBytes * 8;
const cLenBits = cLenBytes * 8;
// GHASH(H, a, c) = L*H + other terms, where
// L = [ aLenBits || cLenBits ]
// and the terms are concatenated as 64-bit big-endian integers.
//
// We can equivalently write this as
// GHASH(H, a, c) = L0*H + L1*H + other terms, where
// L0 = [ low32(aLenBits) || low32(cLenBits) ]
// L1 = [ high32(aLenBits) || high32(cLenBits) ]
//
// If the input "tag" was mistakenly computed without the L1 term, due
// to bug 2009552, then
// GHASH(H, a, c) = tag + L1*H
//
// Likewise, since the addition is mod 2, if the input "tag" was computed
// correctly then "tag + L1*H" is equal to the tag that would have been
// produced with Bug 2009552.
const deltaL = new Uint8Array(16);
let deltaCLenBits = cLenBits - (cLenBits & 0xffffffff);
for (let i = 15; i >= 8; i--) {
deltaL[i] = deltaCLenBits & 0xFF;
deltaCLenBits = Math.floor(deltaCLenBits / 256);
}
let deltaALenBits = aLenBits - (aLenBits & 0xffffffff);
for (let i = 7; i >= 0; i--) {
deltaL[i] = deltaALenBits & 0xFF;
deltaALenBits = Math.floor(deltaALenBits / 256);
}
// Compute deltaL * H in GHASH field
const deltaLMultH = gfmulFull(deltaL, H);
const result = new Uint8Array(16);
for (let i = 0; i < 16; i++) {
result[i] = tag[i] ^ deltaLMultH[i];
}
return result;
}
async function decryptGCM(cek, ciphertext, associatedData) {
// ciphertext format: IV (12 bytes) || encrypted data || tag (16 bytes)
const iv = ciphertext.slice(0, 12);
const ctWithTag = ciphertext.slice(12);
const key = await crypto.subtle.importKey(
'raw', cek, { name: 'AES-GCM' }, false, ['decrypt']
);
try {
const plaintext = await crypto.subtle.decrypt(
{ name: 'AES-GCM', iv, additionalData: associatedData, tagLength: 128 },
key,
ctWithTag
);
return new Uint8Array(plaintext);
} catch (e) {
// Decryption failed - try with the tag fixed up for Bug 2009552
const cLenBytes = ctWithTag.length - 16;
const tag = ctWithTag.slice(cLenBytes);
const aLenBytes = associatedData.length;
const H = await computeH(cek);
const fixedTag = fixupGCMTag(tag, aLenBytes, cLenBytes, H);
const fixedCtWithTag = new Uint8Array(ctWithTag.length);
fixedCtWithTag.set(ctWithTag.slice(0, cLenBytes));
fixedCtWithTag.set(fixedTag, cLenBytes);
const plaintext = await crypto.subtle.decrypt(
{ name: 'AES-GCM', iv, additionalData: associatedData, tagLength: 128 },
key,
fixedCtWithTag
);
return new Uint8Array(plaintext);
}
}
Updated•9 months ago
|
Comment 22•9 months ago
|
||
Updated•9 months ago
|
Updated•9 months ago
|
Comment 23•9 months ago
|
||
Please keep hidden until 115 ESR EOL for Tor Browser 13.5 Legacy.
Comment 24•8 months ago
|
||
CCing anonym for Tails assessment.
Updated•8 months ago
|
Updated•7 months ago
|
Comment 25•7 months ago
|
||
Updated•7 months ago
|
Updated•7 months ago
|
Updated•7 months ago
|
Updated•5 months ago
|
Description
•