Closed Bug 2009552 (CVE-2026-2781) Opened 10 months ago Closed 9 months ago

AES-GCM produces incorrect authentication tags on some platforms for data >= 512 MB due to 32-bit integer overflow in length computation

Categories

(NSS :: Libraries, defect, P2)

ARM64
Unspecified

Tracking

(firefox-esr115150+ fixed, firefox-esr140148+ fixed, firefox147 wontfix, firefox148+ fixed, firefox149+ fixed)

RESOLVED FIXED
Tracking Status
firefox-esr115 150+ fixed
firefox-esr140 148+ fixed
firefox147 --- wontfix
firefox148 + fixed
firefox149 + fixed

People

(Reporter: clayvv, Assigned: jschanck)

References

Details

(Keywords: csectype-intoverflow, reporter-external, sec-moderate, Whiteboard: [client-bounty-form][adv-main148+][adv-esr140.8+][adv-esr115.35+])

Attachments

(2 files, 1 external file)

Firefox WebCrypto AES-GCM Bug Report

Summary

WebCrypto AES-GCM produces incorrect authentication tags for data >= 512 MB due to 32-bit integer overflow in length computation

Affected Versions

  • Tested on: Firefox (current version as of January 2026)
  • Component: Core :: Security: PSM (NSS)

Description

Firefox's WebCrypto implementation of AES-GCM produces incorrect authentication tags when encrypting or decrypting data >= 512 MB (536,870,912 bytes). This causes interoperability failures with all other AES-GCM implementations including Chrome, OpenSSL, Java (SunJCE and BouncyCastle), and Node.js.

The bug appears to be a 32-bit integer overflow in the GCM length block computation. The GCM specification (NIST SP 800-38D) requires 64-bit length fields, but Firefox/NSS appears to use 32-bit arithmetic. When the ciphertext length in bits exceeds 2^32 (which occurs at 512 MB = 2^32 bits / 8), the length wraps around, producing an incorrect GHASH and thus an incorrect authentication tag.

Steps to Reproduce

  1. Open the attached firefox-gcm-bug-test.html in Firefox
  2. Click "Run Tests (10MB - 1024MB)"
  3. Compare the computed tags with the reference values shown on the page
  4. Repeat in Chrome for comparison

Expected Results

Firefox should produce identical tags to Chrome and server-side implementations at all sizes:

Size Expected Tag (Chrome/OpenSSL/Java)
10 MB E0AB33498A9A54A93796DEDC62763441
50 MB 1A213F7F4F53ADF640B03A4E99B859AE
100 MB 9D6D3B30C437D33FCA1C0FFBA7EDF3C1
200 MB 7F4DFED7961C3C5359AC9C3C93409708
512 MB 513254EA22DA5846ABD0BD873BD53F2B
1024 MB A85365DB36D9D1B5298F7E6C98EB9839

Actual Results

Firefox matches expected values for < 512 MB but diverges at >= 512 MB:

Size Firefox Tag Matches Expected?
10 MB E0AB33498A9A54A93796DEDC62763441 ✓ YES
50 MB 1A213F7F4F53ADF640B03A4E99B859AE ✓ YES
100 MB 9D6D3B30C437D33FCA1C0FFBA7EDF3C1 ✓ YES
200 MB 7F4DFED7961C3C5359AC9C3C93409708 ✓ YES
512 MB DA4BA5991D2051E467401216A907A3D4 ✗ NO ← Diverges here
1024 MB 7CA0873D492DC2F0B0AE214FBD4EA1C6 ✗ NO

Chrome produces correct tags at ALL sizes, confirming this is Firefox-specific.

Impact

Summary

  1. Files >= 512 MB encrypted by Firefox cannot be decrypted by any other implementation
  2. Files >= 512 MB encrypted by other implementations cannot be decrypted by Firefox
  3. Firefox-to-Firefox works (both use same buggy computation)
  4. This breaks interoperability for any application using WebCrypto AES-GCM with large files

Real-world consequences for applications using WebCrypto AES-GCM

  1. AES-GCM MAC verification cannot be reliably used for files >= 512 MB

    • Applications cannot know which browser uploaded a file
    • If ANY chance exists that a file was uploaded by Firefox, MAC verification must be skipped
    • This effectively defeats the authentication guarantee of AES-GCM for large files
  2. Interoperability is broken bidirectionally

    • Files >= 512 MB encrypted by Firefox cannot be verified by Chrome, server-side code, or any standards-compliant implementation
    • Files >= 512 MB encrypted by other implementations cannot be verified by Firefox
  3. Data loss risk when this bug is fixed

    • Files currently uploaded by Firefox with size >= 512 MB have incorrect MAC tags stored
    • Once Mozilla fixes this bug, those files will become unverifiable/undownloadable by Firefox
    • Applications must implement fallback decryption (e.g., AES-CTR) to maintain access to legacy Firefox-uploaded files
  4. Security implications

    • Applications are forced to either:
      • (a) Skip MAC verification for large files (losing authentication guarantee), or
      • (b) Block large file uploads entirely, or
      • (c) Block Firefox users from uploading large files
    • None of these options are acceptable for production applications

Technical Analysis

The GCM authentication tag is computed using GHASH over:

  • Additional Authenticated Data (AAD)
  • Ciphertext
  • A length block containing: [len(AAD) in bits || len(ciphertext) in bits]

The length block must use 64-bit big-endian integers per NIST SP 800-38D.

At 512 MB:

  • Ciphertext = 536,870,912 bytes
  • Length in bits = 4,294,967,296 = 2^32 exactly
  • If using 32-bit arithmetic, this wraps to 0

This explains why Firefox diverges at exactly the 512 MB boundary.

Test Parameters

All tests use identical parameters:

Parameter Value
Algorithm AES-256-GCM
Key (hex) F7F698D851DE2EB343CAB95BC3D498E4A6E665DB8D93CE278E595371B4B66594
IV (hex) 4A0F0A225AE4FB34555FC51C
AAD 2026-01-09T19:23:00.000Z (UTF-8 encoded)
Plaintext Zero-filled buffer of specified size
Tag length 128 bits

Attachments

  • firefox-gcm-bug-test.html - Self-contained test page that can be run in any browser

References

  • NIST SP 800-38D: Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC
  • Section 7.1 specifies 64-bit length fields in the GHASH computation
Flags: sec-bounty?

I did not mention above, but the tests I did with the attached HTML were only with Chrome and Firefox on a Mac running Sequoia 15.7.3

I believe the problem is in security/nss/lib/freebl/gcm.c on line 360

Buggy code:
ghash->cLen += (len * PR_BITS_PER_BYTE);

The parameter len is declared as unsigned int (32-bit) in the function signature:
SECStatus gcmHash_Update(gcmHashContext *ghash, const unsigned char *buf, unsigned int len)

When len >= 536,870,912 bytes (512 MB), the multiplication len * 8 overflows
the 32-bit unsigned int (2^32 = 4,294,967,296) before being added to the 64-bit cLen.

This causes the GHASH length block to contain an incorrect bit count, producing
an invalid authentication tag for any data >= 512 MB.

If you agree, the fastest fix with minimal changes would be to cast len to uint64_t before the multiplication:
ghash->cLen += ((uint64_t)len * PR_BITS_PER_BYTE);

A more comprehensive fix that would maintain API consistentency might be to change the function signature of gcmHash_Update so that len is declared as a uint64_t.

Group: firefox-core-security → dom-core-security
Component: Security → DOM: Web Crypto
Product: Firefox → Core
Group: dom-core-security → crypto-core-security

Thank you for this detailed report!

This primarily affects arm64. We use a different GHASH implementation on Intel and PPC.

Assignee: nobody → jschanck
Severity: -- → S2
Status: UNCONFIRMED → ASSIGNED
Component: DOM: Web Crypto → Libraries
Ever confirmed: true
Priority: -- → P2
Product: Core → NSS
Summary: WebCrypto AES-GCM produces incorrect authentication tags for data >= 512 MB due to 32-bit integer overflow in length computation → AES-GCM produces incorrect authentication tags on some platforms for data >= 512 MB due to 32-bit integer overflow in length computation
Attached file (secure) —

@:jschanck - Once the fix has been made though, any files >=512MB in size previously uploaded via Firefox will not pass the MAC verification even if being downloaded with Firefox, right?

This is particularly important for me as I am a Technical Leader at Cisco on the Webex team and this will have a significant impact on us as we allow files of up to 2GB in size to be attached to messages and support using a browser (https://web.webex.com).

As this potentially impacts any web service that does AES-GCM encryption of large file uploads (including in the FedRAMP environment), a MAC check of these large "legacy" files cannot relied upon and skipping the MAC check is not suitable an option either.

Attached file bug2009552.html —

Yes, although I'll add that it's already the case that a file that was encrypted on an arm64 build of Firefox will fail to decrypt on an x64 build of Firefox (and vice versa).

I agree that skipping the MAC check is not a suitable option. Fortunately, using a correct implementation, you can easily compute that tag that you would expect from an implementation with this bug. I'll attach some sample code based on your example. Note that I used Sonnet 4.5 to generate this. I only done a cursory review.

We are shipping 147 today so that's a 'wontfix' for practical reasons, but we should try to get a fix in 148 and the corresponding ESR-140. I doubt this is worth fixing on ESR-115 for the dwindling number of stranded Win7 users.

Although integer overflows can lead to security bugs (e.g. overflowing unexpectedly small buffers), this seems to be a correctness/interoperability problem rather than a security issue.

Contrary to what I said in our meeting yesterday, Dan, there is a security issue here. I missed this earlier because the variable is called cLen, but the integer overflow affects the calculation of the length of the associated data as well.

Here's why that matters. If we view GHASH(H, a, c) as a polynomial in H, then the trailing coefficient is supposed to be len(a) || len(c). This bug causes NSS to use len(a) mod 2^32 || len(c) mod 2^32 instead. The security proof for GHASH (page 14 of https://eprint.iacr.org/2004/193.pdf) uses the fact that the tuple (a, c) can be extracted unambiguously from the polynomial GHASH(H, a, c) (it can because the lengths of a and c are encoded in the trailing coefficient). Reducing the lengths mod 2^32 introduces ambiguity: you can move a 512 MB prefix of c to a suffix of a and vice versa.

The upshot is that if an attacker observes a GCM ciphertext (iv, a, c, tag) where a is multiple of 16 bytes in length and c is more than 512 MB in length, then he can write down a second valid ciphertext (iv, a || c0, c1, tag) where c = c0 || c1 and c0 is (some multiple of) 512 MB. In crypto jargon the fact that this new ciphertext decrypts without an authentication failure is a violation of the "INT-CTXT" property of the AEAD scheme.

If the bug had only affected len(c) there would be no issue, as len(a) || len(c) mod 2^32 is still enough information to extract (a, c) from GHASH(H, a, c).

There may not be a real world security impact from this bug (the forged ciphertext will decrypt to a "random" plaintext). But we can't write this off as "just" an interop bug.

Keywords: sec-moderate

:jschanck, there are only a couple of weeks left in beta for Fx148.
What's the plan for landing this and adding uplift requests for the affected versions?

Flags: needinfo?(jschanck)

Pushed by jschanck@mozilla.com:
https://hg.mozilla.org/projects/nss/rev/245385e16fa6
avoid integer overflow in platform-independent ghash. r=nss-reviewers,nkulatova

Status: ASSIGNED → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Pushed by nkulatova@mozilla.com: https://hg.mozilla.org/projects/nss/rev/e3ce261ac258 avoid integer overflow in platform-independent ghash. r=nss-reviewers,nkulatova
Group: crypto-core-security
Group: core-security-release

:jschanck, the last Fx148 beta builds, so we'll need uplift requests before then

Flags: needinfo?(jschanck)
Flags: needinfo?(jschanck)

https://bugzilla.mozilla.org/show_bug.cgi?id=2016211 We have this for beta, if I understood you correctly

Thanks, Anna. Is the fix already in firefox-main?

The patch under bug 2016211 will need an uplift request so the relman can review and land it.
https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift

Also, :dveditz flagged that the fix is also needed in esr140 and maybe esr115.

Flags: needinfo?(anna.weine)

Esr140 for sure

Flags: needinfo?(anna.weine)
Flags: needinfo?(jschanck)
QA Whiteboard: [sec] [qa-triage-done-c149/b148]

ESR140 builds on Monday 2026-02-16.
Anna, could you provide an uplift for ESR140 later today?

Flags: needinfo?(anna.weine)
Pushed by nkulatova@mozilla.com: https://hg.mozilla.org/projects/nss/rev/e040897557cf avoid integer overflow in platform-independent ghash. r=nss-reviewers,nkulatova

Done

Flags: needinfo?(anna.weine)

(In reply to John Schanck [:jschanck] from comment #7)

Yes, although I'll add that it's already the case that a file that was encrypted on an arm64 build of Firefox will fail to decrypt on an x64 build of Firefox (and vice versa).

I agree that skipping the MAC check is not a suitable option. Fortunately, using a correct implementation, you can easily compute that tag that you would expect from an implementation with this bug. I'll attach some sample code based on your example. Note that I used Sonnet 4.5 to generate this. I only done a cursory review.

Just realized that as a companion to this fix, won't Firefox across all platforms need to have both a correct and an "incorrect" method of MAC check on decryption? Otherwise Intel and PPC builds will continue to not be able to decrypt >=512MB files uploaded by arm64 builds prior to 148 and arm64 builds >=148 will not be able to decrypt >= 512MB files uploaded by arm64 builds <148.

Depends on: 2016717
Whiteboard: [client-bounty-form] → [client-bounty-form][adv-main148+]

(In reply to Clay Ver Valen from comment #20)

Just realized that as a companion to this fix, won't Firefox across all platforms need to have both a correct and an "incorrect" method of MAC check on decryption? Otherwise Intel and PPC builds will continue to not be able to decrypt >=512MB files uploaded by arm64 builds prior to 148 and arm64 builds >=148 will not be able to decrypt >= 512MB files uploaded by arm64 builds <148.

I'm not expecting significant breakage from this, and it's possible to work around the bug from the calling code. Here's an version of the code I posted in comment 6 that handles long associated data inputs as well. There's a comment in the fixupGCMTag function below that explains why it works.

function gfmulFull(X, Y) {
    const Z = new Uint8Array(16);
    const V = new Uint8Array(Y);

    for (let i = 0; i < 128; i++) {
        const byteIndex = Math.floor(i / 8);
        const bitIndex = 7 - (i % 8);
        const bitSet = (X[byteIndex] >> bitIndex) & 1;

        if (bitSet) {
            for (let j = 0; j < 16; j++) {
                Z[j] ^= V[j];
            }
        }

        const lsb = V[15] & 1;
        for (let j = 15; j > 0; j--) {
            V[j] = (V[j] >> 1) | ((V[j-1] & 1) << 7);
        }
        V[0] >>= 1;

        if (lsb) {
            V[0] ^= 0xE1;
        }
    }

    return Z;
}

async function computeH(cek) {
    // Compute H = E(K, 0) - the GHASH key
    const key = await crypto.subtle.importKey(
        'raw', cek, { name: 'AES-CTR' }, false, ['encrypt']
    );
    const zeroBlock = new Uint8Array(16);
    const encrypted = await crypto.subtle.encrypt(
        { name: 'AES-CTR', counter: zeroBlock, length: 128 },
        key,
        new Uint8Array(16)
    );
    return new Uint8Array(encrypted);
}

function fixupGCMTag(tag, aLenBytes, cLenBytes, H) {
    const aLenBits = aLenBytes * 8;
    const cLenBits = cLenBytes * 8;

    // GHASH(H, a, c) = L*H + other terms, where
    // L = [ aLenBits || cLenBits ]
    // and the terms are concatenated as 64-bit big-endian integers.
    //
    // We can equivalently write this as
    // GHASH(H, a, c) = L0*H + L1*H + other terms, where
    // L0 = [ low32(aLenBits) || low32(cLenBits) ]
    // L1 = [ high32(aLenBits) || high32(cLenBits) ]
    //
    // If the input "tag" was mistakenly computed without the L1 term, due
    // to bug 2009552, then
    // GHASH(H, a, c) = tag + L1*H
    //
    // Likewise, since the addition is mod 2, if the input "tag" was computed
    // correctly then "tag + L1*H" is equal to the tag that would have been
    // produced with Bug 2009552.

    const deltaL = new Uint8Array(16);

    let deltaCLenBits = cLenBits - (cLenBits & 0xffffffff);
    for (let i = 15; i >= 8; i--) {
        deltaL[i] = deltaCLenBits & 0xFF;
        deltaCLenBits = Math.floor(deltaCLenBits / 256);
    }

    let deltaALenBits = aLenBits - (aLenBits & 0xffffffff);
    for (let i = 7; i >= 0; i--) {
        deltaL[i] = deltaALenBits & 0xFF;
        deltaALenBits = Math.floor(deltaALenBits / 256);
    }

    // Compute deltaL * H in GHASH field
    const deltaLMultH = gfmulFull(deltaL, H);

    const result = new Uint8Array(16);
    for (let i = 0; i < 16; i++) {
        result[i] = tag[i] ^ deltaLMultH[i];
    }
    return result;
}

async function decryptGCM(cek, ciphertext, associatedData) {
    // ciphertext format: IV (12 bytes) || encrypted data || tag (16 bytes)
    const iv = ciphertext.slice(0, 12);
    const ctWithTag = ciphertext.slice(12);

    const key = await crypto.subtle.importKey(
        'raw', cek, { name: 'AES-GCM' }, false, ['decrypt']
    );

    try {
        const plaintext = await crypto.subtle.decrypt(
            { name: 'AES-GCM', iv, additionalData: associatedData, tagLength: 128 },
            key,
            ctWithTag
        );
        return new Uint8Array(plaintext);
    } catch (e) {
        // Decryption failed - try with the tag fixed up for Bug 2009552
        const cLenBytes = ctWithTag.length - 16;
        const tag = ctWithTag.slice(cLenBytes);
        const aLenBytes = associatedData.length;

        const H = await computeH(cek);
        const fixedTag = fixupGCMTag(tag, aLenBytes, cLenBytes, H);

        const fixedCtWithTag = new Uint8Array(ctWithTag.length);
        fixedCtWithTag.set(ctWithTag.slice(0, cLenBytes));
        fixedCtWithTag.set(fixedTag, cLenBytes);

        const plaintext = await crypto.subtle.decrypt(
            { name: 'AES-GCM', iv, additionalData: associatedData, tagLength: 128 },
            key,
            fixedCtWithTag
        );
        return new Uint8Array(plaintext);
    }
}

Whiteboard: [client-bounty-form][adv-main148+] → [client-bounty-form][adv-main148+] [adv-esr140.8+]
Pushed by nkulatova@mozilla.com: https://hg.mozilla.org/projects/nss/rev/837db60023f6 avoid integer overflow in platform-independent ghash. r=nss-reviewers,nkulatova
Alias: CVE-2026-2781
Flags: sec-bounty? → sec-bounty+

Please keep hidden until 115 ESR EOL for Tor Browser 13.5 Legacy.

Whiteboard: [client-bounty-form][adv-main148+] [adv-esr140.8+] → [keep hidden until ESR-115 EOL] [client-bounty-form][adv-main148+] [adv-esr140.8+]

CCing anonym for Tails assessment.

Keywords: keep-hidden
Blocks: 2031176
Pushed by jschanck@mozilla.com: https://hg.mozilla.org/projects/nss/rev/46be3b4a46f5 avoid integer overflow in platform-independent ghash. r=#nss-reviewers
Whiteboard: [keep hidden until ESR-115 EOL] [client-bounty-form][adv-main148+] [adv-esr140.8+] → [keep hidden until ESR-115 EOL] [client-bounty-form][adv-main148+] [adv-esr140.8+][adv-esr115.35+]
Keywords: keep-hidden
Whiteboard: [keep hidden until ESR-115 EOL] [client-bounty-form][adv-main148+] [adv-esr140.8+][adv-esr115.35+] → [client-bounty-form][adv-main148+][adv-esr140.8+][adv-esr115.35+]
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: