2027326 - Out-of-bounds write in [@ ssl3_HandleCertificateCompressionXtn] via repeated TLS 1.3 Post-Handshake Authentication without numNegotiated reset

Status: RESOLVED DUPLICATE of bug 2026089

(NSS :: Libraries, defect)

Description

[Bugmon [:jkratzer for issues]]

Summary

Intra-object heap buffer overflow in NSS. The fixed-size array TLSExtensionData::negotiated[SSL_MAX_EXTENSIONS] (22 elements, ssl3ext.h:45) is written with an unchecked numNegotiated++ index at 29 locations in NSS. numNegotiated is reset only by ssl3_ResetExtensionData, which is called at ClientHello time and socket init — but not in the TLS 1.3 Post-Handshake Authentication handler tls13_HandleCertificateRequest. A malicious server can send unlimited PHA CertificateRequest messages, each carrying signature_algorithms + compress_certificate extensions that increment numNegotiated by 2 without bound, eventually overwriting adjacent pointer fields (psk_ke_modes.data, sniNameArr, sigSchemes, etc.) inside the heap-allocated sslSocket.

Requires non-default pref security.tls.enable_post_handshake_auth = true. Crash occurs in the parent process Socket Thread.

Affected Code

File: security/nss/lib/ssl/ssl3ext.h, lines 44-45

struct TLSExtensionDataStr {
    ...
    PRUint16 numNegotiated;
    PRUint16 negotiated[SSL_MAX_EXTENSIONS];   /* SSL_MAX_EXTENSIONS = 22, sslt.h:578 */

    /* Fields that get corrupted by the overflow: */
    PRBool ticketTimestampVerified;
    PRBool emptySessionTicket;
    PRBool sentSessionTicketInClientHello;
    SECItem psk_ke_modes;                       /* contains pointer */
    PRUint32 max_early_data_size;
    SECItem *sniNameArr;                        /* POINTER */
    PRUint32 sniNameArrSize;
    SECItem signedCertTimestamps;               /* contains pointer */
    PRBool peerSupportsFfdheGroups;
    SSLSignatureScheme *sigSchemes;             /* POINTER — freed/realloced every PHA round */
    unsigned int numSigSchemes;
    ...
};

File: security/nss/lib/ssl/tls13exthandle.c, line 2013 (UBSan trip point)

for (int j = 0; j < ss->ssl3.supportedCertCompressionAlgorithmsCount; j++) {
    if (ss->ssl3.supportedCertCompressionAlgorithms[j].id == alg) {
        xtnData->compressionAlg = alg;
        xtnData->negotiated[xtnData->numNegotiated++] = ssl_certificate_compression_xtn;  /* NO BOUNDS CHECK */
        algFound = SECSuccess;
        break;
    }
}

File: security/nss/lib/ssl/ssl3exthandle.c, line 1642 (second write per PHA round)

/* Keep track of negotiated extensions. */
xtnData->negotiated[xtnData->numNegotiated++] = ssl_signature_algorithms_xtn;  /* NO BOUNDS CHECK */
return SECSuccess;

File: security/nss/lib/ssl/tls13con.c, lines 3283-3349 — the root cause

if (tls13_IsPostHandshake(ss)) {
    ...
    /* clean up anything left from previous handshake. */
    if (ss->ssl3.clientCertChain != NULL) { ... }
    if (ss->ssl3.clientCertificate != NULL) { ... }
    if (ss->ssl3.clientPrivateKey != NULL) { ... }
    if (ss->ssl3.hs.clientAuthSignatureSchemes != NULL) { ... }
    SECITEM_FreeItem(&ss->xtnData.certReqContext, PR_FALSE);
    ss->xtnData.certReqContext.data = NULL;
    /* BUG: ss->xtnData.numNegotiated is NOT reset here */
}
...
/* Process all the extensions. */
rv = ssl3_HandleExtensions(ss, &extensionsData.data, &extensionsData.len,
                           ssl_hs_certificate_request);  /* increments numNegotiated */

The cleanup block frees per-request certificate state but forgets to reset the extension counter. ssl3_HandleExtensions then dispatches to certificateRequestHandlers which contains three handlers; two of them (ssl3_HandleSigAlgsXtn, ssl3_HandleCertificateCompressionXtn) perform the unchecked write.

ssl3_ResetExtensionData (which zeros numNegotiated) is only called at:

• ssl3con.c:5541 — sending ClientHello
• ssl3con.c:9314 — server handling ClientHello (renegotiation)
• ssl3con.c:14043 — socket init
• sslsecur.c:205

None of these are on the PHA code path.

Correct Pattern

if (tls13_IsPostHandshake(ss)) {
    ...
    SECITEM_FreeItem(&ss->xtnData.certReqContext, PR_FALSE);
    ss->xtnData.certReqContext.data = NULL;

    /* Reset negotiated-extension tracking for this PHA round. */
    ss->xtnData.numNegotiated = 0;
}

Additionally, every negotiated[numNegotiated++] site should defensively guard:

if (xtnData->numNegotiated < SSL_MAX_EXTENSIONS) {
    xtnData->negotiated[xtnData->numNegotiated++] = ssl_certificate_compression_xtn;
}

Exploit Chain

1. Victim has security.tls.enable_post_handshake_auth = true enabled (e.g. for enterprise smart-card auth).
2. Victim navigates to an attacker-controlled page (or any page with attacker subresource).
3. Page issues fetch('https://attacker:port/') — no user interaction, no-cors mode sufficient.
4. Firefox connects to attacker's TLS 1.3 server on the parent-process Socket Thread.
5. Client sends ClientHello with post_handshake_auth extension (because pref is enabled).
6. Server completes TLS 1.3 handshake and negotiates ALPN h2. Firefox calls nsHttpConnection::StartSpdy → SetDenyClientCert(true), so all PHA requests are answered synchronously with an empty Certificate (no user dialog).
7. At handshake completion, numNegotiated holds some value N ≤ 22 from the main handshake's negotiated extensions.
8. Server sends PHA CertificateRequest #1 with signature_algorithms + compress_certificate extensions → tls13_HandleCertificateRequest → ssl3_HandleExtensions → both handlers fire → numNegotiated += 2.
9. Server repeats step 8. After (22 - N) / 2 + 1 rounds, numNegotiated reaches 22 and the next write is OOB.
10. UBSan halts at tls13exthandle.c:2013 with index 22. Without UBSan, writes continue into adjacent TLSExtensionData fields. The server observed 80 rounds = up to ~160 OOB PRUint16 writes (320 bytes past array end), reaching sigSchemes and beyond.
11. Written values are extension type codes: 13 (ssl_signature_algorithms_xtn) and 27 (ssl_certificate_compression_xtn). On little-endian, consecutive writes form patterns like 0x001b000d001b000d.
12. Corrupted sigSchemes pointer + numSigSchemes count is dereferenced on the next PHA round by ssl3_HandleSigAlgsXtn → ssl_ParseSignatureSchemes (realloc/free of attacker-influenced pointer), yielding a write-what-where primitive or controlled free.

Steps to Reproduce

1. Install tlslite-ng: pip install tlslite-ng
2. Generate a CA and server cert for 127.0.0.1 (or use the provided cert.pem/key.pem).
3. Install the CA cert into Firefox via autoconfig:
   • $FIREFOX_DIR/autoconfig.cfg: certdb.addCertFromBase64(caCertBase64, "C,,");
   • $FIREFOX_DIR/defaults/pref/autoconfig.js: pref("general.config.filename", "autoconfig.cfg"); pref("general.config.obscure_value", 0);
4. Start the malicious server: PORT=14436 python3 pha_overflow_server.py
5. Set pref security.tls.enable_post_handshake_auth = true
6. Load test.html which does fetch('https://127.0.0.1:14436/')

Note: In the real world, the attacker uses a valid cert from a public CA — autoconfig is only needed here because the test server is on localhost.

Security Impact

Severity: High (sec-high) — heap memory corruption in the parent process, triggerable from web content, gated by non-default pref.

Attacker capability:

• Intra-object heap overflow with unlimited repetition (server controls PHA round count; numNegotiated is PRUint16 so up to 65535 writes = ~128 KiB of linear overwrite from &negotiated[22]).
• Written values are partially attacker-chosen: the server picks which extensions to include, controlling which extension-type constants (small 16-bit values from the SSLExtensionType enum) are written and in which order.
• Corrupted fields include raw pointers (sniNameArr, sigSchemes, delegCredSigSchemes) and length fields (numSigSchemes, sniNameArrSize) that are freed/dereferenced by subsequent PHA rounds on the same connection.
• Low-address pointer values combined with heap spray give a plausible path to RCE in the parent process (sandbox escape not required — network stack is already in the parent).

Preconditions:

• security.tls.enable_post_handshake_auth = true (default false, StaticPrefList.yaml:18077-18079). This pref exists for enterprise smart-card / client-cert deployments and is documented in support articles.
• Victim visits any page containing a subresource request to attacker's HTTPS server (a single <img>, fetch(), or <script> is sufficient; no-cors works).
• No user interaction required (HTTP/2 ALPN makes Firefox auto-decline each PHA with an empty cert).

ASAN Report

/firefox/security/nss/lib/ssl/tls13exthandle.c:2013:17: runtime error: index 22 out of bounds for type 'PRUint16[22]' (aka 'unsigned short[22]')
    #0 0x7735ea130fa8 in ssl3_HandleCertificateCompressionXtn /firefox/security/nss/lib/ssl/tls13exthandle.c:2013:63
    #1 0x7735ea0ce7f5 in ssl_CallExtensionHandler /firefox/security/nss/lib/ssl/ssl3ext.c:447:22
    #2 0x7735ea0ce7f5 in ssl3_HandleParsedExtensions /firefox/security/nss/lib/ssl/ssl3ext.c:596:14
    #3 0x7735ea0cec5e in ssl3_HandleExtensions /firefox/security/nss/lib/ssl/ssl3ext.c:617:10
    #4 0x7735ea10dea0 in tls13_HandleCertificateRequest /firefox/security/nss/lib/ssl/tls13con.c:3349:10
    #5 0x7735ea10dea0 in tls13_HandlePostHelloHandshakeMessage /firefox/security/nss/lib/ssl/tls13con.c:1452:20
    #6 0x7735ea0b5dfd in ssl3_HandleHandshakeMessage /firefox/security/nss/lib/ssl/ssl3con.c:12916:22
    #7 0x7735ea0bb187 in ssl3_HandleHandshake /firefox/security/nss/lib/ssl/ssl3con.c:13091:18
    #8 0x7735ea0bb187 in ssl3_HandleNonApplicationData /firefox/security/nss/lib/ssl/ssl3con.c:13626:22
    #9 0x7735ea0bd706 in ssl3_HandleRecord /firefox/security/nss/lib/ssl/ssl3con.c:13990:10
    #10 0x7735ea0d94e0 in ssl3_GatherCompleteHandshake /firefox/security/nss/lib/ssl/ssl3gthr.c:560:18
    #11 0x7735ea0da29a in ssl3_GatherAppDataRecord /firefox/security/nss/lib/ssl/ssl3gthr.c:654:14
    #12 0x7735ea0ecca4 in DoRecv /firefox/security/nss/lib/ssl/sslsecur.c:538:14
    #13 0x7735ea0ecca4 in ssl_SecureRecv /firefox/security/nss/lib/ssl/sslsecur.c:892:10
    #14 0x7735ea1052ce in ssl_Recv /firefox/security/nss/lib/ssl/sslsock.c:3250:10
    #15 0x7735d1c77ef8 in PSMRecv(PRFileDesc*, void*, int, int, unsigned int) /firefox/security/manager/ssl/nsNSSIOLayer.cpp:828:7
    #16 0x7735c3be7db7 in mozilla::net::nsSocketInputStream::Read(char*, unsigned int, unsigned int*) /firefox/netwerk/base/nsSocketTransport2.cpp:406:15
    #17 0x7735c49b2ae3 in mozilla::net::nsHttpConnection::OnWriteSegment(char*, unsigned int, unsigned int*) /firefox/netwerk/protocol/http/nsHttpConnection.cpp:1840:28
    #18 0x7735c49b2ae3 in non-virtual thunk to mozilla::net::nsHttpConnection::OnWriteSegment(char*, unsigned int, unsigned int*) /firefox/netwerk/protocol/http/nsHttpConnection.cpp:0:0
    #19 0x7735c4629072 in mozilla::net::Http2Session::NetworkRead(mozilla::net::nsAHttpSegmentWriter*, char*, unsigned int, unsigned int*) /firefox/netwerk/protocol/http/Http2Session.cpp:719:25
    #20 0x7735c4629072 in mozilla::net::Http2Session::WriteSegmentsAgain(mozilla::net::nsAHttpSegmentWriter*, unsigned int, unsigned int*, bool*) /firefox/netwerk/protocol/http/Http2Session.cpp:2910:10
    #21 0x7735c49b350f in mozilla::net::nsHttpConnection::OnSocketReadable() /firefox/netwerk/protocol/http/nsHttpConnection.cpp:1911:25
    #22 0x7735c49b6906 in mozilla::net::nsHttpConnection::OnInputStreamReady(nsIAsyncInputStream*) /firefox/netwerk/protocol/http/nsHttpConnection.cpp:2259:17
    #23 0x7735c3be6759 in mozilla::net::nsSocketInputStream::OnSocketReady(nsresult) /firefox/netwerk/base/nsSocketTransport2.cpp:302:27
    #24 0x7735c3bf851a in mozilla::net::nsSocketTransport::OnSocketReady(PRFileDesc*, short) /firefox/netwerk/base/nsSocketTransport2.cpp:2194:15
    #25 0x7735c3c0d7e8 in mozilla::net::nsSocketTransportService::DoPollIteration() /firefox/netwerk/base/nsSocketTransportService2.cpp:1447:21
    #26 0x7735c3c0b558 in mozilla::net::nsSocketTransportService::Run() /firefox/netwerk/base/nsSocketTransportService2.cpp:1208:7
    #27 0x7735c3c0f1ac in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /firefox/netwerk/base/nsSocketTransportService2.cpp:0:0
    #28 0x7735c378b1ba in nsThread::ProcessNextEvent(bool, bool*) /firefox/xpcom/threads/nsThread.cpp:1173:16
    #29 0x7735c3793d39 in NS_ProcessNextEvent(nsIThread*, bool) /firefox/xpcom/threads/nsThreadUtils.cpp:465:10
    #30 0x7735c51e5da1 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /firefox/ipc/glue/MessagePump.cpp:297:20
    #31 0x7735c5029374 in MessageLoop::RunInternal() /firefox/ipc/chromium/src/base/message_loop.cc:371:10
    #32 0x7735c5029374 in MessageLoop::RunHandler() /firefox/ipc/chromium/src/base/message_loop.cc:364:3
    #33 0x7735c5029374 in MessageLoop::Run() /firefox/ipc/chromium/src/base/message_loop.cc:346:3
    #34 0x7735c3783680 in nsThread::ThreadFunc(void*) /firefox/xpcom/threads/nsThread.cpp:374:10
    #35 0x7b35ef7db88f in _pt_root /firefox/nsprpub/pr/src/pthreads/ptthread.c:191:3
    #36 0x5cb2635cbb86 in asan_thread_start(void*) _asan_rtl_:28
    #37 0x7b35efd6faa3 in pthread_condattr_setpshared ??:?
    #38 0x7b35efdfca63 in clone ??:0:0

SUMMARY: UndefinedBehaviorSanitizer: out-of-bounds-index /firefox/security/nss/lib/ssl/tls13exthandle.c:2013:17

Suggested Fix

Primary fix — reset numNegotiated at the start of each PHA round in security/nss/lib/ssl/tls13con.c:

--- a/security/nss/lib/ssl/tls13con.c
+++ b/security/nss/lib/ssl/tls13con.c
@@ -3312,6 +3312,10 @@ tls13_HandleCertificateRequest(sslSocket *ss, PRUint8 *b, PRUint32 length)
             ss->ssl3.hs.clientAuthSignatureSchemesLen = 0;
         }
         SECITEM_FreeItem(&ss->xtnData.certReqContext, PR_FALSE);
         ss->xtnData.certReqContext.data = NULL;
+
+        /* Reset negotiated-extension tracking so repeated PHA rounds do not
+         * accumulate entries in the fixed-size negotiated[] array. */
+        ss->xtnData.numNegotiated = 0;
     } else {

Defense-in-depth — add a bounds-checked helper and convert all 29 call sites:

/* security/nss/lib/ssl/ssl3ext.c */
void
ssl3_RecordExtensionNegotiated(TLSExtensionData *xtnData, PRUint16 ex_type)
{
    if (xtnData->numNegotiated >= SSL_MAX_EXTENSIONS) {
        PORT_Assert(0); /* should never happen after the PHA reset fix */
        return;
    }
    xtnData->negotiated[xtnData->numNegotiated++] = ex_type;
}

Then replace every xtnData->negotiated[xtnData->numNegotiated++] = X; with ssl3_RecordExtensionNegotiated(xtnData, X);.

Comment 1

[Bugmon [:jkratzer for issues]]

Attachment: test.html

Comment 2

[Bugmon [:jkratzer for issues]]

Attachment: prefs.json

Comment 3

[Bugmon [:jkratzer for issues]]

Attachment: pha_overflow_server.py

Comment 4

[Bugmon [:jkratzer for issues]]

Attachment: key.pem

Comment 5

[Bugmon [:jkratzer for issues]]

Attachment: gen_certs.sh

Comment 6

[Bugmon [:jkratzer for issues]]

Attachment: crash_stack.txt

Comment 7

[Bugmon [:jkratzer for issues]]

Attachment: cert.pem

Comment 8

[Bugmon [:jkratzer for issues]]

Attachment: autoconfig.js

Comment 9

[Bugmon [:jkratzer for issues]]

Attachment: ca-cert.pem

Comment 10

[Bugmon [:jkratzer for issues]]

Attachment: autoconfig.cfg

Comment 11

[Bugmon [:jkratzer for issues]]

Attachment: README.md

Comment 12

[Bugmon [:jkratzer for issues]]

Attachment: prefs.js

Comment 14

[Dennis Jackson]

This is a duplicate of bug 2026089. FWIW the report quality of this bug is much higher than the original!