Closed Bug 1119983 Opened 5 years ago Closed 5 years ago

NSS TLS 1.2 server fails to interoperate with LibreSSL client

Categories

(NSS :: Libraries, defect, major)

3.17.3
x86
Solaris
defect
Not set
major

Tracking

(Not tracked)

RESOLVED FIXED
3.17.4

People

(Reporter: chris.newman, Assigned: chris.newman)

References

Details

Attachments

(1 file)

When NSS is used as a TLS 1.2 server, it does not interoperate with the LibreSSL client due to two bugs in NSS TLS 1.2 implementation of the signature_algorithms extension. First, the matching for signature_algorithms uses the wrong number space as the SSL3SignatureAndHashAlgorithm structure uses a SECOidTag and the hashPreference array is using TLS hash identifier namespace in ssl3con.c. Second, the parser for signature_algorithms in ssl3_ServerHandleSigAlgsXtn handles the SEC_OID_UNKNOWN case incorrectly leaving uninitialized variables in the list.

Because SEC_OID_SHA1 is 4 and tls_hash_sha256 is also 4, the first bug is usually not noticed, but LibreSSL uses a private-use hash identifier and puts SHA1 at the end, so the second bug causes the first bug to manifest and no matching hash algorithm is found. This results in a bogus SSL_ERROR_UNSUPPORTED_HASH_ALGORITHM error from NSS server.
Attachment #8546891 - Flags: review?
Attachment #8546891 - Flags: review? → review?(agl)
See Also: → 480514
Attachment #8546891 - Flags: review?(agl) → review+
Thanks for the review.
Keywords: checkin-needed
Assignee: nobody → chris.newman
Kai, can you help get this fix landed? This is my first time trying to fix a bug in NSS so feel free to point me to RTFM if I did something wrong. Thanks!
Flags: needinfo?(kaie)
Flags: needinfo?(kaie)
https://hg.mozilla.org/projects/nss/rev/72b9d7234fa8
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → 3.18
Keywords: checkin-needed
mass change target milestone to 3.17.4
Target Milestone: 3.18 → 3.17.4
You need to log in before you can comment on or make changes to this bug.