1008120 - Trailing dot in SNI HostName must be stripped according to RFC 3546

Status: UNCONFIRMED

(Core :: Security: PSM, defect, P5)

Description

[Leon Winter]

Attachment: firefox_sni_trailing_dot_fixing_concept.diff

User Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20140509 Firefox/24.0 Iceweasel/24.5.0 (Nightly/Aurora)
Build ID: 20140509105114

Steps to reproduce:

Do a HTTPS request providing a URL containing a trailing dot (for the DNS root zone level), e.g. "https://www.google.com."


Actual results:

HostName in SNI send to the server is "www.google.com."
Also the Apache webserver is rather unhappy and reports the SNI and HTTP HostName are not matching, which in fact they are (exactly), but nonetheless according to the RFC a trailing dot must not occur in a SNI host name. When Firefox is patched (see attachment), Apache is happy, too.


Expected results:

HostName in SNI should be "www.google.com".
Section 3.1 of RFC 3546 [0] states:

> "HostName" contains the fully qualified DNS hostname of the server,
> as understood by the client. The hostname is represented as a byte
> string using UTF-8 encoding [UTF8], without a trailing dot.

See also https://github.com/bagder/curl/commit/5de8d84098db1bd24e7fffefbe14e81f2a05995a
http://curl.haxx.se/mail/lib-2014-04/0161.html

The patch is just to merely a show case where a fix could be made and being an adaption from the cURL patch. If the 'host' argument shall not be modified, a duplication and inplace modification of the host name string might be fine but should be free()d afterwards.
However maybe you want to fix/normalize/strip the host name string at an earlier point in time.
FYI, patch is against 24.5.0esr, however should be applicable on trunk, too.

Comment 2

[Yst Dawson]

You say "If the 'host' argument shall not be modified, a duplication and inplace modification of the host name string might be fine but should be free()d afterwards.". I assume that the "host" argument is the HTTP Host header? According to RFC 7230, the HTTP Host header should indeed not be modified.

<https://tools.ietf.org/html/rfc7230#section-5.4> (HTTP)
	A client MUST send a Host header field in all HTTP/1.1 request
	messages.  If the target URI includes an authority component, then a
	client MUST send a field-value for Host that is identical to that
	authority component, excluding any userinfo subcomponent and its "@"
	delimiter (Section 2.7.1).

If the URI has a trailing dot in the host component, the HTTP Host header must also contain this dot. The fact that cURL removes the trailing dot from the HTTP Host header is in fact a bug in cURL, or is at least nonstandard behavior.

Comment 3

[Leon Winter]

There seems be a lack of specification and implementation of the interplay between SNI and HTTP. While you are correct by RFC7230, a lot of applications are already dropping the trailing dot to avoid further problems. In my original mailing list discussion I also mentioned the PROXY setting of RFC2616 which also dictates that a host field may not be changed:
https://curl.haxx.se/mail/lib-2014-04/0206.html
I can think one can reasonably argue for both sides in terms of not breaking poorly set up web sites or adhering to the standard. I am aware that dropping the trailing dot as done in curl can break other things as well, but I have no insights as to which way more of the web breaks. Daniel seems to suggest that going by the standard would break more.

The old links to the mailing list seem to be broken, so I am just repost the working ones:
Start of thread: https://curl.haxx.se/mail/lib-2014-04/0166.html
Continuation: https://curl.haxx.se/mail/lib-2014-05/0022.html

For further research on this issue, I also like to refer to your very own research:
https://y.st/en/URI_research/SNI_bug.xhtml

Also Daniel of curl discussed this issue on the HTTP WG mailing list: 
https://lists.w3.org/Archives/Public/ietf-http-wg/2016JanMar/0430.html
Triggered by your bug report in curl:
https://github.com/curl/curl/issues/716

The ideal outcome of this effort would of course be a consistent behavior across all applications but this is unlikely in the short term.

Comment 4

[Leon Winter]

Looking at my old patch, it seems to only affect the host name used in SNI and should not touch the HTTP header field of the host. Since the hostname parameter is passed as const char* this indicates it is passed from a higher layer and may not be modified from within the function. Since SNI requires stripping of the trailing dot, we therefore duplicate and modify the host name for the SSL/TLS connection but leaving the hostname passed to us intact.

Comment 5

[Anne (:annevk)]

Bug 134402 comment 38 suggests Chrome does not strip it either. If that's still the case I think it would be better for the relevant SNI standard to be updated. We definitely don't want to be a first-mover here.

Comment 6

[coder]

Shouldn't Firefox always implement standards correctly? Is it Chrome that determines how things work? Isn't it a motivation for Firefox to be the first-mover to properly implement a standard?

This ticket was opened 8 years ago. It referred to RFC 3546 (from 2003), now obsolete, replaced by RFC 6066 (from 2011), which still maintains the same procedure for the SNI extension:

https://datatracker.ietf.org/doc/html/rfc6066#section-3
Server Name Indication
"HostName" contains the fully qualified DNS hostname of the server,
as understood by the client. The hostname is represented as a byte
string using ASCII encoding without a trailing dot.

Failure to comply with the standard may result in the server delivering an inappropriate certificate (e.g. adobe.com) or even no certificate at all, resulting in the TLS tunnel not being established (e.g. vimeo.com).

With the above two examples, Firefox behaves as follows:

# https://adobe.com./ -> SSL_ERROR_BAD_CERT_DOMAIN
$ echo Q |
> openssl s_client -servername adobe.com. -connect adobe.com.:443 |&
> openssl x509 -noout -issuer -subject -serial -dates -ext subjectAltName
issuer=C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
subject=C = US, ST = Massachusetts, L = Cambridge, O = "Akamai Technologies, Inc.", CN = a248.e.akamai.net
serial=04630FAC0AD3C8AC44FF72DBB3673F4B
notBefore=Jun 28 00:00:00 2022 GMT
notAfter=Jun 30 23:59:59 2023 GMT
X509v3 Subject Alternative Name:
    DNS:a248.e.akamai.net, DNS:*.akamaihd-staging.net, DNS:*.akamaihd.net, DNS:*.akamaized-staging.net, DNS:*.akamaized.net

# https://vimeo.com./ -> SSL_ERROR_NO_CYPHER_OVERLAP
$ echo Q |
> openssl s_client -servername vimeo.com. -connect vimeo.com.:443 |&
> grep 'alert'
140632084976256:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1543:SSL alert number 40

But the correct and standard way would be:

# https://adobe.com./
$ SNI="adobe.com"; HOST="adobe.com."; \
> printf "GET / HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n" "$HOST" |
> openssl s_client -ign_eof -servername "$SNI" -connect ${SNI}:443 |&
> sed -n -e '/chain/,+2p' -e '\#^HTTP/#,+4p'
Certificate chain
 0 s:C = US, ST = California, L = San Jose, O = Adobe Inc, CN = *.adobe.com
   i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
HTTP/1.1 301 Moved Permanently
Server: AkamaiGHost
Content-Length: 0
Location: https://www.adobe.com/
Date: Sun, 03 Jul 2022 16:43:26 GMT

# https://vimeo.com./
$ SNI="vimeo.com"; HOST="vimeo.com."; \
> printf "GET / HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n" "$HOST" |
> openssl s_client -ign_eof -servername "$SNI" -connect ${SNI}:443 |&
> sed -n -e '/chain/,+2p' -e '\#^HTTP/#,+4p'
Certificate chain
 0 s:C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = vimeo.com
   i:C = US, O = "Cloudflare, Inc.", CN = Cloudflare Inc ECC CA-3
HTTP/1.1 200 OK
Date: Sun, 03 Jul 2022 16:45:12 GMT
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Connection: close

I consider this a bug that Firefox should fix as soon as possible.

Comment 7

[mjl]

FYI, Go's crypto/tls does not accept trailing dot in the SNI domain name: https://github.com/golang/go/issues/63117
For example, going to https://www.xmox.nl./ with Firefox causes a TLS handshake error.
Note: I have no need for absolute URLs in Firefox, so Firefox's current behaviour is not bothering me, but it does look incorrect.

Comment 8

[Chris Siebenmann]

Based on experimentation, it appears that OpenSSL strips a terminal dot when building the client SNI; Go's code definitely does. GnuTLS doesn't seem to and will generate requests with such a SNI (and then fail to verify the resulting server certificate, which doesn't have such a name in it). When given a HTTPS URL with a terminal dot in the hostname, iOS Safari appears to strip it (it is hard for me to confirm by inspecting the resulting TLS traffic). RFC 6066 section 3 is specific that the SNI is not supposed to have the terminal dot, and as far as I know that section hasn't been updated by later RFCs.