Closed Bug 410339 Opened 17 years ago Closed 17 years ago

error -12263 when accessing IIS 5.1 server that requests client certificates

Categories

(Core :: Security: PSM, defect)

Product:
Core
Component:
Security: PSM
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
major

Tracking

()

Status:
RESOLVED INVALID

People

(Reporter: tlauck, Assigned: KaiE)

References

(
URL
)

Details

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 Firefox unable to make an https connection to an IIS 5.1 web site identified by a Startcom Class 1 server certificate that is accepting client certificates. IE 7.0 does not have this problem. Reproducible: Always Steps to Reproduce: 1. Install Startcom free server certificate in IIS 5.1 web site 2. Enable "accept client certificates" in web site 3. Access web site with https from another computer using Firefox Actual Results: "<web site> has sent an incorrect or unexpected message. Error code: -12263" Expected Results: A request for the user to present a client certificate This problem happens only when client certificates are being accepted and a particular server certificate is in use. Disabling "accept client certificates" or changing to a privately certified server certificate eliminates the problem.
From the "NSS and SSL Error Codes" page (http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html): Error code -12263 is SSL_ERROR_RX_RECORD_TOO_LONG, "SSL received a record that exceeded the maximum permissible length." This generally indicates that the remote peer system has a flawed implementation of SSL, and is violating the SSL specification.
This is either a Firefox and/or an IIS bug. Perhaps the certificate isn't right, it might have been improperly generated or installed. But in that event, IIS should have rejected it. But even if IIS is sending records that are too long, Firefox should be tolerant of them and accepting them (up to the limits of available buffering).
Confirmed this also with the OpenSSL toolkit: New, TLSv1/SSLv3, Cipher is RC4-MD5 Server public key is 1024 bit Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1 Cipher : RC4-MD5 Session-ID: 4E16000026037AFA509F132DF9057E3EC6FDDC8D85D8C779CCE428954E7E4A94 Session-ID-ctx: Master-Key: ACD1087F31405FA7105548802FF1CDC0E617F673B74B70F319C42C823EBA17C1EF961357A15118477C5578D33A8331B9 Key-Arg : None Krb5 Principal: None Start Time: 1199126354 Timeout : 300 (sec) Verify return code: 0 (ok) --- GET / 9419:error:1408F0C6:SSL routines:SSL3_GET_RECORD:packet length too long:s3_pkt.c:302:
moving to PSM but this looks like invalid
Assignee: nobody → kengert
Component: General → Security: PSM
Product: Firefox → Core
QA Contact: general → psm
I did some further investigation. I removed the SSL configuration on the IIS web site. I removed the key from the key store. I reinstalled it. At this point, SSL connection to the web site worked with Firefox and IE but no client certificates were being requested. I then configured IIS so the site would accept client certificates. At this point, IE requested the user to choose a client cert, but Firefox gave an error message. Further, I tried installing a different server cert obtained from a different CA. It had the same problem. However, when the server cert was certified by a private CA there was no problem. I surmise that the length of the server CA certificate chain somehow is affecting whether the error occurs or not. The working chain had no intermediate CAs between the root CA and the server cert. The failing chains had 1 or 2 intermediate CAs. Perhaps others who have made Client Certs work between Firefox and IIS might comment.
OK, I found that the problem reported is simply that his IIS server is talking plain text on port 443. This bug can be closed. However I suggest to enhance the message displayed by Firefox if possible!
Status: UNCONFIRMED → RESOLVED
Closed: 17 years ago
Resolution: --- → INVALID
It might be tricky to enhance the error message. Cc'ing Nelson for the following idea: "Have libssl detect that the other side is sending a plaintext message, which is, does not send any binary bytes, but ascii bytes only, and use a different error code."
If the server was serving http (not https) on port 443 and IE "just worked" then that's a P1 bug, a MAJOR security flaw, in IE, not a mozilla bug. The complaint should not be about mozilla, it should be about IE. Go make a name for yourself by publicly reporting the IE bug.
Actually it might be that this wasn't the case after all, but I presumed that it was like that because I received the exactly same error code when accessing https on port 443 which speaks plain text. I had some further exchange with the reporter of the bug but we couldn't "fix" it on his IIS server. As such, the bug remains unresolved really, but there is no further information available for now except if the reporter can reproduce it once again...
I gave up on client certificates with the IIS based site because of this bug. The site is now up and working with non-public pages secured by other means. In case someone is still interested in investigating, the bug can be seen at the following URL: https://www.aglauck.com/Ctest/
(In reply to comment #6) > OK, I found that the problem reported is simply that his IIS server is talking > plain text on port 443. This bug can be closed. I can not confirm the plaintext observation. When I connect to https://www.aglauck.com/Ctest/ I don't get a plaintext response, I see a handshake - which fails with error ssl_error_rx_record_too_long
Kai, that's correct. My conclusion was premature because when connecting to https://something/ to a plain speaking server one receives exactly the same error message (-12263). Suggest to reopen the bug and leave it currently unconfirmed (it might be a bug of IIS and IE). Also suggest to check for cases when plain text was used when SSL/TLS was expected.
(In reply to comment #12) > Suggest to reopen the bug Reopening as suggested
Status: RESOLVED → UNCONFIRMED
Resolution: INVALID → ---
I just found the same issue when going to <https://bugzilla.mozilla.org:80/query.cgi>. I receive a complaint about a ssl_error_rx_record_too_long. The reason is that I used port 80 here, while it's supposed to use port 443. So the response is coming from the regular http server, in plain text. And mozilla isn't using IIS obviously :-) But in this case it's not a misconfigured server that sends plaintext on port 443, it's a mistake in the url.
So the result is perhaps the same, the URL is completely valid. SSL servers don't have to run on port 443, this can be also port 80 (also inconvenient obviously). Thanks for confirming with this little test the issue at hand. Checking for plain text response (and according message) would give a good indication to the user.
This bug is ostensibly about a problem with some server that requests client certificate authentication. The problem is manifest as a record_too_long error. There are many causes of record_too_long errors. It is well known that a server that sends plain text, instead of SSL records, to the client will cause the client to experience a record_too_long error. I think this bug should not be about any and all causes of record too long errors, but rather about this one cause, which is this server that is requesting client authentication. Unfortunately, unless and until we can reproduce it with this server (or some other server that is requesting client authentication), we are not likely to know the real cause of this reported problem. I rather doubt that it is the act of the server requesting authentication from the client that is the cause of this reported behavior, but until the problem can be reproduced, and captured with our ssltap tool (or a similar tool such as ssldump), we can only conjecture about the real cause.
Yes, you are correct. Just on the way it would be nice to get a "SERVER_TALKS_PLAIN" error or something like this. It would be really helpful for diagnosis and some hint for the troubled user. And what happens if the server sends an empty page or 0 bytes. Most likely the record will not be too long in that case.... Or how about a "HAND_SHAKE_FAILED" error? This would give some indication and I suspect it would be possible from NSS without adding too much code (compared to a test of plain text). Any of this feasible?
I just used Firefox 2.0.0.11 to access the test URL on my server that I previously posted, and the bug is still reproducable: https://www.aglauck.com/Ctest/ I suggest that someone with the appropriate tools and expertise examine in detail the exchange that takes place between Firefox and this server while accessing this URL, comparing message contents with appropriate specifications.
It also happens with Gecko/2008020513 Firefox/3.0b3
OK, thanks for the URL. I tested. The error message being reported by your browser is EXACTLY correct. The problem is not that ASCII text was sent instead of an SSL record. The server attempted to send an SSL record to the client with a length of 19921 bytes (after the 5 byte header), and that is simply too long. The maximum length of an SSL3/TLS record is 16K bytes (after the 5 byte header). The sequence of events is that the first handshake completes normally, without any client authentication request from the server. Then the client sends its get http request for /Ctest, and the server responds by requesting a second handshake (for the purpose of requesting client authentication, no doubt). So, the client begins a second handshake, sending its client hello record to the server. Then the server replies with that whopper record that is over 19K bytes. And the browser rightfully says, not on your life. Here is a dump of the first 128 bytes of that whopper record. 0: 16 03 01 4d d1 bd c6 38 70 6e c3 d8 46 8e 81 70 | ...M...8pn..F..p 10: a2 8d 17 12 01 9b b9 28 85 6a c7 e1 a5 32 1e c3 | .......(.j...2.. 20: 1d d6 2f 77 96 5f d8 05 7e 64 05 b8 8b 6b 81 5d | ../w._..~d...k.] 30: 45 80 77 c0 2b a9 8d 3c 7b 37 c5 03 71 09 7e b9 | E.w.+..<{7..q.~. 40: 17 f5 ab 1e 75 c5 84 c9 bd 43 44 53 37 e8 6e b1 | ....u....CDS7.n. 50: 1a c1 aa ca de 02 19 71 6a ce 62 c2 49 3f 2f 28 | .......qj.b.I?/( 60: 64 75 16 78 24 b0 32 ea ed 6e 45 63 37 bd 2d 77 | du.x$.2..nEc7.-w 70: 31 4c 9f c5 10 eb 57 34 53 5d 2a 2a 33 f8 73 d9 | 1L....W4S]**3.s. Let's look at the first 5 bytes, the only ones that are not encrypted. They are: 0: 16 03 01 4d d1 Those bytes are: hex decimal Means 16 22 This is a handshake record 03 01 N/A SSL version 3.1, better known as TLS 1.0 4d d1 19921 number of bytes to follow, exceeds 16K. Sorry, tlauck, you've got a bad server implementation. There's no client bug here. The error reported by the client was EXACTLY correct. No other message would have stated the problem better than the one you got.
Status: UNCONFIRMED → RESOLVED
Closed: 17 years ago → 17 years ago
Resolution: --- → INVALID
Very cool analysis, Nelson! What about the plain connection receiving the same error message? Isn't that somewhat odd (and contributes to some confusion)?
(In reply to comment #16) > There are many causes of record_too_long errors. It is well known that a > server that sends plain text, instead of SSL records, to the client will > cause the client to experience a record_too_long error. Is it worthwhile to file a bug ? I can't find an existing bug about it.
Great! Thanks, Nelson. This explains everything, including that IE worked -- presumably IE allowed the excessive length records, not exactly a "fix". I suppose that someone should report this problem with IIS 5.1 (included with Windows XP Pro SP2)to MS, now that it is adequately documented. MS might get around to fixing it someday, or not...
Summary: error -12263 when accessing server that accepts client certificates → error -12263 when accessing IIS 5.1 server that requests client certificates
This KB article contains a number of workarounds for a similar problem. http://support.microsoft.com/kb/933430 Tlauck, You might try them and see if they solve the problem you're encountering. If so, please tell us here.
Thank you, Nelson, for pointing me to the KB article. Of the various workarounds in the article, suggested workaround #1 was applicable to my server. When I followed the suggestion and removed all the unneeded root certificates the problem was cured.
You need to log in before you can comment on or make changes to this bug.