Last Comment Bug 702111 - Servers intolerant to 1/n-1 record splitting. "The connection was reset"
: Servers intolerant to 1/n-1 record splitting. "The connection was reset"
Status: RESOLVED WONTFIX
[qa+] [Workarund: Env. var of NSS_SSL...
: regression, relnote
Product: NSS
Classification: Components
Component: Libraries (show other bugs)
: 3.13
: All All
: -- normal with 4 votes (vote)
: ---
Assigned To: nobody
:
Mentors:
https://bugzilla.mozilla.org/show_bug...
: 701945 704568 704664 723370 726329 727196 728107 728530 735720 (view as bug list)
Depends on: 704990
Blocks: 723109 723551 723805 CVE-2011-3389 669061 723898 724478
  Show dependency treegraph
 
Reported: 2011-11-13 05:42 PST by Alice0775 White
Modified: 2012-10-23 00:44 PDT (History)
56 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
unaffected
+
fixed
+
wontfix
+
wontfix
-


Attachments
Disable BEAST workaround for Firefox 9 only [NOT FOR mozilla-central OR mozilla-aurora] (1.02 KB, patch)
2011-12-06 14:59 PST, Brian Smith (:briansmith, :bsmith, use NEEDINFO?)
honzab.moz: review+
akeybl: approval‑mozilla‑beta+
Details | Diff | Review

Description Alice0775 White 2011-11-13 05:42:07 PST
Build Identifier:
http://hg.mozilla.org/releases/mozilla-beta/rev/31302afe89b3
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20100101 Firefox/9.0 ID:20111109112850
http://hg.mozilla.org/releases/mozilla-aurora/rev/dcfa8d61f3c5
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0a2) Gecko/20111112 Firefox/10.0a2 ID:20111112042038
http://hg.mozilla.org/mozilla-central/rev/470d6053b749
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0a1) Gecko/20111112 Firefox/11.0a1 ID:20111112031541


Cannot load https://store.toto-dream.com/toto/member/ToHPLogin.do
*This happens on Firefox9.0beta, Aurora10.0a2 and Nightly11.0a2.
*This does not happen on Firefox8.0 and 3.6.24.


Error page is shown.
The connection was reset
  The connection to the server was reset while the page was loading.
  ・The site could be temporarily unavailable or too busy. Try again in a few
    moments.
  ・If you are unable to load any pages, check your computer's network
    connection.
  ・If your computer or network is protected by a firewall or proxy, make sure
    that Nightly is permitted to access the Web.


Reproducible: Always

Steps to Reproduce:
1 Start Firefox with clean profile
2 Open https://store.toto-dream.com/toto/member/ToHPLogin.do

Actual Results:
  Error page "The connection was reset" is shown

Expected Results:
  Login form should be shown.


Regression window(m-c),
Works:
http://hg.mozilla.org/mozilla-central/rev/35954e6f3167
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0a1) Gecko/20111007 Firefox/10.0a1 ID:20111007120745
Fails:
http://hg.mozilla.org/mozilla-central/rev/8f011395145e
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0a1) Gecko/20111007 Firefox/10.0a1 ID:20111007144324
Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=35954e6f3167&tochange=8f011395145e
Triggered by:
8f011395145e	Brian Smith — Bug 669061: Upgrade to NSS 3.13 RC0, r=wtc


Regression window(m-aurora),
Works:
http://hg.mozilla.org/releases/mozilla-aurora/rev/b16983396f30
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a2) Gecko/20111104 Firefox/9.0a2 ID:20111104042026
Fails:
http://hg.mozilla.org/releases/mozilla-aurora/rev/505216fa1c65
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a2) Gecko/20111105 Firefox/9.0a2 ID:20111105042036
Pushlog:
http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=b16983396f30&tochange=505216fa1c65
Triggered by:
bf9edc46e39f	Kai Engert — Bug 669061, Upgrade Mozilla to NSS 3.13.1, r=kaie, r=wtc, a=clegnitto
Comment 1 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2011-11-13 15:54:52 PST
*** Bug 701945 has been marked as a duplicate of this bug. ***
Comment 2 Alice0775 White 2011-11-18 01:15:22 PST
This page ( https://www.billing.ntt-east.co.jp/entrance ) also fail to load and I got same regression range.
Comment 3 Matthias Versen [:Matti] 2011-11-22 14:22:20 PST
*** Bug 704568 has been marked as a duplicate of this bug. ***
Comment 4 Silanea 2011-11-23 02:29:32 PST
This is not limited to one website. In the hope that this may help pinpoint the cause of the breakage here is what I saw in Wireshark:

Affected builds (apparently anything >= 9.0b1) initiate a TLS handshake, then simply stop communicating with the server. Unaffected builds (FF 8 and earlier) initiate a TLS handshake that fails with the state shown below and then initiate an SSL 3 handshake that succeeds.


Here is what apparently breaks the TLS handshake:

<quote>
Secure Sockets Layer
  TLSv1 Record Layer: Alert (Level: Fatal, Description: Illegal Parameter)
    Content Type: Alert (21)
    Version: TLS 1.0 (0x0301)
    Length: 2
    Alert Message
      Level: Fatal (2)
      Description: Illegal Parameter (47)
</quote>
Comment 5 Matthias Versen [:Matti] 2011-11-23 15:35:06 PST
*** Bug 704990 has been marked as a duplicate of this bug. ***
Comment 6 Alice0775 White 2011-11-24 01:35:47 PST
FYI, It works if I set security.ssl3.rsa_aes_256_sha to false.
Comment 7 Kai Engert (:kaie) 2011-11-24 03:41:15 PST
I can reproduce the bug.
Using the same application profile, FF 8 works, FF 9 fails.

After initial tracing of the SSL interaction I can see, the handshake completes, both client and server have completed sending handshake messages (the last one seen is a "change cipher spec" message sent by the server). Given that hex dumps produced by "ssltap" are identical for both sessions (except the "random"), I conclude that both versions used the same parameters for the SSL session.

Now the difference:

FF 8:
- continues by sending a single SSLRecord with application data, 368 bytes
- server responds with a single SSLRecord with application data, 352 bytes

FF 9:
- continues by sending *TWO* SSLRecords with application data, 32 bytes, then 304 bytes
- server closes the connection
Comment 8 Kai Engert (:kaie) 2011-11-24 05:18:52 PST
We see two separate SSL records because of the patch added for bug 665814 (attachment 563777 [details] [diff] [review]), which deliberately starts by sending only a single byte, then adds the remaining bytes.

For testing purposes I have reverted that patch on my system, and this allows me to successfully connect to https://extranet.thw.de/
Comment 9 Kai Engert (:kaie) 2011-11-24 05:26:06 PST
A question to everyone affected by this bug:
If you control the server that doesn't work, can you please tell us:

Which server software, SSL toolkit (and version numbers) are being used?

Thanks
Comment 10 Kai Engert (:kaie) 2011-11-24 06:58:18 PST
I made a couple of experiments, in order to better understand the server behaviour.

Instead of using "1 plaintext byte" for the initial SSL record, I played with other numbers.
(Note the patch from bug 665814 intends to protect against an attack on SSL connection.)

The results are:

Failure:
- 1 byte
- 8 bytes
- 15 bytes

Works:
- 16 bytes
- 32 bytes


I don't know why a server might behave that way.

Is there any reason why a server might conclude that a "less than 15 bytes plaintext SSL record" followed by another SSL record is a reason to abort a connection?

Maybe a receive buffer used by the server is set to 16 bytes, and if it cannot even fill this very small buffer, it refuses to splice the records?
Comment 11 Kai Engert (:kaie) 2011-11-24 07:07:00 PST
More tests.

For the following sites, an initial plaintext buffer smaller than 16 bytes fails, but with an initial plaintext buffer of at least 16 bytes it works:
- https://extranet.thw.de
- https://store.toto-dream.com/
- https://www.billing.ntt-east.co.jp/

However, for some other sites (found in the duplicate bug reports), using a 16 bytes initial record still fails. For the following sites, it works with a 32 bytes buffer:
- https://sslportal.ipoc.btservices.fr/
- https://212.121.161.50/
Comment 12 Kai Engert (:kaie) 2011-11-24 08:10:00 PST
In my understanding the reported sites are misbehaving and they should upgrade/fix their server software.

Because of the security issue described in bug 665814, we don't want want to increase the amount of plaintext data sent in the initial record.

SSL/TLS servers are expected to combine a sequence of separate SSL records, and must not abort when seeing such a sequence. (But the reported servers do abort the connection.)
Comment 13 Boris Zbarsky [:bz] (Out June 25-July 6) 2011-11-24 10:59:11 PST
We should probably have separate evangelism bugs per server here, right?

Is it worth considering a blacklist of known-intolerant hostnames?
Comment 14 hannu.nyman 2011-11-24 11:15:58 PST
Additional case:
I noticed that Nordea's (a major bank in Sweden, Finland & other Nordic/Baltic countries) netbank web site crashes with the same regression window (07 Oct works, 08 Oct fails). 

The difference to the earlier cases is that this happens after a successful login in the middle of the netbank session. I have been wondering about this error for some time in my normal banking in the pas few weeks, but today I realized that this error is actually related to this bug.

Based on Wireshark output, the netbank session with 62.13.0.79 (https://solo1.nordea.fi/nsp/engine) works normally during login and also during browsing the info pages at netbank, but when selecting account information or some other real banking functionality, the session may crash and you end up with an error page. 

Based on Wireshark output, the crash might not actually happen with the main netbank, but with the embedded Webtrends tracking cookies/javascript from 66.150.117.25 (which produces first an "Change Cipher Spec", then an "Encrypted Alert" message to the Wireshark's output), and the netbank probably recognizes that and breaks the session giving an error page.

I tested several times and could not figure out a reliable way of reproducing the error, as the error does not always surface with the visit to a certain page, but will in any case eventually materialize. I even managed to get the error a few times on the login page of the bank by just repeatedly switching languages without logging.

Having the Webtrends cookie/javascript as a possible culprit may make this a more widely spread problem.
Comment 15 Silanea 2011-11-25 14:24:03 PST
Not sure if it is related, but https://shop.lenovo.com now gives me "SSL received a record that exceeded the maximum permissible length.". It worked the last time I visited the website with a nightly build about one month ago, so this fits the timeframe for the issue discussed in this bug.
Comment 16 Kai Engert (:kaie) 2011-11-28 09:14:03 PST
(In reply to Silanea from comment #15)
> Not sure if it is related, but https://shop.lenovo.com now gives me "SSL
> received a record that exceeded the maximum permissible length.".

I can connect fine to that site successfully, maybe it was a temporary, different problem.
The error message you mention is frequently seen when using https to connect to a server that is configured to speak plain http.

For this bug, only sites that produce "connection was reset" are likely to be caused by the same problem.
Comment 17 Kohei Yoshino [:kohei] 2011-11-29 23:41:41 PST
Retrieved from Firefox Input (where I can reproduce the reset issue):

https://runnet.jp/
https://store.toto-dream.com/
https://www.billing.ntt-east.co.jp/
https://www.wnp.waseda.jp/
(the above are Japanese sites)
https://aviva.rewardgateway.co.uk/
https://www.the-house.com/
https://www.my-hammer.de/
https://eee.uci.edu/
Comment 18 Kai Engert (:kaie) 2011-11-30 09:21:59 PST
https://www.bund.de
Comment 19 [:Cww] 2011-11-30 10:16:13 PST
(In reply to hannu.nyman from comment #14)
I think that the Nordea case may be something more since it's crashing (and not just producing error messages).  Look at bug 704664
Comment 20 hannu.nyman 2011-11-30 12:51:22 PST
(In reply to [:Cww] from comment #19)
> (In reply to hannu.nyman from comment #14)
> I think that the Nordea case may be something more since it's crashing (and
> not just producing error messages).  Look at bug 704664

I maybe formulated my report weakly, as the for me Firefox does not fully crash, instead the bank's server gives an error message screen containing the following text:
"Järjestelmävirhe / Systemfel / System error
System failure
Järjestelmävirhe
Verkkopankki-istunnossasi on tapahtunut virhe ja istunto on suljettu. Tallentamattomat tiedot ovat voineet kadota. Kirjaudu uudestaan verkkopankkiin.
Systemfel
Ett systemfel har tyvärr uppstått och din nätbanksession har stängts. Data som inte har sparats kan ha försvunnit. Logga in på nytt i nätbanken.
System error
A system error has occurred in the service and your Netbank session has been closed. Unsaved information may have been lost. Please log in again to Netbank."

Like I said in comment #14, it looks like the netbank session detects breakage, possible in connection to some tracking jscript, and reacts to that giving the error. So it is not a full crash of Firefox.
Comment 21 Kai Engert (:kaie) 2011-11-30 13:14:08 PST
Dear Hannu, this sounds like a different issue. If your browser crashes, please file a separate bug report. If instead it's simply that your bank site displays an error message, maybe it's better to contact the bank.
Comment 22 Alex Keybl [:akeybl] 2011-11-30 14:33:13 PST
From my read of this bug, I believe this is a matter of outreach. Including Kev here.
Comment 23 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2011-12-06 14:59:55 PST
Created attachment 579472 [details] [diff] [review]
Disable BEAST workaround for Firefox 9 only [NOT FOR mozilla-central OR mozilla-aurora]
Comment 24 Honza Bambas (:mayhemer) 2011-12-06 15:19:28 PST
Comment on attachment 579472 [details] [diff] [review]
Disable BEAST workaround for Firefox 9 only [NOT FOR mozilla-central OR mozilla-aurora]

Review of attachment 579472 [details] [diff] [review]:
-----------------------------------------------------------------

r=honzab

Confirmed this fixes the problem.
Comment 25 Alex Keybl [:akeybl] 2011-12-06 16:32:29 PST
Comment on attachment 579472 [details] [diff] [review]
Disable BEAST workaround for Firefox 9 only [NOT FOR mozilla-central OR mozilla-aurora]

[Triage Comment]
Approved for beta. Thanks for getting this done so quickly!
Comment 26 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2011-12-06 18:38:03 PST
Comment on attachment 579472 [details] [diff] [review]
Disable BEAST workaround for Firefox 9 only [NOT FOR mozilla-central OR mozilla-aurora]

This patch was checked in to mozilla-beta *only*:

https://hg.mozilla.org/releases/mozilla-beta/rev/07882283b9e0

Leaving the bug open to track Firefox 10 and later.
Comment 27 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2011-12-06 18:48:25 PST
Backed out of the wrong release branch I landed it on:
https://hg.mozilla.org/releases/mozilla-beta/rev/019a66262a21

Pushed to the default branch as intended:
https://hg.mozilla.org/releases/mozilla-beta/rev/4952c336f367
Comment 28 Daniel Veditz [:dveditz] 2011-12-07 13:28:38 PST
Marking status-firefox9 "fixed" per comment 27.

Other than bug 704990 I don't see any Tech Evangelism bugs filed for the sites from comment 11, comment 17, and comment 18 plus others we may find from input.
Comment 29 Robert Kaiser (not working on stability any more) 2011-12-12 05:00:34 PST
*** Bug 704664 has been marked as a duplicate of this bug. ***
Comment 30 Boris Zbarsky [:bz] (Out June 25-July 6) 2011-12-19 20:09:03 PST
Looks like this issue is why login to my verizon account doesn't work.  It used to be broken on Fx9 betas, but was fixed with the pushes in comment 27.

Should I file a separate bug, or are we tracking everything here somehow?
Comment 31 j.j. 2011-12-22 19:00:56 PST
Changing status flags as I understand this bug, please correct if wrong.
Comment 32 Stefan Becker 2011-12-28 08:30:25 PST
FYI: I just debugged this issue for another use case: 

      pidgin-sipe protocol plugin
  via libpurple/ssl-nss.so SSL plugin
  to  Microsoft Lync server

This problem has been at least discussed here:

  <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=649456>
  <http://sourceforge.net/projects/sipe/forums/forum/688534/topic/4820655>

Running pidgin with NSS_SSL_CBC_RANDOM_IV=0 fixes the problem, i.e. login to Lync accounts works again. So also SSL connection outside Firefox are affected.
Comment 33 christian 2011-12-29 11:33:58 PST
So, what's the plan for this bug and Firefox 10 (currently in Beta?)
Comment 34 Martin Wilck 2012-01-11 06:05:48 PST
We are seeing the same problem here with a an embedded web server in one of our Server Management applications. Our SSL implementation is based on OpenSSL (tested with 1.00e and 1.00f).

I'd appreciate a hint what must be changed in an OpenSSL-based https server to make it compliant with NSS 3.13.1.
Comment 35 Nick Lowe 2012-01-11 06:32:13 PST
Fixes from Microsoft for this issue in SCHANNEL are now available:

http://technet.microsoft.com/en-us/security/bulletin/ms12-006

Also see:
http://support.microsoft.com/kb/2643584
Comment 36 Kai Engert (:kaie) 2012-01-11 06:38:16 PST
(In reply to Martin Wilck from comment #34)
> 
> I'd appreciate a hint what must be changed in an OpenSSL-based https server
> to make it compliant with NSS 3.13.1.


You must make your application code smarter.

Whenever your application attempts to read application (data) bytes from the SSL socket, you must be prepared to receive the data in multiple chunks.

Your current code is probably like this:
- read from socket
- you see one byte
- you try to process the one byte according to your inner protocol
- your code decides that one byte isn't a valid protocol command, 
  and you abort the connection because of a failure

You must change your code to continue to read from the socket, and concatenate the received bytes, until you are able to process the data according to your protocol.
Comment 37 Brad Wetmore 2012-01-11 11:29:19 PST
(In reply to Kai Engert (:kaie) from comment #36)

> You must make your application code smarter.
> 
> Whenever your application attempts to read application (data) bytes from the
> SSL socket, you must be prepared to receive the data in multiple chunks.

Over the last two months, I have chased several escalations with exactly these problems, generally falling into one of two patterns.  It's amazing how much code either doesn't check return values:

    read(fd, buf, size);   // assume we get "size" bytes (!)

or expects size to be a full read:

    retval = read(fd, buf, size);
    if (retval != size) {
        // fail...short read
    }

Note that TCP layer could also decide to segment packets, code like the above will fail.
Comment 38 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2012-01-12 16:58:16 PST
See http://support.microsoft.com/kb/2643584

Microsoft has decided to make the record splitting opt-in by default; it isn't the default in SChannel. I have not checked if it is the default in IE or in Microsoft's plugins (e.g. Silverlight) that would be embedded in our applications.

Microsoft is suggesting to enable TLS 1.1 and/or TLS 1.2 support as the best solution.

We may need to reconsider our defaults, and/or reprioritize TLS 1.1 support. And, if IE isn't doing the record splitting by default, it seems like we are likely going to need to disable the record splitting in Firefox and Chrome. I believe that it is actually safe to do so at least in Firefox, and/or we may be able to do more to prevent a successful attack even without the record splitting in Firefox and probably in other browsers.

Unfortunately, Oracle and vendors of similar plugins seem to be in a difficult place, because their flexible networking APIs cause problems for the embedding browser in a way that the browsers' less flexible networking APIs do not.
Comment 39 Adam Langley 2012-01-12 18:02:55 PST
In http://support.microsoft.com/kb/2643584

"This security update also fixes the application layers that are involved in web browsing by using Internet Explorer to send the Secure flag, in order to help secure the browser usage scenarios."

It appears that, although it may be disabled by default for SChannel, IE specifically will be using it. I'll try to confirm tomorrow.
Comment 40 Nick Lowe 2012-01-12 20:41:25 PST
Anecdotally, I think they are using it by default for IE because, when I last checked, the hotfix was not installed by default by Windows Update under Vista and 7 and had to be manually chosen. Curiously, it was by default under XP.

I suspect they've gone for the phased approach to see what breaks and to try to get things fixed as, if and when they do.
Comment 41 Adam Langley 2012-01-13 07:53:38 PST
I've confirmed that, with the latest update, IE performs 1/n-1 record splitting by default.
Comment 42 Alex Keybl [:akeybl] 2012-01-17 10:00:28 PST
(In reply to Brian Smith (:bsmith) from comment #26)
> Comment on attachment 579472 [details] [diff] [review]
> Disable BEAST workaround for Firefox 9 only [NOT FOR mozilla-central OR
> mozilla-aurora]
> 
> This patch was checked in to mozilla-beta *only*:
> 
> https://hg.mozilla.org/releases/mozilla-beta/rev/07882283b9e0
> 
> Leaving the bug open to track Firefox 10 and later.

After discussing with release drivers, let's do the same for FF10 beta, FF11 Aurora, and FF12 m-c. There's no security need for the BEAST workaround in Firefox at this time. My understanding is that this will have to be re-evaluated in the future as Firefox's landscape changes.
Comment 43 Kai Engert (:kaie) 2012-01-17 10:46:03 PST
(In reply to Alex Keybl [:akeybl] from comment #42)
> 
> There's no security need for the BEAST workaround in
> Firefox at this time.

Why not?
Comment 44 Nick Lowe 2012-01-17 10:55:02 PST
Microsoft in Windows XP, Vista and 7 for IE6, IE7, IE8 and IE9 (all using SCHANNEL) have implemented this behaviour change.

... And Google have in Chrome.
... And Opera Software have in Opera.

But Mozilla choose not to in Firefox?

What's wrong with doing things consistently in the industry, in lockstep?

Cheers,

Nick
Comment 45 Alex Keybl [:akeybl] 2012-01-17 12:00:43 PST
(In reply to Nick Lowe from comment #44)
> What's wrong with doing things consistently in the industry, in lockstep?

The security team should be able to address the technical specifics, but there is currently no need for the workaround in Firefox. The other browsers you mention may have leveraged code that require the workaround. We have weighed the benefits, and decided to maintain web compatibility rather than take a currently unnecessary security fix.
Comment 46 Nick Lowe 2012-01-17 12:14:41 PST
Thanks :)

It would indeed be good to know the specifics of the architectural difference in Firefox that makes it unnecessary to correct a security flaw that is in the design of the SSL 3.0 and TLS 1.0 protocols themselves, that other Web browser developers have seen fit to patch.

With a security flaw like this, it is not a zero sum game - by having consistent behaviour in all, it would force the hand of those that offer services with out-of-date SSL/TLS implementations to actually patch and maintain them to retain compatibility with the Web browsers people actually use.

By not patching it, aside from active advocacy, you are relying on the market share and impetus of the other browsers to give the feedback loop that is needed to get services with non-compliant SSL/TLS implementations patched.
Comment 47 Boris Zbarsky [:bz] (Out June 25-July 6) 2012-01-17 12:19:41 PST
Are other browsers shipping the 1/n-1 splitting without exception lists?  Last I checked Chrome was at least considering an exception list...  The exception list issue affects the "force the hand" argument, obviously.  I'd be much happier with shipping 1/n-1 splitting if other UAs are applying it across the board than if they're applying it selectively, unless we're willing to duplicate the selective application.
Comment 48 Adam Langley 2012-01-17 12:22:41 PST
Chrome does have an exception list in which we permitted customers of a specific vendor who didn't have a firmware update available in time. This has been resolved so we'll be removing the exceptions.

Microsoft doesn't have an exception list.
Comment 49 [Baboo] 2012-01-17 12:45:41 PST
(In reply to Alex Keybl [:akeybl] from comment #45)
> The security team should be able to address the technical specifics, but
> there is currently no need for the workaround in Firefox. The other browsers
> you mention may have leveraged code that require the workaround.

Considering that IE and Opera have TLS 1.1 support while Firefox has not, I would argue in the opposite direction.
Comment 50 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2012-01-17 15:46:06 PST
We are going to keep the record splitting behavior enabled.

I met with product and clarified some things that I had not previously not communicated clearly to them. Although it seems to be the case that we do not need the record splitting behavior right now, we may very well need to do it later (and not much later). Also, having to revisit this issue every time we make a change to our networking code is going to be error-prone and slow down future feature development. Plus, I feel like I gave my word to other implementers (perhaps too eagerly) that Mozilla would go ahead with this change, and everybody at Mozilla wants to respect the informal agreement we had.

On what timeframe is Chrome planning to drop its exception list? I would like to avoid rushing to create an exception mechanism for this in Firefox. I think it would make our lives unnecessarily difficult w.r.t. cross-browser compatibility if we were to all implement our own exception mechanisms maintained independently. But, we (Mozilla) also cannot let just let Chrome or IE or other browsers be more compatible.
Comment 51 Alex Keybl [:akeybl] 2012-01-17 15:59:24 PST
(In reply to Brian Smith (:bsmith) from comment #50)
> We are going to keep the record splitting behavior enabled.

Thanks for getting us all on the same page Brian.
Comment 52 Adam Langley 2012-01-17 16:10:14 PST
(In reply to Brian Smith (:bsmith) from comment #50)
> On what timeframe is Chrome planning to drop its exception list?

We only added a handful of sites to the exception list for record splitting. All but three of them were handled by myself and I was clear that we could remove them at any time. Normally I'd leave them for the regular erosion process (where we remove a random percentage of the blacklist), but I'll manually remove them tomorrow.

I'll have to check with my colleague about the other three sites but I suspect that, now IE has updated, they've decided to fix their sites pretty promptly.
Comment 53 Martin Wilck 2012-01-18 03:06:31 PST
A hard-wired exception list won't solve all problems. In our case we are talking about a server management application which exhibits a web interface (and fixing it unfortunately is not as simple as outlined in comment #37). This application will never be run on a public web server in the Internet, but on many hosts inside a data center.

The only way to avoid our problems would be a user-customizable exception list.
Comment 54 Nick Lowe 2012-01-18 03:26:33 PST
@Martin - By the "only way to avoid", do you mean that you cannot feasibly develop / deliver a fix to the SSL/TLS implementation used?

If so, a global option to disable record splitting would, perhaps, be more appropriate than an exception list - and it should still only be seen as a temporary measure.

Record splitting has always been perfectly legal with regards to the the SSL 3.0 and TLS 1.0 specification.
Comment 55 Martin Wilck 2012-01-18 03:43:16 PST
> do you mean that you cannot feasibly develop / deliver a fix to the SSL/TLS implementation 
> used?

It's certainly feasible. Just that we haven't found the solution yet.

> If so, a global option to disable record splitting would, perhaps, be more appropriate

There's NSS_SSL_CBC_RANDOM_IV=0 for NSS based applications (under Linux, at least). But that affects all sites the browser connects to, not just our application.

> Record splitting has always been perfectly legal

Sure, but no application ever used it, so bugs related to record splitting never surfaced before.
Comment 56 John Beranek 2012-01-20 03:48:50 PST
(In reply to Nick Lowe from comment #54)
> @Martin - By the "only way to avoid", do you mean that you cannot feasibly
> develop / deliver a fix to the SSL/TLS implementation used?

Surely completely feasible that there are many systems that will never get a software/firmware update to change their SSL/TLS implementations. Think of obsoleted software or firmware...
Comment 57 Honza Bambas (:mayhemer) 2012-02-02 09:19:14 PST
*** Bug 723109 has been marked as a duplicate of this bug. ***
Comment 58 ben 2012-02-02 14:21:15 PST
So was there not a workaround to disable record splitting in Firefox 10? 

This problem doesn't just affect web servers. Websense Triton (SSL Filter) is NOT compatible with record splitting. Firefox 10 now no longer works inside my corporate environment (and a lot of other corporations too).

I'm able to get around the problem with Chrome and IE because they have workarounds. There needs to be a method to disable record splitting!
Comment 59 Adam Langley 2012-02-02 14:26:32 PST
(In reply to ben from comment #58)
> This problem doesn't just affect web servers. Websense Triton (SSL Filter)
> is NOT compatible with record splitting. Firefox 10 now no longer works
> inside my corporate environment (and a lot of other corporations too).

This is http://code.google.com/p/chromium/issues/detail?id=107728 for Chrome. I phoned WebSense on Dec 20th, 2011 and they indicated that they were aware and working to fix it.

Ben: for Firefox you can disable by exporting NSS_SSL_CBC_RANDOM_IV=0, I believe.
Comment 60 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2012-02-03 09:33:03 PST
(In reply to Adam Langley from comment #59)
> Ben: for Firefox you can disable by exporting NSS_SSL_CBC_RANDOM_IV=0, I
> believe.

Right, you can put NSS_SSL_CBC_RANDOM_IV=0 in your system or user environment, and that will turn off this workaround for all Gecko-based products (and maybe Chrome?). Or, you can create a startup wrapper script (batch file) that sets NSS_SSL_CBC_RANDOM_IV=0 before running a product, if you want to control do it for a single product.

It is important to contact your vendor (e.g. WebSense) about getting a fix so that you can undo the NSS_SSL_CBC_RANDOM_IV=0 workaround on your systems.
Comment 61 ben 2012-02-03 10:37:23 PST
Thank you for the advice. Adding the variable to my user environment did fix the issue I'm having with Websense.

Hopefully our sysadmins will fix the issue soon. I'm at their mercy as an end user.

I rely on Firefox for web development work.
Comment 62 WADA 2012-02-04 20:58:06 PST
FYI.
Developement team of Fedora looks to have decided to change default of ssl_defaults.cbcRandomIV to false on 2012-01-11, for compatibility. 
https://bugzilla.redhat.com/show_bug.cgi?id=770682
https://bugzilla.redhat.com/attachment.cgi?id=551038
>    If Environmnt variable of NSS_SSL_CBC_RANDOM_IV = 0, ssl_defaults.cbcRandomIV = PR_FALSE
> -> If Environmnt variable of NSS_SSL_CBC_RANDOM_IV = 1, ssl_defaults.cbcRandomIV = PR_TRUE
Comment 63 WADA 2012-02-04 21:37:59 PST
FYI. TechEvang Bug 698203 was found. It was somehow closed as FIXED on 2012-01-10.
Comment 64 Matthias Versen [:Matti] 2012-02-13 15:27:41 PST
*** Bug 726329 has been marked as a duplicate of this bug. ***
Comment 65 Matthias Versen [:Matti] 2012-02-14 15:09:29 PST
*** Bug 727196 has been marked as a duplicate of this bug. ***
Comment 66 James White 2012-02-16 19:28:32 PST
*** Bug 728107 has been marked as a duplicate of this bug. ***
Comment 67 Matthias Versen [:Matti] 2012-02-18 08:26:55 PST
*** Bug 728530 has been marked as a duplicate of this bug. ***
Comment 68 Matthias Versen [:Matti] 2012-03-14 17:04:19 PDT
*** Bug 735720 has been marked as a duplicate of this bug. ***
Comment 69 Brian Smith (:briansmith, :bsmith, use NEEDINFO?) 2012-03-26 23:59:32 PDT
*** Bug 723370 has been marked as a duplicate of this bug. ***
Comment 70 Gian-Carlo Pascutto [:gcp] 2012-05-07 11:35:48 PDT
>FYI. TechEvang Bug 698203 was found. It was somehow closed as FIXED on 2012-01-10.

Reopened because the users who reported it fixed now started complaining en masse again (I can easily reproduce the problem). They also report the workaround environment variable in this bug does not help.
Comment 71 Kai Engert (:kaie) 2012-05-07 11:44:14 PDT
(In reply to Gian-Carlo Pascutto (:gcp) from comment #70)
> >FYI. TechEvang Bug 698203 was found. It was somehow closed as FIXED on 2012-01-10.
> 
> Reopened because the users who reported it fixed now started complaining en
> masse again (I can easily reproduce the problem). They also report the
> workaround environment variable in this bug does not help.

Since when? Reported where? Which versions were better and which one is worse?
Comment 72 Gian-Carlo Pascutto [:gcp] 2012-05-07 11:46:14 PDT
Appears to have regressed in a recent Nightly *or* it's a regression of the site itself. The issue is easily reproducible, see bug 698203.
Comment 73 Robert Relyea 2012-05-07 12:45:10 PDT
I don't think it's the same issue. I just connected to both servers with FF 10.0.0.4 and NSS 3.13.1 and NSS_SSL_CBC_RANDOM_IV set to 1. Could this be a TLS 1.1 issue instead?
Comment 74 Kai Engert (:kaie) 2012-05-07 12:49:33 PDT
(In reply to Robert Relyea from comment #73)
> I don't think it's the same issue.

I agree. See also bug 698203 comment 26
Comment 75 Berni Elbourn 2012-10-23 00:21:00 PDT
What is the situation in ESR?  I am getting the "Connection Reset" behavior at times accessing https://twitter.com/ with Debian Squeeze iceweasel/Firefox with 10.0.9 Esr. No proxy involved. Refreshing the page does succeed.
Comment 76 Gian-Carlo Pascutto [:gcp] 2012-10-23 00:44:33 PDT
There is a workaround (see comments above) that you can use if you absolutely, positively need to communicate with a broken server. But it should be very hard to find those by now, as none of the newest browsers will work correctly with them.

It's highly unlike Twitter has those, so you are seeing something completely unrelated.

This bug should probably have been closed as WONTFIX - it's not a Firefox bug. In fact I will do this now.

Note You need to log in before you can comment on or make changes to this bug.