983243 - WSS WebSocket leaks TCP/IP connection if closed prematurely

Status: RESOLVED FIXED

(Core :: Networking: WebSockets, defect)

Description

[Benoit]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.74.9 (KHTML, like Gecko) Version/7.0.2 Safari/537.74.9

Steps to reproduce:

I run the following JavaScript against a WSS server listening on localhost port 10003

    console.log("opening socket");
    var fd = new WebSocket("wss://127.0.0.1:10003", "ice.zeroc.com");
    fd.onclose = function(e) { console.log("closed socket "); };
    setTimeout(function() {
        console.log("closing socket ");
        fd.close();
    }, 200);



Actual results:

The console reports:

18:29:04.480 "opening socket" Client.js:12
18:29:04.681 "closing socket " Client.js:16
18:29:04.684 "closed socket "  Client.js:14

But the TCP/IP connection isn't closed (check with netstat -an | grep 10003). Closing the tab doesn't close it either, the only way to close the connection is to close the browser.


Expected results:

The TCP/IP connection should have been closed.

Note that this occurs only for WSS connections... got the problem with Firefox 27 on OS X and Windows.

Comment 1

[Valentin Gosu [:valentin] (he/him)]

(In reply to Benoit from comment #0)
> But the TCP/IP connection isn't closed (check with netstat -an | grep
> 10003). Closing the tab doesn't close it either, the only way to close the
> connection is to close the browser.
> 
> Expected results:
> 
> The TCP/IP connection should have been closed.
> 
> Note that this occurs only for WSS connections... got the problem with
> Firefox 27 on OS X and Windows.

Hi Benoit,

Can you please tell me what the state of the socket is? (ESTABLISHED, TIME_WAIT, etc)
Could you paste the output of netstat in bugzilla?
Also, can you check if the connection is still open after 5 minutes?

Comment 2

[Benoit]

Attachment: test.html

Here's the test.html file I use to reproduce the bug.

Comment 3

[Benoit]

Attachment: log.txt

Here's the log file obtained from Firefox 29.0.1 on OS X 10.9.x

Comment 4

[Benoit]

To reproduce the leak:

- you must have a web socket server listening for WSS on localhost and port 10003
- you must ensure the server certificate is valid (in my case, I added a CA cert to allow connections to my server which uses a certificate that wasn't signed by a well-known CA).
- it looks like it's a race condition, it doesn't always happen. It happens if the connection is closed during the connection establishment.

When it occurs the socket is left opened:

tcp4       0      0  127.0.0.1.10003        127.0.0.1.52095        ESTABLISHED
tcp4     164      0  127.0.0.1.52095        127.0.0.1.10003        ESTABLISHED
tcp4       0      0  127.0.0.1.10003        *.*                    LISTEN     

Here's firefox console output when this occurs:

GET http://localhost/test.html [HTTP/1.1 304 Not Modified 0ms]
GET https://127.0.0.1:10003/ [HTTP/1.1 101 Switching Protocols 10ms]
Firefox ne peut établir de connexion avec le serveur à l'adresse wss://127.0.0.1:10003/. test.html:6
"opening socket" test.html:5
"closing socket " test.html:9
"closed socket " test.html:7
La connexion avec wss://127.0.0.1:10003/ a été interrompue pendant le chargement de la page.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Attachment: WSS WebSocket leaks TCP/IP connection if closed prematurely

Prevent WebSocketChannel::OnTransportAvailable from reinitializing the channel's references after it has been closed.

Comment 6

[Valentin Gosu [:valentin] (he/him)]

Thanks for the great info and logs, Benoit!

Analysis of the bug:
It seems that if the websocket is closed before it has finished negotiating, WebSocketChannel::OnTransportAvailable is called after WebSocketChannel::AbortSession(). While AbortSession will release mSocketIn/mSocketOut/mTransport, OnTransportAvailable gets the references, so the connection wasn't being closed until shutdown.

The patch returns early in OnTransportAvailable if mStopped is set.

The bug is easily reproducible. After applying the patch I was unable to reproduce it again.
This patch should also fix Bug 765738 and possibly Bug 788913 as well.

Comment 7

[Patrick McManus [:mcmanus]]

Comment on attachment 8432025 [details] [diff] [review]
WSS WebSocket leaks TCP/IP connection if closed prematurely

Review of attachment 8432025 [details] [diff] [review]:
-----------------------------------------------------------------

nice explanation. a 3 fer.. dup those bugs!

Comment 8

[Valentin Gosu [:valentin] (he/him)]

try: https://tbpl.mozilla.org/?tree=Try&rev=80a0f9ecaff9
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/dbbd754c9a24

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/dbbd754c9a24

Comment 12

[Paul Fee]

I still see this (or a similar) issue.

I'm using Firefox 30 on Linux (Distro: Fedora 20, Package: firefox-30.0-4.fc20.x86_64).  A page visited using http (port 80) opens a couple of websockets (to the same server port 80).  A websocket has remained open even after the page's tab has been closed.

Netstat and lsof show that firefox still has an established TCP socket to the server.

Closing all tabs, with the one remaining pointed to "about:blank" should close all sockets to remote servers.  However the socket remains open indefinitely until the Firefox process exits.

Since there's an "about:webrtc" page, it would be nice if there was a similar "about:websockets" diagnostics page.

Comment 13

[Patrick McManus [:mcmanus]]

paul - this bug is marked fixed in firefox 32 and you're testing with firefox 30.

Comment 14

[Paul Fee]

Thanks for the clarification.  I misinterpreted the "Version" field as the fix version rather than the version the bug was originally reported against.  I'll investigate the Aurora channel to see if the fix works.