1451951 - Firefox opens several TCP connections and ignores some of them, and may trigger TCP SYN Flood alerts

Status: RESOLVED INVALID

(Core :: Networking: HTTP, defect)

Description

[kuko]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36

Steps to reproduce:

We have a large amount of users accessing the same Web application
Recently, we found out that a we were accidentally triggering the SYN Flood protection on that system because we were reaching 625 incomplete TCP connection



Actual results:

After some tests we found out that on some of our windows clients, Firefox were opening several connections (sending a TCP SYN) and ignoring the SYN+ACK response from the server on some of those connections, leaving those connections half-open and triggering the SYN Flood protection on the server

We checked the TCP checksums and options and everything seems ok. It seems Firefox is not interested on those connections anymore because it doesn't retransmit the initial SYN packet as it should if it just dropped the SYN+ACK response because it was something wrong with it

We have seen this behavior only when using Firefox version 59.0.2, we have also users using Chrome and Firefox 46.0.1 and none of them seems to ignore the server's SYN+ACK response



Expected results:

Firefox should send the ACK to complete the TCP three-way-handshake for all TCP connections it initiates

Comment 1

[Gingerbread Man]

(In reply to kuko from comment #0)
> Steps to reproduce:

These aren't steps to reproduce the issue.

> We have seen this behavior only when using Firefox version 59.0.2, we have
> also users using Chrome and Firefox 46.0.1

It would help if you could find the regression range.
https://mozilla.github.io/mozregression/quickstart.html

It's also worth testing the latest Nightly in a brand new profile to see if the issue hasn't already been fixed.
https://nightly.mozilla.org
https://support.mozilla.org/kb/profile-manager-create-and-remove-firefox-profiles

Comment 2

[kuko]

Sorry, I don't know exactly how to reproduce it. It just happens once in a while
It seems to happen mostly with a web app that uses javascript

I can not test it because I'm not there (I just diagnosed the problem), But they told me it seems it's doesn't affect newer versions

I will try to confirm

Comment 3

[u408661]

The tcp-level stuff (the 3-way handshake, for example) is not controlled by firefox - we call connect(), and let the OS tcp stack set up the connection for us. So, if connections are, in fact, getting stuck half-open, that's almost certainly on the OS, some kind of anti-malware software on the host, or some middlebox messing with the tcp flow.

Given that you mention this *only* happens on 59.0.2, my best guess is there's some kind of anti-malware messing with 59.0.2 specifically. If you can verify that this doesn't happen on other versions, we can close this as invalid.

Comment 4

[kuko]

Yes, the 3-way handshake is handled by the OS. But I guess that if you call connect() from 10 threads and 5 of them dies just after calling connect, the SYN packet is already sent, but the OS won't be interested in opening the 5 connections initiated from the thread that died, and will ignore the SYN+ACK response

Comment 5

[Honza Bambas (:mayhemer)]

(In reply to kuko from comment #4)
> Yes, the 3-way handshake is handled by the OS. But I guess that if you call
> connect() from 10 threads and 5 of them dies just after calling connect, the
> SYN packet is already sent, but the OS won't be interested in opening the 5
> connections initiated from the thread that died, and will ignore the SYN+ACK
> response

No.  If you don't close the socket handle the handshake goes on in the kernel.  This has nothing to do with threads.  Handles are automatically cleaned up on a process exit, not thread destruction.  Also, we use only one thread for all net io that lives through out the whole browser session.

I'm afraid there is nothing we could do with this bug, if it's even a bug in our code base.  This is hard to reproduce for us to investigate.

Comment 6

[Damian Yerrick]

Bug 1622859 speculates that it may be related to race cache with network (RCWN).