1655372 - Websockets fails to connect and enters a connection loop when using an http/2 proxy

Status: UNCONFIRMED

(Core :: Networking: Proxy, defect, P3)

Description

[u667372]

Attachment: log.txt-main.11080.moz_log

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 Edg/84.0.522.44

Steps to reproduce:

In Firefox 79.0b9 Developer Edition on Windows 10 1909:

1. Configure Firefox to use an HTTP/2 TLS web proxy with a PAC URL.
2. Click connect button on a simple Websocket test page:
   http://websocket.org/echo.html

Actual results:

The WebSocket connection is not established. In Wireshark with TLS decryption configured, I see that Firefox sends a SETTINGS frame immediately followed by a GOAWAY frame to the proxy server, before the server has a chance to acknowledge the SETTINGS frame or send any kind of reply to Firefox. This repeats over and over in a loop, flooding the proxy server with new connections that are immediately closed with the GOAWAY frame sent by Firefox. Firefox never sends a HEADERS frame with a CONNECT request to the proxy to connect to the WebSocket server, only the SETTINGS and GOAWAY frames.

Regular HTTP and HTTPS pages load fine with the same HTTP/2 TLS proxy, it's only WebSockets that don't work. The same problem occurs with the HTTPS version of the test web site. https://websocket.org/echo.html

If network.http.spdy.websockets is set to false, the WebSocket connection test works correctly without any problems with the same HTTP/2 TLS web proxy.

Expected results:

The WebSocket connection should have been established normally with the HTTP/2 TLS web proxy.

The proxy server is HAProxy 2.0 (TLS h2 listener) with Apache httpd 2.4 mod_proxy. The specific proxy software isn't important, because the error condition occurs before it gets any response from the proxy server.

I found some similar bugs but they didn't help with identifying the root cause: 1514688, 1523427, 1563394.

I captured a Firefox network log (see attached file) with the following settings to help with identifying the problem: timestamp,nsHttp:4,cache2:3,nsSocketTransport:3,nsHostResolver:3,nsWebSocket:5

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[u667372]

After testing with nghttpx (a simple HTTP/2 proxy), the connection loop problem was the same there. However, I discovered one case where the WebSocket test always seems to connect successfully through the HTTP/2 proxy without the connection loop:

1. Start nghttpx in --http2-proxy mode with a simple HTTP/1.1 forward proxy such as Apache httpd mod_proxy as the backend.
2. Navigate to http://websocket.org/echo.html
3. Leave the web page open in Firefox, stop nghttpx (which ends all the connections to the proxy from Firefox)
4. Start nghttpx again, do not reload the page in Firefox.
5. Click the Connect button on the WebSocket test web page.
   (nghttpx is part of the nghttp2 package on Fedora/Ubuntu, or nghttp2 Cygwin package on Windows.)

It looks like the connection loop occurs when Firefox already has existing open connections to the proxy. See attached log of the successful connection: log.txt-main.1724.moz_log

This was tested with Firefox Developer Edition 79.0b9 (same as before):
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0

Comment 3

[u667372]

Attachment: log.txt-main.1724.moz_log - successful connection

Comment 4

[Yegor]

Can confirm we're having the same problem with http/2 proxies as of FF79. Downgrading to http/1.1 "fixes" the problem.

Comment 5

[BMO Automation]

Moving bug to Core/Networking: Proxy.

Comment 6

[BMO Automation]

Moving bug to Core/Networking: Proxy