Closed Bug 1698307 Opened 3 years ago Closed 3 years ago

Firefox [Nightly] can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream.

Categories

(Core :: Networking: WebSockets, defect, P1)

Product:
Core
Component:
Networking: WebSockets
Version:
Firefox 88
Platform:
x86_64
Windows 10
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
88 Branch
Tracking Flags:
Tracking Status
firefox-esr78 --- unaffected
firefox86 --- unaffected
firefox87 --- unaffected
firefox88 + verified

People

(Reporter: alex_mayorga, Assigned: kershaw)

References

(Regression,
URL
)

Details

(Keywords: regression, Whiteboard: [necko-triaged])

Attachments

(3 files)

bug-1698307.png
97.70 KB, image/png
Details
bug-1698307-101.png
130.74 KB, image/png
Details
Bug 1698307 - Make sure the npnToken is cleared if we don't want to use h2
48 bytes, text/x-phabricator-request
Details | Review

¡Hola!

I can't join any Discord server with Firefox Nightly.

Here's the output of the console when trying to join
https://discord.com/channels/696139060561051688/696139060561051691

Please let me know if there's anything further needed from the affected profile.

The messages start to repeat over and over.

¡Gracias!
Alex

23:11:18.658 Navigated to https://discord.com/channels/696139060561051688/696139060561051691
23:11:18.730 [GatewaySocket] [WS CLOSED] (true, 1006, ) retrying in 2.97 seconds. 31296b03bdb291fe6072.js:2:126313
23:11:18.814 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:11:19.798 Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). onloadwff.js:71:790852
23:11:20.423 [FAST CONNECT] wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream, encoding: json, version: 8 696139060561051691:47:334
23:11:20.549 unreachable code after return statement 2276137a5d7df04f45db.js:2:2904464
23:11:20.549 unreachable code after return statement 2276137a5d7df04f45db.js:2:2904513
23:11:20.778 [Logger.tsx] Logger has loaded 31296b03bdb291fe6072.js:2:126313
23:11:20.969 Cookie “locale” has “SameSite” policy set to “Lax” because it is missing a “SameSite” attribute, and “SameSite=Lax” is the default value for this attribute. 2276137a5d7df04f45db.js:2:1440047
23:11:21.214 unreachable code after return statement 2276137a5d7df04f45db.js:2:2904464
23:11:21.221 unreachable code after return statement 2276137a5d7df04f45db.js:2:2904513
23:11:21.595 [default] [BUILD INFO] Release Channel: stable, Build Number: 79478, Version Hash: b329d0aea41807066b25c73e2feca0f3ef4baa5b 31296b03bdb291fe6072.js:2:126313
23:11:22.424 [Flux] Slow batch emitChanges took 178ms recentActions: EXPERIMENT_REGISTER_LEGACY -> I18N_LOAD_START -> STREAMING_UPDATE -> ACCESSIBILITY_SYSTEM_PREFERS_REDUCED_MOTION_CHANGED -> I18N_LOAD_SUCCESS 31296b03bdb291fe6072.js:2:126313
23:11:22.426 [Flux] Slow dispatch on I18N_LOAD_SUCCESS: 185ms 31296b03bdb291fe6072.js:2:126313
23:11:22.643 Source map error: Error: request failed with status 404
Resource URL: https://discord.com/assets/31296b03bdb291fe6072.js
Source Map URL: 31296b03bdb291fe6072.js.map
23:11:22.709 Source map error: Error: request failed with status 404
Resource URL: https://discord.com/assets/2276137a5d7df04f45db.js
Source Map URL: 2276137a5d7df04f45db.js.map
23:11:26.533 [GatewaySocket] Connected, current state is 0 31296b03bdb291fe6072.js:2:126313
23:11:26.537 [GatewayDiscovery] [STICKY] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:11:26.628 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:11:46.645 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20014 ms - did not receive OP_HELLO in time.) retrying in 2.76 seconds. 31296b03bdb291fe6072.js:2:126313
23:11:49.406 [GatewayDiscovery] [STICKY] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:11:49.408 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:11:52.205 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:12:09.414 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20004 ms - did not receive OP_HELLO in time.) retrying in 2.91 seconds. 31296b03bdb291fe6072.js:2:126313
23:12:12.202 Firefox can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream. 696139060561051691:47:396
23:12:12.335 [GatewayDiscovery] [STICKY] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:12:12.337 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:12:32.346 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20005 ms - did not receive OP_HELLO in time.) retrying in 5.24 seconds. 31296b03bdb291fe6072.js:2:126313
23:12:37.584 [GatewayDiscovery] [STICKY] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:12:37.586 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:12:57.604 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20015 ms - did not receive OP_HELLO in time.) retrying in 12.21 seconds. 31296b03bdb291fe6072.js:2:126313
23:13:09.823 [GatewayDiscovery] [STICKY] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:13:09.824 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:13:12.206 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:13:29.829 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20003 ms - did not receive OP_HELLO in time.) retrying in 34.31 seconds. 31296b03bdb291fe6072.js:2:126313
23:13:29.830 [GatewaySocket] [RESET] (false, 0, The connection timed out after 20003 ms - did not receive OP_HELLO in time.) 31296b03bdb291fe6072.js:2:126313
23:13:29.839 [AuthenticationStore] handleConnectionClosed called with code 0.
Object { tokenManagerHasToken: true, storageHasToken: false }
31296b03bdb291fe6072.js:2:126313
23:13:32.209 Firefox can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream. 2276137a5d7df04f45db.js:2:3634847
23:14:04.139 [GatewayDiscovery] [DISCOVERING] 31296b03bdb291fe6072.js:2:126313
23:14:04.246 [GatewayDiscovery] [DISCOVERED] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:14:04.247 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:14:24.262 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20007 ms - did not receive OP_HELLO in time.) retrying in 60.00 seconds. 31296b03bdb291fe6072.js:2:126313
23:14:24.263 [GatewaySocket] [RESET] (false, 0, The connection timed out after 20007 ms - did not receive OP_HELLO in time.) 31296b03bdb291fe6072.js:2:126313
23:14:24.264 [AuthenticationStore] handleConnectionClosed called with code 0.
Object { tokenManagerHasToken: true, storageHasToken: false }
31296b03bdb291fe6072.js:2:126313
23:14:32.225 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:14:32.267 Firefox can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream. 2276137a5d7df04f45db.js:2:3634847
23:15:24.816 [GatewayDiscovery] [DISCOVERING] 31296b03bdb291fe6072.js:2:126313
23:15:24.902 [GatewayDiscovery] [DISCOVERED] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:15:24.903 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:15:32.267 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:15:32.314 Firefox can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream. 2276137a5d7df04f45db.js:2:3634847
23:15:45.041 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20136 ms - did not receive OP_HELLO in time.) retrying in 60.00 seconds. 31296b03bdb291fe6072.js:2:126313
23:15:45.041 [GatewaySocket] [RESET] (false, 0, The connection timed out after 20136 ms - did not receive OP_HELLO in time.) 31296b03bdb291fe6072.js:2:126313
23:15:45.042 [AuthenticationStore] handleConnectionClosed called with code 0.
Object { tokenManagerHasToken: true, storageHasToken: false }
31296b03bdb291fe6072.js:2:126313
23:16:32.319 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:16:32.359 Firefox can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream. 2276137a5d7df04f45db.js:2:3634847
23:16:45.053 [GatewayDiscovery] [DISCOVERING] 31296b03bdb291fe6072.js:2:126313
23:16:45.155 [GatewayDiscovery] [DISCOVERED] wss://gateway.discord.gg 31296b03bdb291fe6072.js:2:126313
23:16:45.156 [GatewaySocket] [CONNECT] wss://gateway.discord.gg, encoding: json, version: 8, compression: zlib-stream 31296b03bdb291fe6072.js:2:126313
23:17:05.292 [GatewaySocket] [WS CLOSED] (false, 0, The connection timed out after 20132 ms - did not receive OP_HELLO in time.) retrying in 60.00 seconds. 31296b03bdb291fe6072.js:2:126313
23:17:05.294 [GatewaySocket] [RESET] (false, 0, The connection timed out after 20132 ms - did not receive OP_HELLO in time.) 31296b03bdb291fe6072.js:2:126313
23:17:05.296 [AuthenticationStore] handleConnectionClosed called with code 0.
Object { tokenManagerHasToken: true, storageHasToken: false }
31296b03bdb291fe6072.js:2:126313
23:17:32.361 Content Security Policy: Ignoring “'unsafe-inline'” within script-src or style-src: nonce-source or hash-source specified
23:17:32.393 Firefox can’t establish a connection to the server at wss://gateway.discord.gg/?encoding=json&v=8&compress=zlib-stream. 2276137a5d7df04f45db.js:2:3634847
23:18:05.732 [GatewayDiscovery] [DISCOVERING] 31296b03bdb291fe6072.js:2:126313

¡Hola y'all!

This is super weird, once I load the exact same URL in Chromium. Firefox Nightly will then be able to establish the WebSocket and work.

Please see the screen captures attached.

¡Gracias!
Alex

Attached image bug-1698307.png —
Attached image bug-1698307-101.png —

Log from about:networking#logging is at https://we.tl/t-5Wur7h4qIu

Kershaw, can you take a look?

Flags: needinfo?(kershaw)

(In reply to alex_mayorga from comment #4)

Log from about:networking#logging is at https://we.tl/t-5Wur7h4qIu

It seems the log is not available anymore. Could you upload it again?
As per comment #1, are you able to reproduce this now?

Thanks.

Flags: needinfo?(kershaw) → needinfo?(alex_mayorga)

(In reply to alex_mayorga from comment #1)

¡Hola y'all!

This is super weird, once I load the exact same URL in Chromium. Firefox Nightly will then be able to establish the WebSocket and work.

Please see the screen captures attached.

¡Gracias!
Alex

Looks like this problem is related to HTTP/2. I can see in the logs that firefox tries to migrate all new connections to an existing one over and over again:

02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: E/nsHttp nsHttpConnection::SetupSSL 1bb2e456c00 caps=0x40810C .S......[tlsflags0x00000000]gateway.discord.gg:443 {NPN-TOKEN h2}^partitionKey=%28https%2Cdiscord.com%29
02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: V/nsHttp FindCoalescableConnectionByHashKey() found match conn=1bb2e456c00 key=162.159.134.234~.:443/[^partitionKey=%28https%2Cdiscord.com%29]viaDNS newCI=.S...X..[tlsflags0x00000000]gateway.discord.gg:443 {NPN-TOKEN h2}^partitionKey=%28https%2Cdiscord.com%29 matchedCI=.S......[tlsflags0x00000000]gateway.discord.gg:443 {NPN-TOKEN h2}^partitionKey=%28https%2Cdiscord.com%29 join ok
02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: V/nsHttp FindCoalescableConnection(.S...X..[tlsflags0x00000000]gateway.discord.gg:443 {NPN-TOKEN h2}^partitionKey=%28https%2Cdiscord.com%29) match conn 1bb2e456c00 on dns key 162.159.134.234~.:443/[^partitionKey=%28https%2Cdiscord.com%29]viaDNS
02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: V/nsHttp UpdateCoalescingForNewConn() found existing active conn that could have served newConn graceful close of newConn=1bb3ed27400 to migrate to existingConn 1bb2e456c00
02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: V/nsHttp nsHttpConnection::AddTransaction [this=1bb3ed27400] for SPDY websocket
02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: I/nsHttp Http2Session::AddStream first session=1bb3b8cd000 trans=1bb423ae000 
02021-03-14 06:44:00.776000 UTC - [Parent 640: Socket Thread]: I/nsHttp Http2Session::AddStream 1bb3b8cd000 atrans=1bb423ae000 trans=1bb423ae000 session unusable - resched.

If you hit the problem again, you can work around it by setting network.http.spdy.websockets preference to false.

(In reply to Kershaw Chang [:kershaw] from comment #6)

(In reply to alex_mayorga from comment #4)

Log from about:networking#logging is at https://we.tl/t-5Wur7h4qIu

It seems the log is not available anymore. Could you upload it again?
As per comment #1, are you able to reproduce this now?

Thanks.

Ah, I can download the log now.

Flags: needinfo?(alex_mayorga)

This bit me today, and I can confirm that setting network.http.spdy.websockets preference to false fixed the issue for me.

I'm seeing this too since yesterday. Would be great if we could get a regression range.

status-firefox86: --- → unaffected
status-firefox87: --- → unaffected
status-firefox-esr78: --- → unaffected
tracking-firefox88: --- → +

Trying to open Discord in a fresh profile worked, so I'm confused. Maybe this is connected to cached connection settings?

QA Whiteboard: [qa-regression-triage]

This is regressed by bug 1686421.

The problem is that gateway.discord.gg has an HTTPS RR 1 gateway.discord.gg (alpn="h2" ipv4hint="162.159.130.234, 162.159.133.234, 162.159.134.234, 162.159.135.234, 162.159.136.234" ), so we set the npnToken of the connection info to h2. After a while, DisableSpdy() is called, but in nsHttpConnectionInfo::SetNoSpdy, the npnToken is not cleared. In the end, we still use this h2 npnToken to connect next time.

Assignee: nobody → kershaw
Severity: -- → S2
Keywords: regressionwindow-wanted
Priority: -- → P1
Regressed by: 1686421
Whiteboard: [necko-triaged]
Has Regression Range: --- → yes
Pushed by kjang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e82cd674b22c
Make sure the npnToken is cleared if we don't want to use h2 r=necko-reviewers,dragana

https://hg.mozilla.org/mozilla-central/rev/e82cd674b22c

Status: NEW → RESOLVED
Closed: 3 years ago
status-firefox88: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 88 Branch
Flags: qe-verify+

We tried to reproduce this issue on an affected Nightly build 2021-03-13 using the steps from comment 0, but we couldn't reproduce it on Windows 10x64. We were able to connect to Discord without any problems.

Are there any other details we can use in order to reproduce this? Thanks.

Flags: needinfo?(alex_mayorga)

I was able to reproduce this easily back when it originally broke. No issues since the fix landed.

Status: RESOLVED → VERIFIED
status-firefox88: fixed → verified
Flags: needinfo?(alex_mayorga)

Thanks Ryan for verifying this issue.

Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: