1528850 - ws-over-h2 receive seems unreliable

Status: VERIFIED FIXED

(Core :: Networking: WebSockets, defect, P1)

Description

[Andy Green]

User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

On Firefox65, vist
https://libwebsockets.org/testserver/

and select Server Info tab. Or, to reproduce locally

$ git clone https://libwebsockets.org/repo/libwebsockets
$ cd libwebsockets
$ mkdir build
$ cd build
$ cmake .. -DLWS_WITH_HTTP2=1
$ make -j 12 && sudo make install
$ sudo ldconfig
$ libwebsockets-test-server -s

Actual results:

If no other server connected, your server info does not appear and the browser closes the connection after 30s.

The server believes it sent the data, and the data always appears correctly in Chrome or Chrome Canary with --enable-websocket-over-http2 (ie, ws-over-h2 mode).

Expected results:

The browser user agent should have been listed as connected.

The data is sent from the server as JSON using ws fragmentation, always a TEXT + NO FIN at the start, as many CONTINUATION + NO FIN as there are browsers connected and then a CONTINUATION with a FIN at the end.

If another browser connects, so the data sent is in more fragments / larger, then the data showing both appears correctly. But the first message is lost.

Comment 1

[u554753]

Moving to Core: Networking WebSockets. Please correct if this is not the right component for this issue.

Comment 2

[Junior [inactive]]

Hello reporter,
Can you check if network.http.spdy.websockets is true in about:config?
If not, set to true and hard refresh again to test.

Comment 3

[Andy Green]

Can you check if network.http.spdy.websockets is true in about:config?

Yes, it's true.

I previously corresponded with the author of the related code at Mozilla, and helped him with testing a few months ago, but his email doesn't work any more.

Comment 4

[Junior [inactive]]

Gecko 65 works for me with https://libwebsockets.org/testserver/

Could you try to reproduce in a new profile?

Comment 5

[Honza Bambas (:mayhemer)]

Please feel free to provide any new information and reopen

Comment 6

[Andy Green]

Have you tried to reproduce it using the local method I originally reported?

The nature of the bug as told in the original title is it is "unreliable". Broadly it works, but dome rx packets are missing (and having gone on using it, there's some problem with RFC8441 stream close as well, closes are delayed in a sticky way that outlasts the tab closing).

There are not that many implementations of RFC8441, libwebsockets was AFAIK the earliest server-side one and works very well with Chrome Canary's implementation. Thee problems are pretty sure to indicate some real problem either with libwebsockets or firefox RFC8441 implementation...

Comment 7

[Junior [inactive]]

(In reply to Andy Green from comment #6)

Have you tried to reproduce it using the local method I originally reported?

Simply tried the test page only.

Comment 8

[Michal Novotny [:michal]]

(In reply to Andy Green from comment #6)

Have you tried to reproduce it using the local method I originally reported?

I tried to build the test locally, but when I run it the server resets the connection without returning anything. How can I find out what's wrong with the installation? It doesn't seem to print any error:

./libwebsockets-test-server -s --port=8888
[2019/11/05 18:17:46:5792] N: libwebsockets test server - license LGPL2.1+SLE
[2019/11/05 18:17:46:5793] N: (C) Copyright 2010-2018 Andy Green <[e-mail]>
Using resource path "/opt/moz/bugzilla/bug1528850_libwebsocket_unreliable/libwebsockets_install/share/libwebsockets-test-server"
[2019/11/05 18:17:46:5842] N: SSL ciphers: 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4:!HMAC_SHA1:!SHA1:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES128-SHA256:!AES128-GCM-SH...
[2019/11/05 18:17:46:5842] N: Using SSL mode
[2019/11/05 18:17:46:5869] N: SSL ECDH curve 'prime256v1'
[2019/11/05 18:17:46:6071] N: vhost default: cert expiry: 9998d

Comment 9

[Andy Green]

For this test app, it wants -p 8888

that's probably all it is

Comment 10

[Michal Novotny [:michal]]

I can reproduce it using a local test and it's definitely a bug in Firefox.

Comment 11

[Michal Novotny [:michal]]

Attachment: Bug 1528850 - ws-over-h2 receive seems unreliable, r=dragana

The patch fixes following problems in TunnelUtils:

• InputStreamShim::AsyncWait() doesn't notify the callback immediately when some data is available in the buffer
• InputStreamShim and OutputStreamShim don't hold strong reference to the callback
• InputStreamShim::AsyncWait() and OutputStreamShim::AsyncWait() are not one-shot, i.e. the reference to the callback is not released after calling nsIInputStreamCallback::OnInputStreamReady() and nsIOutputStreamCallback::OnOutputStreamReady()

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bc2e858eac77
ws-over-h2 receive seems unreliable, r=dragana

Comment 13

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/bc2e858eac77

Comment 14

[Andy Green]

I tried the Firefox 72 beta, this seems to be in good shape now.

Thanks a lot for fixing it.