WebTransport datagram write() promises never settle once neqo queues a datagram above its high water mark, stalling datagrams.writable
Categories
(Core :: Networking: HTTP, defect)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox-esr115 | --- | unaffected |
| firefox-esr153 | --- | unaffected |
| firefox160 | + | fixed |
| firefox161 | + | fixed |
| firefox162 | + | fixed |
People
(Reporter: jesup, Assigned: jesup, NeedInfo)
References
(Regression)
Details
(Keywords: regression, Whiteboard: [necko-triaged], [wptsync upstream error])
Attachments
(6 external files)
Reported in https://github.com/mozilla/neqo/issues/4306 against Firefox 160.0 (neqo 0.33.0).
Http3Session::SendDatagram settles a datagram write immediately only when neqo's enqueue returns DatagramQueueOutcome::Ok. For any other outcome (MaxBufferedReached, Overflowed, Rejected) it waits for a per-datagram OutgoingDatagramOutcome, which never arrives:
- Http3Client drops ConnectionEvent::OutgoingDatagramOutcome, so no outcome reaches the glue.
- neqo has no outcome at all for a queued datagram that later expires or is evicted, and a Rejected datagram is never queued.
Once outgoingMaxBufferedDatagrams (default 5) such writes are outstanding, content withholds every further write, so datagrams.writable accepts nothing for the rest of the session. The page's writer.ready stays pending; incoming datagrams keep arriving.
Steps to reproduce (from the GitHub report):
- Run the aioquic sink (server.py in the issue) and serve the index.html page from the issue over a local HTTP server.
- Open the printed URL; the page writes maxDatagramSize-4 byte datagrams as fast as writer.ready allows.
Actual: within about 0.1s a write never settles (5 of 5 unpaced runs over loopback, around write #250, where neqo's 256 KiB queue budget fills). Paced at 140-300 Mbit/s against a remote server it also stalls.
Expected: every write settles, as https://w3c.github.io/webtransport/#webtransportdatagramswritable requires; dropped datagrams free their write.
The real fix is the credit-grant series in bug 1818883, which sends datagrams untracked and paces write() against neqo's queue capacity, but it depends on APIs new in neqo 0.34. For 160 a Gecko-only stopgap settles every write once neqo accepts the datagram, and reports a Rejected datagram as dropped (DROPPED_QUEUE_FULL), with a WPT that every write() of a burst overrunning the outgoing queue settles.
Related: bug 2084924 (same symptom when the session closes concurrently), bug 2083166 (datagram backpressure).
| Assignee | ||
Comment 1•5 days ago
|
||
neqo reports no outcome for a queued datagram that later expires or is
evicted, and Http3Client drops the outcomes it does report, so a write()
waiting for one never settled and the datagram writable stalled for the
rest of the session. Settle every write once neqo has accepted the
datagram, and report a datagram neqo refuses as dropped.
| Assignee | ||
Comment 2•5 days ago
|
||
Comment 4•5 days ago
|
||
@jesup, is this low risk to include in the dot release that builds on Monday?
If so, given it's an S2, could you add beta and release uplift requests now?
Updated•5 days ago
|
| Assignee | ||
Comment 6•5 days ago
|
||
neqo reports no outcome for a queued datagram that later expires or is
evicted, and Http3Client drops the outcomes it does report, so a write()
waiting for one never settled and the datagram writable stalled for the
rest of the session. Settle every write once neqo has accepted the
datagram, and report a datagram neqo refuses as dropped.
Original Revision: https://phabricator.services.mozilla.com/D342570
Updated•5 days ago
|
Comment 7•5 days ago
|
||
firefox-beta Uplift Approval Request
- User impact if declined/Reason for urgency: Regression that breaks datagrams in WebTransport
- Code covered by automated testing?: yes
- Fix verified in Nightly?: no
- Needs manual QE testing?: no
- Steps to reproduce for manual QE testing:
- Risk associated with taking this patch: low
- Explanation of risk level: Exceedingly simple patch; low/no usage of datagrams in production websites
- String changes made/needed?: none
- Is Android affected?: yes
| Assignee | ||
Comment 8•5 days ago
|
||
Original Revision: https://phabricator.services.mozilla.com/D342571
| Assignee | ||
Comment 9•5 days ago
|
||
neqo reports no outcome for a queued datagram that later expires or is
evicted, and Http3Client drops the outcomes it does report, so a write()
waiting for one never settled and the datagram writable stalled for the
rest of the session. Settle every write once neqo has accepted the
datagram, and report a datagram neqo refuses as dropped.
Original Revision: https://phabricator.services.mozilla.com/D342570
Updated•5 days ago
|
Comment 10•5 days ago
|
||
firefox-release Uplift Approval Request
- User impact if declined/Reason for urgency: Regression that breaks datagrams in WebTransport
- Code covered by automated testing?: yes
- Fix verified in Nightly?: no
- Needs manual QE testing?: no
- Steps to reproduce for manual QE testing:
- Risk associated with taking this patch: low
- Explanation of risk level: Exceedingly simple patch; low/no usage of datagrams in production websites
- String changes made/needed?: none
- Is Android affected?: yes
| Assignee | ||
Comment 11•5 days ago
|
||
Original Revision: https://phabricator.services.mozilla.com/D342571
Comment 13•5 days ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/e04bcc8b9ab2
https://hg.mozilla.org/mozilla-central/rev/71e3c94e7d2c
Updated•4 days ago
|
Updated•4 days ago
|
Updated•4 days ago
|
Comment 14•4 days ago
|
||
| uplift | ||
Updated•3 days ago
|
Updated•3 days ago
|
Updated•3 days ago
|
Comment 15•3 days ago
|
||
| uplift | ||
Comment 16•2 days ago
|
||
(In reply to Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org) from comment #12)
Failed to create upstream wpt PR due to merge conflicts. This requires fixup
from a wpt sync admin.
Creating the upstream PR requires https://github.com/web-platform-tests/wpt/pull/64272 to be merged first.
Updated•1 day ago
|
Description
•