1793342 - fetch(), HTTP/3, and Streaming Responses result into promises that don’t resolve - unknownDecoder

Status: UNCONFIRMED

(Core :: DOM: Networking, defect, P3)

Description

[Leandro Facchinetti]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.1.15

Steps to reproduce:

I was talking to the developer of Caddy and it seems like this may be an issue in Firefox (see https://github.com/caddyserver/caddy/issues/5096).

Setup an HTTP/3 server with a streaming response:

$ npm install @leafac/caddy express

# Caddyfile
localhost
reverse_proxy localhost:4000

// index.mjs
import express from "express";
const app = express();
app.get("/", (req, res) => {
 res.send(`fetch("/stream").then(() => {console.log("PROMISE RESOLVED")})`);
});
app.get("/stream", (req, res) => {
 let heartbeatTimeout;
 (function heartbeat() {
   res.write("\n");
   heartbeatTimeout = setTimeout(heartbeat, 15 * 1000);
 })();
 res.once("close", () => {
   clearTimeout(heartbeatTimeout);
 });
});
app.listen(4000);

Note how the server has two routes: /, which is just a “hello world”, and /stream, which streams a heartbeat (a new line) every 15 seconds while the connection is kept alive.

You may run this with:

$ node index.mjs
$ npx @leafac/caddy run

Now go to Firefox (for what it’s worth, I’m on 105.0.1 in macOS), and try and get Firefox to load https://localhost with HTTP/3. You may check that you succeeded by going to the developer tools, under the network tab.

I’m not sure yet what makes Firefox prefer HTTP/3 over HTTP/2, but here are some of the things that worked for me:

• “Disable Cache” under the Network tab in the developer tools.
• Restart the browser.
• Reload a bunch of times.
• Repeat…

If y’all know of a better way, please let me know!

Anyway, when you finally get it, go to the Console in the developer tools and run the command shown to you on the page:

fetch("/stream").then(() => {console.log("PROMISE RESOLVED")})

Note how the promise never gets resolved!

What’s more, sometimes the requests don’t even show in the Network tab. Firefox really stopped playing ball… 🤷

Now, to show that this seems to be related to HTTP/3, change the Caddyfile to the following:

# Caddyfile
{
  servers {
    protocols h1 h2
  }
}
localhost
reverse_proxy localhost:4000

Note how we’re not listing HTTP/3 among the protocols.

Restart Caddy.

This time Firefox will make the request and resolve the promise every time.

Thank you very much!

---

For what it’s worth, here’s my browser user agent: User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:105.0) Gecko/20100101 Firefox/105.0

Actual results:

(Answer included in “What did you do? (steps to reproduce)”)

Expected results:

(Answer included in “What did you do? (steps to reproduce)”)

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Networking' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Dragana Damjanovic [:dragana]]

HTTP/2 response contains a content-type header and HTTP/3 response does not.

so in HTTP/3 case, it does not. Browser implements sniffing of types by looking at up to 521bytes (or until the end of the stream) to figure out the type.

Firefox is waiting for 512bytes or the end of the stream and this is what is blocking the promise.

I tried on Chrome, but I could not make it use HTTP/3 and HTTP/2 response contains the header the same as with Firefox.

Does it work for you with Chrome when HTTP/3 is used?

Comment 3

[Leandro Facchinetti]

Hi Dragana Damjanovic,

Thanks for the follow-up.

Does it work for you with Chrome when HTTP/3 is used?

Yes, it does 👍

II tried on Chrome, but I could not make it use HTTP/3 and HTTP/2 response contains the header the same as with Firefox.

Getting browsers to use HTTP/3 requires some patience 🤷

(By the way, do you know a way of forcing Firefox to use HTTP/3?)

HTTP/2 response contains a content-type header and HTTP/3 response does not.

so in HTTP/3 case it does not. BBrowser implement sniffingo f types by looking at up to 521bytes (or untill the end of stream) to figure out the type.

Firefox is waiting for 512bytes or the end of the steam and this is what is blocking the promise.

Great catch!

I confirmed the behavior you described:

1. Using Chrome I confirmed that the HTTP/2 response included a Content-Type header, while the HTTP/3 response did not. (By the way, how do you see the response headers of a streaming response in Firefox?)

2. I modified the server to always include a Content-Type header, and then Firefox resolved the promises even in HTTP/3.

   import express from "express";
   const app = express();
   app.get("/", (req, res) => {
     res.send(`fetch("/stream").then(() => {console.log("PROMISE RESOLVED")})`);
   });
   app.get("/stream", (req, res) => {
     let heartbeatTimeout;
     res.contentType("text/plain");
     (function heartbeat() {
       res.write("\n");
       heartbeatTimeout = setTimeout(heartbeat, 15 * 1000);
     })();
     res.once("close", () => {
       clearTimeout(heartbeatTimeout);
     });
   });
   app.listen(4000);
   

3. Alternatively, I removed the Content-Type header again and this time I made the heartbeat much larger, to fill in Firefox’s buffer and force content sniffing. This also made Firefox resolve promises.

   import express from "express";
   const app = express();
   app.get("/", (req, res) => {
     res.send(`fetch("/stream").then(() => {console.log("PROMISE RESOLVED")})`);
   });
   app.get("/stream", (req, res) => {
     let heartbeatTimeout;
     (function heartbeat() {
       res.write("\n".repeat(100_000));
       heartbeatTimeout = setTimeout(heartbeat, 15 * 1000);
     })();
     res.once("close", () => {
       clearTimeout(heartbeatTimeout);
     });
   });
   app.listen(4000);
   

All this evidence confirms your explanation: Firefox seems to wait indefinitely for enough bytes to try and sniff the content type.

At this point, there are two things that I think we should consider:

1. Firefox should resolve the promises. Perhaps with something like a timeout on the content sniffing…

2. Caddy should behave the same in HTTP/2 & HTTP/3 when it comes to adding the Content-Type header. (Note that it was Caddy who added the Content-Type header to the HTTP/2 response—the Express.js server didn’t generate that header.) Either add the header to both, or to neither. I’m going to bring this to Caddy’s team.

Thanks again for your help in this issue.

Comment 4

[Leandro Facchinetti]

Upon further experimentation I learned that even with the Content-Type header set, sometimes Firefox still doesn’t resolve the promise. Perhaps surprisingly, the solution was to pass the {cache: "no-store"} argument to fetch():

fetch("/stream", {cache: "no-store"}).then(() => {console.log("PROMISE RESOLVED")})

🤷

Comment 5

[Dragana Damjanovic [:dragana]]

Valentin, this is an interesting fetch bug. worth looking at why Chrome works and we do not. See comment #2 and also the following comments.

Comment 6

[Dragana Damjanovic [:dragana]]

I will not have time to work on this bug, so I am unassigning myself.

Comment 7

[Valentin Gosu [:valentin] (he/him)]

(In reply to Leandro Facchinetti from comment #4)

Upon further experimentation I learned that even with the Content-Type header set, sometimes Firefox still doesn’t resolve the promise. Perhaps surprisingly, the solution was to pass the {cache: "no-store"} argument to fetch():

fetch("/stream", {cache: "no-store"}).then(() => {console.log("PROMISE RESOLVED")})

This is an odd cornercase. I guess we ought to resolve it if possible. The impact is low, but you never know when this will cause webcompat issues.