1544313 - fetch/response/body stream buffers data when Content-Type is not set

Status: UNCONFIRMED

(Core :: DOM: Networking, defect, P3)

Description

[ondra zara]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

Streaming the fetch's response body:

let response = await fetch("/");
let reader = response.body.getReader();
while (1) {
  let { ... } = await reader.read();
  ...
}

Actual results:

The read() call waits for the whole response, even for chunked Content-Transfer-Encoding. This happens only when there is no Content-Type response header.

Expected results:

The read() should resolve a promise for every chunk received. This happens when a Content-Type header is present.

Comment 1

[Luca Casonato [:lucacasonato]]

I can still reproduce this bug. The buffered amount at the start of the stream seems to be around 512 bytes. After that the entire first 512 bytes are emitted in a single chunk, and after that everything works as expected. Would the addition of a WPT test here be helpful?

Comment 2

[Luca Casonato [:lucacasonato]]

To elaborate, for me the promise returned from fetch does not even resolve until the first 512 bytes of data have been sent over the stream.

Comment 3

[Anne (:annevk)]

A WPT test would be great!

I suspect you are trying to use this for streaming and having to stream 512 bytes first is rather problematic?

Comment 4

[Luca Casonato [:lucacasonato]]

Added a test in https://github.com/web-platform-tests/wpt/pull/29698.

I suspect you are trying to use this for streaming and having to stream 512 bytes first is rather problematic?

It's mainly just really unexpected that in Chrome the streaming works right away, but in FF it hangs until the first 512 bytes have been sent. Took me a few hours to debug this and narrow it down to a small reproduction, and wouldn't want other people to run into the same issue. Once I figured out what the issue was I just added a content-type header and everything worked as expected.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

This is probably caused by the content type sniffer.
We should probably bypass it for fetch requests.