1684634 - WebAssembly custom fetch Response error (ignoring headers)

Status: RESOLVED FIXED

(Core :: DOM: Networking, defect, P2)

Description

[olliedawes]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0

Steps to reproduce:

I am attempting to fetch and compile a .wasm file with WebAssembly.compileStreaming while getting progress events so that I can display a progress indicator to the user.

I am doing this by taking the Response object from my fetch request and creating a new Response object which a custom ReadableStream which calls my progressHandler event function.

This approach works in Chrome but in FireFox I get: TypeError: Response has unsupported MIME type. Despite the Response object containing "application/wasm" for it's content type in res.headers.get("content-type")

For a code example see my original slatckoverflow question on the topic: https://stackoverflow.com/a/65529994/14807704

Or see: https://www.canonic.com/ for an actual "live" example of the problem.
The error occurs in the file https://www.canonic.com/qtloader.js line 313 in this block of code:

return WebAssembly.compileStreaming(res).catch(function(error) {
    console.log(error)
    // compileStreaming may/will fail if the server does not set the correct
    // mime type (application/wasm) for the wasm file. Fall back to fetch,
    // then compile in this case.
    return fetchThenCompileWasm(res);
});

Actual results:

WebAssembly.compileStreaming throws: TypeError: Response has unsupported MIME type

Expected results:

No exception should have been thrown.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Kershaw Chang [:kershaw]]

I think you probably need to set content-type to applicaiton/wasm for the newly created Response object.
For example, the code would be like:

  var headerList = {
    "Content-Type": "application/wasm",
  };
  var headers = new Headers(headerList);
    var res = new Response(new ReadableStream({
        ...
    }, {
        "status" : response.status,
        "statusText" : response.statusText,
         "headers" : header
   }));

Could you check if this is helpful?

Comment 3

[olliedawes]

Hi Kershaw, thanks for the suggestion.

I've just tested and passing the headers directly to the Response constructor works in both Firefox and Chrome.

var headerList = {};
for (var pair of response.headers.entries()) {
    headerList[pair[0]] = pair[1];
}
var headers = new Headers(headerList);

var res = new Response(new ReadableStream({
    ...
}, {
    "status" : response.status,
    "statusText" : response.statusText,
    "headers" : headers 
}));

Note that before I was using the following block of code to set the headers after creating the response:

for (var pair of response.headers.entries()) {
    res.headers.set(pair[0], pair[1]);
}

The original response already contains a content-type header with the value application/wasm and the above fixed it on Chrome but not Firefox. Looking at the MDN docs for Response (https://developer.mozilla.org/en-US/docs/Web/API/Response) I see that the headers property is marked as Read only. If response headers are supposed to be immutable then I guess the problem actually lies with Chrome. If this is expected behaviour then feel free to close.

Comment 4

[Kershaw Chang [:kershaw]]

According to the spec, headers should be read only.

I'm glad that this works for you. :)

Comment 5

[Tom S [:evilpie]]

I don't think this a correct interpretation of the spec. Yes the property is marked readonly so doing something like res.header = headers is invalid. But the returned object is not marked [Frozen] so modifying it is still supposed to be possible.

Comment 6

[Kershaw Chang [:kershaw]]

I think setting headers on the Response object should be working on Firefox.
I've created a simple test case (https://jsfiddle.net/qr02vpo5/) to verify it.

Could you try again to see if setting headers works for you?
Thanks.

Comment 7

[Tom S [:evilpie]]

So I have a theory on what happens here:

• We read the MIME type (FetchBody::mMimeType) in FetchUtil::StreamResponseToJS from Respsonse
• We only set the MIME type in the Response constructor: https://searchfox.org/mozilla-central/source/dom/fetch/Response.cpp#78

So any modification to the headers afterwards are invisible. We probably need to lookup the "Content-Type" in the headers object every time FetchBody::MimeType is called.

Comment 8

[olliedawes]

@kershaw Your example does appear to work correctly for me (the 'Content-Type' header is alerted correctly for both responses)

I retried my original code since it was doing pretty much the same thing as your example but this time printed out the contents of the new response's headers before passing to WebAssembly.compileStreaming. All the headers print out correctly so they are in fact added to the response but I still get the TypeError: Response has unsupported MIME type (In firefox only).

                var res = new Response(new ReadableStream({
                        ....
                    }), {
                        "status" : response.status,
                        "statusText" : response.statusText,
                        //"headers" : headers
                    });

                for (var pair of response.headers.entries()) {
                    res.headers.set(pair[0], pair[1]);
                }

                for (var pair of res.headers.entries()) {
                   console.log(pair[0], pair[1]);
                }

                return WebAssembly.compileStreaming(res).catch(function(error) {
                    console.log(error)
                    // compileStreaming may/will fail if the server does not set the correct
                    // mime type (application/wasm) for the wasm file. Fall back to fetch,
                    // then compile in this case.
                    return fetchThenCompileWasm(res);
                });

This behaviour is in line with the Tom's theory above, if MIME type is only read once then this is beginning to make sense.

Comment 9

[Tom S [:evilpie]]

I will give this a shot. I already looked at the code anyway.

Comment 10

[Kershaw Chang [:kershaw]]

I think maybe we need to call SetMimeType() after setting headers.

Anyway, thanks for taking this bug, Tom.

Comment 11

[Anne (:annevk)]

https://webassembly.github.io/spec/web-api/#compile-a-potential-webassembly-response looks directly at the headers of the Response object and not the internal MIME type field. So Chrome is correct. (There's a test for Chrome's behavior at https://github.com/web-platform-tests/wpt/pull/27100 though that also contains a test for the proposed changes mentioned below.)

I filed https://github.com/whatwg/fetch/issues/1135 to see if there is interest in making the overall situation more consistent and logical. Thanks for bringing this to my attention Tom!

Comment 12

[Tom S [:evilpie]]

Attachment: Bug 1684634 - Use InternalHeaders for MimeType instead of caching in the Response/Request constructor.

Comment 13

[Tom S [:evilpie]]

This patch implements the new behavior proposed for fetch. We need a different more targeted fix if we only want to do this for wasm.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/da3c25d91d76
Use InternalHeaders for MimeType instead of caching in the Response/Request constructor. r=baku

Comment 15

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/da3c25d91d76