1423877 - Update MIME type parser

Status: NEW

(Core :: Networking, enhancement, P3)

Description

[Anne (:annevk)]

I've filed a couple other bugs on more specific changes we might want to make sooner, but overall I think we want to change our parser more significantly to align with

  https://mimesniff.spec.whatwg.org/#parsing-a-mime-type

We should also make it reusable so it can be used outside of Necko for various JavaScript APIs.

https://github.com/w3c/web-platform-tests/pull/7764 has a lot of tests, primarily written against APIs, but also against navigation, which is affected directly by the Necko implementation.

Comment 1

[Henri Sivonen (:hsivonen)]

Currently https://github.com/w3c/web-platform-tests/blob/master/mimesniff/mime-types/resources/generated-mime-types.json#L967 passes for the wrong reason. It passes, because currently it uses UTF-8 to UTF-16 XPCOM string conversion, which produces the empty string if the input is not valid UTF-8. Instead, it should use Latin1 to UTF-16 XPCOM string conversion and then discover that the string contains code points that are not valid HTTP token code points.

Comment 2

[Henri Sivonen (:hsivonen)]

It looks like a new MIME parser is not happening in time to unblock bug 1402247. What's the recommended action that I should take about comment 1 here to unblock bug 1402247? Should I just add an ad hoc check for valid HTTP token code points?

Comment 3

[Anne (:annevk)]

Even if that test passes it seems others, such as https://github.com/web-platform-tests/wpt/blob/master/mimesniff/mime-types/resources/generated-mime-types.json#L951 would fail. So it doesn't seem like a big problem for a single test to incorrectly pass, unless I'm missing something.

Comment 4

[Henri Sivonen (:hsivonen)]

The test now passes, but for the wrong reason. Are you suggesting that I change it to expected fail as part of bug 1402247?

Comment 5

[Anne (:annevk)]

I think that's probably okay, yes. I guess to be sure it'd be nice to know which test ends up failing since there's a couple of consumers of that JSON resource and it's not immediately clear to me which would fall victim to what is described in comment 1.

Comment 6

[Henri Sivonen (:hsivonen)]

See https://treeherder.mozilla.org/#/jobs?repo=try&revision=7127052a4fae881669e48f0dfc4cc01ea56ba25f&selectedJob=183755844 for a try run with the current patch for bug 1402247.

Comment 7

[Anne (:annevk)]

Thanks, I doubt that'll cause problems in practice, especially given it only affects the Request/Response constructor in combination with serializing that to a Blob.

Comment 8

[Henri Sivonen (:hsivonen)]

Bug 1454590 added a spec-compliant MIME parser. Now we need to use it from more places. (It operates on UTF-16, but maybe it could be turned into a template that would allow it to work on either UTF-16 or Latin1/identity.)

Comment 9

[Anne (:annevk)]

https://github.com/whatwg/mimesniff/pull/79 makes a minor tweak to the parser (and serializer). I guess now we have an implementation a new bug would be good to track that?