Last Comment Bug 465007 - Harmonize content sniffing in HTML 5 and Firefox
: Harmonize content sniffing in HTML 5 and Firefox
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: Networking (show other bugs)
: Trunk
: All All
: -- normal with 1 vote (vote)
: ---
Assigned To: Adam Barth
:
Mentors:
http://webblaze.cs.berkeley.edu/2009/...
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-11-14 16:39 PST by Adam Barth
Modified: 2009-04-25 13:38 PDT (History)
9 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Conservative patch (3.96 KB, patch)
2008-11-14 18:16 PST, Adam Barth
joe: review+
bzbarsky: superreview+
Details | Diff | Splinter Review

Description Adam Barth 2008-11-14 16:39:49 PST
My colleagues and I have put together a web site that makes it easy to compare the mime signatures used by Internet Explorer 7, Firefox 3, Safari 3.1, Google Chrome, and the HTML 5 specification:

http://webblaze.cs.berkeley.edu/2009/content-sniffing/

The portion of the HTML 5 that discusses content sniffing is available at:

http://www.whatwg.org/specs/web-apps/current-work/#content-type-sniffing

I'm hoping we can use this information to converge the content sniffing algorithms used by different browsers and then update the HTML 5 spec to reflect the consensus.

Although some changes to the content sniffing algorithm are likely to be contentious, I'm hoping some of the changes will be clearer:

== image/png ==

Firefox 3
DATA[0:3] == 0x89504e47

Internet Explorer 7
DATA[0:7] == 0x89504e470d0a1a0a

Safari 3.1
No signature

Google Chrome
DATA[0:7] == 0x89504e470d0a1a0a

HTML 5
DATA[0:7] == 0x89504e470d0a1a0a

Recommendation: Match HTML 5 spec

== application/postscript ==

Firefox 3
(strncmp(DATA,"%!PS-Adobe-",11) == 0) || 
(strncmp(DATA,"%! PS-Adobe-",12) == 0)

Internet Explorer 7
strncmp(DATA,"%!",2) == 0

Safari 3.1
No signature

Google Chrome
strncmp(DATA,"%!PS-Adobe-",11) == 0

HTML 5
strncmp(DATA,"%!PS-Adobe-",11) == 0

Recommendation: Match HTML 5 spec

== image/gif ==

Firefox 3
strncasecmp(DATA,"GIF8",4) == 0

Internet Explorer 7
(strncasecmp(DATA,"GIF87",5) == 0) || 
(strncasecmp(DATA,"GIF89",5) == 0)

Safari 3.1
No signature

Google Chrome
(strncmp(DATA,"GIF87a",6) == 0) || 
(strncmp(DATA,"GIF89a",6) == 0)

HTML 5
(strncmp(DATA,"GIF87a",6) == 0) || 
(strncmp(DATA,"GIF89a",6) == 0)

Recommendation: Match HTML 5 spec

== image/x-icon ==

Firefox 3
(DATA[0:3] == 0x00000100) || 
(DATA[0:3] == 0x00000200)

HTML 5
(DATA[0:3] == 0x00000100)
(although it uses the mime type image/vnd.microsoft.icon)

Recommendation: Eliminate this signature

We collected data from opt-in Google Chrome users about how often various signatures triggered, and the Firefox signature for image/x-icon almost never triggers.  For more details about this data set, see:

http://webblaze.cs.berkeley.edu/2009/content-sniffing/metrics.html
Comment 1 Adam Barth 2008-11-14 18:16:36 PST
Created attachment 348297 [details] [diff] [review]
Conservative patch

This patch makes the PostScript, GIF, and PNG changes.  I didn't touch x-icon or x-xbitmap because removing those signatures is not zero-risk.  I also adjusted the size of the content sniffing buffer to match the HTML5 spec.  This is should almost never make a difference in a non-attack scenario.

BTW, looking at the GIF decoder, I don't think it could decode a GIF that doesn't match this signature anyway.  Not sure who the right reviewers are, but these folks have made changes to / reviewed changes to this code before.
Comment 2 Adam Barth 2008-11-14 18:20:02 PST
Comment on attachment 348297 [details] [diff] [review]
Conservative patch

Oops.  Actually nominate reviewers this time.
Comment 3 Boris Zbarsky [:bz] 2008-11-17 18:08:31 PST
Not sure whether Stuart or Joe is a better reviewer for the imagelib stuff here.
Comment 4 Boris Zbarsky [:bz] 2008-11-17 18:11:12 PST
Comment on attachment 348297 [details] [diff] [review]
Conservative patch

The unknown decoder changes look ok, though watch out for local file regressions from the s/1024/512/ thing.
Comment 5 Joe Drew (not getting mail) 2009-03-26 15:21:25 PDT
Comment on attachment 348297 [details] [diff] [review]
Conservative patch

This patch is likely bitrotted due to my inaction, but I'd like to get it in to mozilla-central at least.
Comment 6 Robert O'Callahan (:roc) (email my personal email if necessary) 2009-04-05 22:11:33 PDT
http://hg.mozilla.org/mozilla-central/rev/e6812f79d2ba

We should really have tests for content sniffing, I guess.

Thanks Adam!

Note You need to log in before you can comment on or make changes to this bug.