540859 - Block Hotlinking of Webfont Files

Status: VERIFIED FIXED

(www.mozilla.org :: General, defect)

Description

[Steven Garrity [:sgarrity]]

Our license for the Meta* fonts in /img/fonts that we're using on the firstrun/whatsnew pages for 3.6 requires "reasonable measures against hotlinking and direct access to font files."

Here's what they recommend:

Blocking Hotlinking of Webfont Files
When other websites are linking directly to your hosted webfont files (also called hotlink-ing), they are using your fonts, which they haven’t licensed properly, and they are using your bandwidth, adding to your site’s traffic costs. The FontFont Webfont EULA (End User Licence Agreement) requires that you take reasonable measures against hotlinking and direct access to font files.

A reasonable measure is utilizing referrer checking in your web server. Each request for your font files which doesn’t come from inside your own site will be blocked after you add these rules to your rewrite configuration (example for Apache HTTP Server):

  RewriteCond %{HTTP_REFERER} !^http://(www\.)?example\.com/.*$ [NC]
  RewriteRule \.(woff|eot)$ - [F,NC,L]

Replace ‘example\.com’ by your own domain name. The dot must be preceded by a backslash. The expression ‘(www\.)?’ means that both URLs with and without the ‘www’ prefix are valid for your site. Requests for all files ending in .woff or .eot with an external or empty referrer will now receive an HTTP error 403 (‘forbidden’).

Comment 1

[John Slater]

Sorry about the last minute nature of this bug, but this needs to get done before the launch tomorrow. Alex or Mike, can you guys help?

Thanks much-

Comment 2

[Alex Buchanan [:abuchanan]]

I think this will break any devel. site that doesn't end in *.mozilla.com

For the short term, since this is legal and 3.6 related, I think that's OK.

For the long term, I think hosting our fonts under something like fonts.mozilla.com, and using absolute URLs from our CSS code would work around the problem.

e.g.

@font-fact {
  src:  url(fonts.mozilla.com/foo.woff);
}

Comment 3

[Alex Buchanan [:abuchanan]]

(In reply to comment #2)
> For the long term, I think hosting our fonts under something like
> fonts.mozilla.com, and using absolute URLs from our CSS code would work around
> the problem.

I don't know why I thought this, but the hosting location has nothing to do with restricting access based on referrer.  d'oH!

We could...

* implement this at the vhost level, so it only applies to production
* implement it in .htaccess, and make devs turn it off manually if they need to work with fonts.

Mike, any other ideas?

Comment 4

[Alex Buchanan [:abuchanan]]

oremj, can we implement this on production only?  by doing it at the vhost level?

also, technically, someone could still hotlink to svn.mozilla.org, e.g.

http://svn.mozilla.org/projects/mozilla.com/trunk/img/fonts/MetaWebPro-Book.woff

so, we should protect that too.

Comment 5

[John Slater]

Sounds like a plan to me...thanks Alex.

Oremj, is that doable?

Comment 6

[Jeremy Orem [:oremj]]

Can't we just do something dynamic like:

  RewriteCond %{HOST} \.?([^\.]+\.[^\.]+)$
  RewriteCond %{HTTP_REFERER} !^http(s)://.*%1/.*$ [NC]
  RewriteRule \.(woff|eot)$ - [F,NC,L]

Comment 7

[Alex Buchanan [:abuchanan]]

(In reply to comment #6)
> Can't we just do something dynamic like:
> 
>   RewriteCond %{HOST} \.?([^\.]+\.[^\.]+)$
>   RewriteCond %{HTTP_REFERER} !^http(s)://.*%1/.*$ [NC]
>   RewriteRule \.(woff|eot)$ - [F,NC,L]

Ah, yes, I meant to ask if that is possible.  Thank jebus for your Apache skillz. :)  I can test that.

What about svn.m.o?  Is it possible to apply a similar rule there?  

I don't get the hotlinking rules there, because anyone could download this from our SVN repo. and host it themselves if they wanted it that badly.  So, preventing hotlinking doesn't prevent much.

Comment 8

[Jeremy Orem [:oremj]]

(In reply to comment #7)
> I don't get the hotlinking rules there, because anyone could download this from
> our SVN repo. and host it themselves if they wanted it that badly.  So,
> preventing hotlinking doesn't prevent much.

I was thinking the same thing. Do we really need to block hotlinking off svn.m.o?

Comment 9

[Wil Clouser [:clouserw]]

Preventing hotlinking will prevent me and anyone else with referrers off from seeing the site with the font.

That said, having the fonts in SVN seems to conflict with "direct access to font files" with or without the referrer check.

Comment 10

[John Slater]

Hey guys. This is a new area for us, and I think it's probably ok if our first effort at it isn't 100% foolproof. The key is to make sure we're taking "reasonable measures", and then keep refining it over time so the system gets better and better.

But, for launch tomorrow let's find the simplest solution to block hotlinking as best as we can (and in a way that doesn't block people from viewing the font, of course).

If oremj's suggestion in comment #6 meets that criteria, I say let's do it.

Comment 11

[Alex Buchanan [:abuchanan]]

(In reply to comment #6)
> Can't we just do something dynamic like:
> 
>   RewriteCond %{HOST} \.?([^\.]+\.[^\.]+)$
>   RewriteCond %{HTTP_REFERER} !^http(s)://.*%1/.*$ [NC]
>   RewriteRule \.(woff|eot)$ - [F,NC,L]

This didn't work for me.  Maybe it needs a tweak?

I tried the recommended code from comment 0, and it worked.

I'll check back early in the morning on this.

Comment 12

[John Slater]

Any updates here?

Comment 13

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Do we sent Origin: headers on font requests?  Though I'm not sure that I care a _ton_ about people who have referrers disabled in this case, tbqh.

Comment 14

[Alex Buchanan [:abuchanan]]

Checked in the recommended  code from comment 0 to trunk in r60853.

Comment 15

[Alex Buchanan [:abuchanan]]

seems to work fine on trunk,

https://www-trunk.stage.mozilla.com/en-US/img/fonts/MetaWebPro-Bold.woff
^ is a 404

https://www-trunk.stage.mozilla.com/en-US/firefox/3.6/whatsnew/
^ uses that font, and still looks the same

Comment 16

[Alex Buchanan [:abuchanan]]

r60861 merges with stage.


Oremj, could you move this to production when you can?  thanks

Comment 17

[John Slater]

We'll be able to publish this fix today, right? We're currently violating the terms of our licensing agreement...need to correct that, asap.

Thanks!

Comment 18

[Jeremy Orem [:oremj]]

I've already pushed .htaccess out. Can someone test?

Comment 19

[Alex Buchanan [:abuchanan]]

(In reply to comment #18)
> I've already pushed .htaccess out. Can someone test?

works for me

Comment 20

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED on authstage:

https://www.authstage.mozilla.com/en-US/img/fonts/MetaWebPro-Bold.woff is a 404, as Alex notes in comment 15, and https://www.authstage.mozilla.com/en-US/firefox/3.6/whatsnew/ still references it:

 @font-face { font-family: 'MetaWebPro-Book'; src: url('/img/fonts/MetaWebPro-Book.woff') format('woff'); }
        @font-face { font-family: 'MetaWebPro-Bold'; src: url('/img/fonts/MetaWebPro-Bold.woff') format('woff'); }

Comment 21

[John Slater]

thanks guys!

Comment 22

[Stephen Donner [:stephend] Not actively reading bugmail]

And also on production (since I mid-aired with Jeremy):

http://www.mozilla.com/en-US/firefox/3.6/whatsnew/ links to the font, but http://www.mozilla.com/en-US/img/fonts/MetaWebPro-Bold.woff is a 404 (as expected).

Comment 23

[John Slater]

Got the following note from our FontShop contact when I told him this had been fixed:

"Doesn’t seem to be working. I can still download the WOFF files looking into the source code and following the file’s URL: http://www.mozilla.com/img/fonts/MetaWebPro-Bold.woff."

Any ideas?

Comment 24

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Yeah, I think Stephen and Alex were testing the wrong URLs.  This isn't fixed.

Comment 25

[Steven Garrity [:sgarrity]]

I think the current rule will fail for HTTPS as well, though I don't think that's the issue here.

Comment 26

[Alex Buchanan [:abuchanan]]

ok, apologies for the false start.

r61039 checks in a new fix to trunk.  This should fix the original bug, and handle HTTPS


font file == 404
https://www-trunk.stage.mozilla.com/img/fonts/MetaWebPro-Bold.woff

whatsnew page, font is working
https://www-trunk.stage.mozilla.com/en-US/firefox/3.6/whatsnew/



fwiw, the new code is...

RewriteCond "%{HTTP_HOST}_%{HTTP_REFERER}" "!\.?([^\.]+\.[^\.]+?)_https?://.*\1/.*$"
RewriteRule \.(woff|eot)$ - [F,NC,L]




oremj mentioned this code can be added to allow empty referers...

RewriteCond %{HTTP_REFERER} !^$

...but I think that qualifies as "direct access" so I *didn't implement it*

Comment 27

[Alex Buchanan [:abuchanan]]

r61042 merges to stage.

IT will need to `svn up` production manually (we're not back to normal yet, from the launch)  I pinged oremj

Comment 28

[Alex Buchanan [:abuchanan]]

oremj updated production. 

http://www.mozilla.com/en-US/img/fonts/MetaWebPro-Bold.woff
^ is a 404 for me

(if you're getting a page of jibberish, it's coming from the web cache. Try adding ?foobar to the end of that URL)

Comment 29

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED; I used Firebug's "Net" panel this time, to avoid the snafu of comment 20 (made sure the font was loaded from the /img/ dir without the locale code, as expected); here's hoping!

Comment 30

[John Slater]

thanks dudes-

Comment 31

[John Slater]

Attachment: Web FontFont User Guide

I hate to say this, but our FontShop contact is saying that the WOFF Meta is still accessible. Quoting his email to me:

"You should recive an HTTP error 403 (‘forbidden’) typing  http://www.mozilla.com/img/fonts/MetaWebPro-Book.woff into your browser bar. Please see page 10 of our Web FontFont User Guide for more details. You can use an .htaccess file configuration or an Apache server configuration for instance."

Note: the user guide he refers to is attached. 

The technical details of this have admittedly gone way beyond me, but this is something we need to fix once and for all. Alex, Jeremy - what's the best next step here?

Comment 32

[Alex Buchanan [:abuchanan]]

Attachment: jibberish screenshot

I *do* receive a 403, and I'm thinking he saw the cached, jibberish page that I mentioned in comment 28

attaching screenshot of what I mean.

oremj, could you clear the cache of any .woff files?

Comment 33

[Steven Garrity [:sgarrity]]

A bit more info: I'm still able to load the font file with wget:

 wget http://www.mozilla.com/img/fonts/MetaWebPro-Bold.woff

but if I add a URL variable:

 wget http://www.mozilla.com/img/fonts/MetaWebPro-Bold.woff?asdf

then I get a 403 Forbidden, as intended. Must be something caching on the server side?

Comment 34

[Jeremy Orem [:oremj]]

If someone comes in with a referrer it will stick in the cache. I think you need to Vary on referrer.

Comment 35

[Alex Buchanan [:abuchanan]]

r61114  adds "Vary: Referer" header for .woff and .eof files

r61116 merges to tags/stage.

oremj will need to manually update production.

Comment 36

[Alex Buchanan [:abuchanan]]

(In reply to comment #35)
> r61114  adds "Vary: Referer" header for .woff and .eof files
> 
> r61116 merges to tags/stage.
> 
> oremj will need to manually update production.

should also clear the cache of any font files, of course

Comment 37

[Jeremy Orem [:oremj]]

Alright, this should be fixed now.

Comment 38

[Alex Buchanan [:abuchanan]]

(In reply to comment #37)
> Alright, this should be fixed now.

Looks fixed to me.  I get the 404 page consistently, without needing to bypass the cache.  And the firstrun page still has the font.

Comment 39

[Stephen Donner [:stephend] Not actively reading bugmail]

John: mind confirming with our FontShop contact that this is indeed fixed?  Thanks.

Comment 40

[John Slater]

(In reply to comment #39)
> John: mind confirming with our FontShop contact that this is indeed fixed? 
> Thanks.

Definitely. I pinged him yesterday about this and haven't heard back yet (which is probably a good sign, given that he got back to me right away when the earlier fixes were incorrect). But, I'll wait for a definite confirmation from him before I verify anything.

Comment 41

[John Slater]

Well, good news/bad news. The word is that hotlinking is blocked, but direct access to the font is still partially available. 

I ran this by Blizzard, and he recommends setting up caching so it has to check every time (i.e. don't add an Expires: header). Alex, what do you think? Doable? Not doable?

For reference, here's the full note from Ivo at FontShop:

"We finally discovered the problem. Hotlinking is blocked now, but direct access only partially.

Firefox seems to employ a quite aggressive caching strategy.

These is an example HTTP request and response. As you can see, we turned off all cache-related headers like Etag, Expires and Cache-Control.

----------------------------------------------------------
GET /test.woff HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive

HTTP/1.x 200 OK
Date: Wed, 27 Jan 2010 10:49:54 GMT
Server: lighttpd/1.4.19
Access-Control-Allow-Origin: *
Content-Type: application/x-woff
Accept-Ranges: bytes
Last-Modified: Mon, 18 Jan 2010 18:50:16 GMT
Content-Length: 118212
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
----------------------------------------------------------

When the same file is requested again (by entering the URL of the woff file into the address field), Firefox sends no request at all to the server, but serves the file apparently from its cache.

Due to this behaviour the referrer checking mechanism has no effect, because the server never sees a second request which it could block. Effectively this means, if you have viewed a page with a css-linked font first, and find out the font's URL, you can always download it.

This means that all customers who will license our WOFF fonts can’t really ensure at all that others cannot access the Font Software. I do see the need for this behavior of your browser, but with this information do you have any idea how you (as well as our other customers of course) could prevent direct access to the WOFF anyway? It would be of great help to us."

Comment 42

[Alex Buchanan [:abuchanan]]

At first glance I don't like the idea of not caching the biggest files on our site:  each .woff is > 70K

I'm still wrapping my mind around how this constitutes direct access, or how we could do it without turning off caching.

Comment 43

[Jeremy Orem [:oremj]]

This seems kind of silly, someone can just craft a request with 'Referrer: http://www.mozilla.com/' and download the font directly.

curl -v -H 'Referer: http://www.mozilla.com/en-US/firefox/3.6/firstrun/'  'http://www.mozilla.com/img/fonts/MetaWebPro-Book.woff'

Comment 44

[Jeremy Orem [:oremj]]

I also don't like the idea of our users noting being able to locally cache a 70K font file. Maybe we should think about switching to a free or open source font.

Comment 45

[Wil Clouser [:clouserw]]

(In reply to comment #44)
> I also don't like the idea of our users noting being able to locally cache a
> 70K font file. Maybe we should think about switching to a free or open source
> font.

Hire this man!

Comment 46

[Pascal Chevrel:pascalc (PTO until April 26)]

+1 !! With an open font, we could have web font consistency between mozilla.com, affiliates, MCS portals, and all our community portals in Europe!

Comment 47

[Reed Loden [:reed]]

I agree. There are always going to be ways to download the font if somebody really wants to, so continuing to do silly things in attempting to prevent such from happening is pointless. If we can't cache the font file, we just shouldn't use it at all.

Comment 48

[John Slater]

Per the above comments and our conversation on the #webdev IRC channel, I'll let FontShop know that we consider the situation in comment 41 an edge case.

Will verify this bug once I've resolved things with them.

Comment 49

[Stephen Donner [:stephend] Not actively reading bugmail]

I'm assuming this is a-OK now, since we haven't yet heard back from them?

Comment 50

[John Slater]

(In reply to comment #49)
> I'm assuming this is a-OK now, since we haven't yet heard back from them?

Yes, my apologies for that. I confirmed with FontShop via email that the current setup is ok. Thanks to everyone for all the work on this.

Verifying-