587065 - Restrict requests to StAMN

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Wil Clouser [:clouserw]]

We've got StAMO acting as a proxy now (awesome), but if you load the front page it works:

https://static.addons.mozilla.org/en-US/firefox/

We should add some restrictions on this.  I think we can 403 or 404 any requests that don't match:

.*(gif|jpg|png|css|js|jsi18n)/?(\?[A-Za-z0-9=-]).*?

Someone double check that regex please.  Also, StAMO doesn't need to handle anything other than GET.

I just checked a request to StAMO and I'm still sending 8 cookies (most of them that __utm stuff) for each request.  One of the benefits of the CDN was less cookie traffic - perhaps we should be using static.addons.mozilla.net.

Comment 1

[Jeff Balogh (:jbalogh)]

(In reply to comment #0)
> .*(gif|jpg|png|css|js|jsi18n)/?(\?[A-Za-z0-9=-]).*?

I don't think Apache looks at the query string.  If that's the case we could do

(gif|jpg|png|css|js|jsi18n)$

Comment 2

[Jeremy Orem [:oremj]]

Does any of (gif|jpg|png|css|js|jsi18n) need the app code to function? If it doesn't I can create another vhost that doesn't run wsgi/php.

Comment 3

[Wil Clouser [:clouserw]]

it needs app code

Comment 5

[Jeremy Orem [:oremj]]

Wanted to confirm the regex in comment 1 is really all we need. Can I do this whenever?

Comment 6

[Jeff Balogh (:jbalogh)]

If we use this to serve addon icons, the regex will block them since we don't put a file extension on those requests.

https://addons.mozilla.org/en-US/firefox/images/addon_icon/1191/?modified=1290785704

Comment 7

[Wil Clouser [:clouserw]]

(In reply to comment #6)
> If we use this to serve addon icons, the regex will block them since we don't
> put a file extension on those requests.
> 
> https://addons.mozilla.org/en-US/firefox/images/addon_icon/1191/?modified=1290785704

Andy is fixing those up in zamboni so they'll have .png in them.  I think the regex in comment 1 is fine.  I'd like to do it during a push, SAMO uses this site right now.  Can schedule for next tuesday's push if you want

Comment 8

[Wil Clouser [:clouserw]]

We're slightly behind our code schedule so we aren't ready for this.  Please hold off for now.

Comment 9

[Wil Clouser [:clouserw]]

Also, for the record, we'll want to allow _files too - I'll come up with a new regex soon

Comment 10

[Corey Shields [:cshields]]

Any ETA here?  Can we close and reopen whenever ready?

Comment 11

[Wil Clouser [:clouserw]]

I'll take it out of server ops if it'll help.  Will give it back soon.

Comment 12

[Wil Clouser [:clouserw]]

Back to you guys.  There are two regex's that validate what we should serve off the CDN, so I think we'll need RewriteCond.  The regex's:


^/_files/
(css|gif|jpg|js|jsi18n|png|woff)/?$


Can you enable this filter on addons-cdn.allizom.org so we can make sure it works correctly?  Thanks.

Comment 13

[Jeremy Orem [:oremj]]

Added this to PAMO:
    RewriteCond %{HTTP_HOST} ^addons-cdn.allizom.org$
    RewriteCond %{REQUEST_URI} !^/_files/.*$
    RewriteCond %{REQUEST_URI} !(css|gif|jpg|js|jsi18n|png|woff)/?$
    RewriteRule . - [F]

and this to NAMO:

    RewriteCond %{HTTP_HOST} ^addons-next-cdn.allizom.org$
    RewriteCond %{REQUEST_URI} !^/_files/.*$
    RewriteCond %{REQUEST_URI} !(css|gif|jpg|js|jsi18n|png|woff)/?$
    RewriteRule . - [F]

Comment 14

[Jeremy Orem [:oremj]]

Accidentally closed. I'm not sure if you want to reopen until we do this in production or just leave it closed and add to a push bug.

Comment 15

[Wil Clouser [:clouserw]]

Closed is fine.  I'll plan on pushing next thurs (9 days)