982690 - Assets from the CDN aren't being cached when loaded from the Marketplace

Status: RESOLVED FIXED

(Marketplace Graveyard :: Code Quality, defect, P1)

Description

[Wil Clouser [:clouserw]]

When loading the front page of the Marketplace our assets from the CDN all come in as 200 OK, but when loading directly (eg. https://marketplace.cdn.mozilla.net/img/uploads/addon_icons/464/464384-64.png?modified=1384661293 ) they come in as 304 Not Modified, which is what we want.

This bug is to figure out what strangeness we're doing that is preventing the CDN caching.

Comment 1

[Mathieu Pillard [:mat]]

How are you testing this ? It's possible you are seeing 200 OK but they are in fact served from your browser cache.

Comment 2

[Mathieu Pillard [:mat]]

I took a quick look and it seems everything is working fine, the images and other assets are cached by the browser for a long time (we send far-future expires/cache-control headers), and are served from cache (meaning we don't need to do a 304 at all, unless reloading the specific resource)

Firefox Developer Tools, even in nightly, show a request with a 200 OK response, but Chrome & Firebug doesn't, so I suspect that's the issue here. I'll make more tests with a network sniffer to be certain.

Comment 3

[Mathieu Pillard [:mat]]

Oops, submitted too soon : I meant to say that Chrome and Firebug specifically show the responses as coming from the cache (Chrome says "from cache", Firebug uses a light grey color and adds a "Cache" tab on the request properties)

Comment 4

[Wil Clouser [:clouserw]]

I was using the Developer tools built in to Firefox.

Comment 5

[Wil Clouser [:clouserw]]

(In reply to Wil Clouser [:clouserw] from comment #4)
> I was using the Developer tools built in to Firefox.

When I use the dev tools on other sites it shows the status codes I'd expect (ie. 304) or doesn't show the assets loading over the network at all.  I'm looking in the Network tab of the dev tools.

Comment 6

[Mathieu Pillard [:mat]]

There is something weird going on:

My findings:
- In Chrome, no network request are made to our CDN once you have loaded the Marketplace once - everything is cached properly, and the network panel in their devtools correctly mention that the assets are coming from the cache. It also does 304s if you explicitly reload the page (it doesn't otherwise since its cached is still considered valid thanks the far future expires)

- In Firefox, some requests are definitely being made - I checked with a network sniffer - however it's still using the cache, which you can see when using Firebug instead of the developer tools. In addition, those network requests are very small, and completely different from the normal ones you make with an empty cache - you can see that in the "Timings" tab of any of those requests in the devtools, comparing with an empty/filled cache.

I'm not sure what's going on. I strongly suspect something related to appcache as I'm not seeing the same behaviour locally.

Comment 7

[Wil Clouser [:clouserw]]

Thanks for investigating.  Maybe it's doing HEAD requests?  Definitely interested to hear more about what's up.

Comment 8

[Christopher Van Wiemeersch [:cvan]]

https://github.com/mozilla/zamboni/commit/a04384e
https://github.com/mozilla/zamboni/commit/0b58c48
https://github.com/mozilla/zamboni/commit/59d5cb3

Fixed:
https://marketplace-dev.allizom.org/

I changed

NETWORK:
/manifest.appcache
*

to

NETWORK:
*

and Firefox started respecting the appcache. Chrome never cared FWIW.

Big thanks to Potch for the pair programming help!

Comment 9

[Christopher Van Wiemeersch [:cvan]]

Actually the problem is that the CDN is on a different origin: https://marketplace.cdn.mozilla.net/

And appcache cannot cache resources from different origins (even if the manifest is hosted on a different origin).

This explains the weirdness we were seeing:

> Over SSL, all resources in the manifest must respect the same-origin policy. The exception is Google Chrome, which doesn't follow the specification in this regard. Over SSL, Chrome will load resources from different origins so long as they are still served over SSL.

Comment 10

[Christopher Van Wiemeersch [:cvan]]

https://github.com/mozilla/zamboni/commit/4bc97f1

I've permanently disabled appcache because we can't cache cross-origin resources on the CDN. We're instead going to rely on Cache-Control headers for the assets and ETags for the API.

Comment 11

[Christopher Van Wiemeersch [:cvan]]

Disabled tests: https://github.com/mozilla/zamboni/commit/2e4f84e

Comment 12

[Iulian Timis]

Please add STR here or mark it with [qa-] if no QA is needed.