687758 - Any cross-origin requests fail if instantiated from offline cache even while online

Status: RESOLVED WONTFIX

(Core :: Networking: Cache, defect)

Description

[Rob Crowther]

Attachment: Test page

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0a1) Gecko/20110919 Firefox/9.0a1
Build ID: 20110919030912

Steps to reproduce:

I set up a test page with a manifest file which in turn references a font on the Google servers.  When initially loading the page everything is fine - the font is downloaded and the page is rendered with the @font-face font.


Actual results:

However, then I selected 'work offline' from the file menu and then reloaded.  Although the page was still displayed, the @font-face font disappears, everything is shown in default sans-serif.  Now even if the browser is set to online mode again, the custom @font-face is never used.


Expected results:

The @font-face font should be in the appcache in both cases and used to display the text.

Comment 1

[Rob Crowther]

Attachment: CSS for test page

Comment 2

[Rob Crowther]

Attachment: Manifest for test page

Comment 3

[Rob Crowther]

If the fonts are switched to be on the same server as the page everything works as expected, both when online and offline.  It's loading the fonts from another server which triggers the problem.

I'm not sure if the font not rendering when offline then never rendering again are facets of a single issue, or two separate issues.

Comment 4

[Boris Zbarsky [:bzbarsky]]

I assume the other server normally sends the right CORS headers and the appcache is just not preserving those?

Comment 5

[Rob Crowther]

Google is sending this header with the font (among others):

    Access-Control-Allow-Origin: *


The page renders with the custom font in place until you switch to offline mode, then it never uses the custom font again, whether online or offline.

Comment 6

[Boris Zbarsky [:bzbarsky]]

Rob, is there by chance a URI where the test page + manifest is set up so I can test?  Or do I really need to set it up on another server somewhere?

Comment 7

[Rob Crowther]

I was doing it from a directory on my machine with `python -m SimpleHTTPServer` :)  Do you want me to upload it somewhere publically accessible?

Comment 8

[Boris Zbarsky [:bzbarsky]]

If it's easy for you, then please yes.

Comment 9

[Rob Crowther]

Here:

http://www.boogdesign.com/examples/offline-fonts/index-remote.html

An example with font in the same directory is available if you remove the -remote.

Comment 10

[Boris Zbarsky [:bzbarsky]]

Rob, thanks.

The http channel for the font file is failing with NS_ERROR_DOCUMENT_NOT_CACHED.

Honza, why are we not caching that file (or failing to retrieve it from the cache)?

Comment 11

[Honza Bambas (:mayhemer)]

It is a cross-origin load.  Therefor we use LOAD_ANONYMOUS flag.  And it prefixes the cache key in http channel with "anon&".  Offline cache is not capable to find it because it keeps the key w/o the "anon&" prefix.

Comment 12

[Honza Bambas (:mayhemer)]

So, offline cache update should respect anonymous loads for cross origin requests, right?

Comment 13

[Boris Zbarsky [:bzbarsky]]

Presumably, yes...  Whatever it takes to make this reasonable use case work.  ;)

Comment 14

[Honza Bambas (:mayhemer)]

-> me

Comment 16

[Kohei Yoshino]

I just encountered this issue, first found Bug 755072, while creating an offline Web app to use the Bugzilla API :(
https://wiki.mozilla.org/Bugzilla:REST_API

Comment 17

[Kohei Yoshino]

Ping? I cannot implement some features in my Web app unless this bug is fixed. See http://bit.ly/bzdeck-faq

Comment 18

[Kohei Yoshino]

Is this bug difficult to fix? As noted above, I'd like to use cross-site XHR from my offline app.

Comment 19

[Honza Bambas (:mayhemer)]

(In reply to Kohei Yoshino [:kohei] from comment #18)
> Is this bug difficult to fix? As noted above, I'd like to use cross-site XHR
> from my offline app.

This needs some discussion first.  It has not been settled whether we want to cache the fonts anonymously during the 'offline cache update process' - i.e. store them in the 'anon' namespace, or if we want to just load any resource (by means of both 'non-anon' and 'anon') from 'non-anon' namespace when loading from the offline cache.

The letter is a simple one line change.  In the new cache backend we develop I've already introduced that change, but more just because I'm lazy ;)

So, the question here is whether we should serve non-anonymously loaded content stored in the offline cache as response to anonymous requests to the cache.  This potentially could expose a content that is accessible only when authenticated by the user to any arbitrary site.

Boris, Jonas, what do you think?  I personally tend to do caching of 3rd party resources anonymously, but I think that not all 3rd party resources are loaded anonymously by sites (e.g. images are not).  So, I lack a criteria to decide whether a resource should be cached using LOAD_ANONYMOUS flag or not during the offline cache update process.  We could, however, do also both loads, non-anon and anon and have the resource cached twice...

Comment 20

[Kohei Yoshino]

Hope this could be fixed in Firefox OS 1.2 based on Gecko 26 (the current Nightly).

Comment 21

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Honza Bambas (:mayhemer) from comment #19)
> (In reply to Kohei Yoshino [:kohei] from comment #18)
> This needs some discussion first.  It has not been settled whether we want
> to cache the fonts anonymously during the 'offline cache update process' -
> i.e. store them in the 'anon' namespace, or if we want to just load any
> resource (by means of both 'non-anon' and 'anon') from 'non-anon' namespace
> when loading from the offline cache.
> 
> The letter is a simple one line change.  In the new cache backend we develop
> I've already introduced that change, but more just because I'm lazy ;)
> 
> So, the question here is whether we should serve non-anonymously loaded
> content stored in the offline cache as response to anonymous requests to the
> cache.  This potentially could expose a content that is accessible only when
> authenticated by the user to any arbitrary site.

LOAD_ANONYMOUS means "When making this request, don't send cookies or HTTP auth credentials and don't use an SSL connection with client auth."

Even without LOAD_ANONYMOUS, often we will make requests without cookies or HTTP auth or SSL client auth. If we make a request without any such credentials (without LOAD_ANONYMOUS) we will store it in the cache normally. When we make a second request for that now-cached resource, that second request may have cookies or HTTP auth headers. Whether or not we can use the cached response depends on whether the normal rules for Vary plus whatever special behavior we have for cookies and HTTP auth. And, the same kind of argument applies applies in the converse case, where we cached a response that was for a request that had cookies and/or HTTP auth and/or SSL auth, and then we get a subsequent request without cookies and/or HTTP auth and/or SSL auth.

I cannot think of any reason why LOAD_ANONYMOUS should change the behavior of the cache. AFAICT, LOAD_ANONYMOUS should only change the behavior of which headers we put in the request (and which type of SSL connection we use), and then the cache should make its decision purely as a function of the request and the response plus any cache-specific flags (e.g. LOAD_FROM_CACHE).

> flag or not during the offline cache update process.  We could, however, do
> also both loads, non-anon and anon and have the resource cached twice...

I read the AppCache spec and it doesn't say anything about special cookie handling for AppCache. Consequently, the offline cache should handle cookies using the same logic as the regular HTTP cache.

Comment 22

[Boris Zbarsky [:bzbarsky]]

> whether we should serve non-anonymously loaded content stored in the offline cache as
> response to anonymous requests to the cache. 

Absolutely not.

This is an interesting question, though: what should the offline cache actually cache in this situation:  That is, how should the fetch for the offline cache be performed?  Anne?

> Consequently, the offline cache should handle cookies using the same logic as the
> regular HTTP cache.

The way the regular HTTP cache works is you make a request, you get a response, you cache it.

For the offline cache, the question is what that the request that populates the cache looks like.  The problem is that you don't know what the future fetch to get that data will look like; how can you then populate the cache properly?  This seems like a fundamental issue with appcache.  :(

Comment 23

[Honza Bambas (:mayhemer)]

(In reply to Boris Zbarsky [:bz] from comment #22)
> > whether we should serve non-anonymously loaded content stored in the offline cache as
> > response to anonymous requests to the cache. 
> 
> Absolutely not.

That's what I think too.

> 
> This is an interesting question, though: what should the offline cache
> actually cache in this situation:  That is, how should the fetch for the
> offline cache be performed?  Anne?
> 

We might want to cache both for 3rd party requests or let those be marked in the cache manifest (a new directive) as to be loaded anonymously.

Comment 24

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Honza Bambas (:mayhemer) from comment #23)
> (In reply to Boris Zbarsky [:bz] from comment #22)
> > > whether we should serve non-anonymously loaded content
> > > stored in the offline cache as response to anonymous
> > > requests to the cache. 
> > 
> > Absolutely not.
> 
> That's what I think too.

Please explain your thinking. I disagree and I already explained why in comment 21.

Comment 25

[Boris Zbarsky [:bzbarsky]]

LOAD_ANONYMOUS basically changes the value of Cookie, right?  And as I recall we always assume Vary:Cookie in our HTTP code...  So the effect of what you are proposing would end up being the same.

Comment 26

[Honza Bambas (:mayhemer)]

(In reply to Brian Smith (:briansmith, was :[e-mail]) from comment #24)
> Please explain your thinking. I disagree and I already explained why in
> comment 21.

I agree with you: we must *not* serve non-anonymously loaded content as a response to anonymous requests.

(In reply to Boris Zbarsky [:bz] from comment #25)
> LOAD_ANONYMOUS basically changes the value of Cookie, right?  

LOAD_ANONYMOUS removes (doesn't add) the Cookie header.

> And as I
> recall we always assume Vary:Cookie in our HTTP code...  

Hmm... I don't think we do this implicitly.  We check cookie header when the server announced Vary: Cookie, but otherwise not, I believe.

> So the effect of
> what you are proposing would end up being the same.

Not sure I follow based also on my comments above.

Comment 27

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

I don't know what the right thing to do here is :( The old appcache spec simply isn't expressive enough to support many types of cross-site requests.

Generally speaking, we should probably follow what other browsers do here. And then make sure that navigationcontroller/serviceworker handles this properly (which I believe is already the case)

Comment 28

[Boris Zbarsky [:bzbarsky]]

WebKit and Blink don't follow the "do a CORS load" spec for @font-face at all....

Should check what IE does.

Comment 29

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

They do support <img src="http://otherorigin.org/pic.jpg" crossorigin> though.

The problem as far as I can tell is that at the time when you're reading the manifest and start downloading resources, you have no idea how those resources are going to be used.

I.e. should use use CORS? Should you use CORS-with-cookies? Should you just do a normal load?

I don't think it's worth spending time on trying to fix this. Let's focus on the ServiceWorker (aka navigation controller) instead.

Comment 30

[Kohei Yoshino]

I just tested a cross-origin XHR with my BzDeck app but couldn't reproduce the issue. After switching back to online, the app could do XHR to the remote Bugzilla API normally. I don't know about the font issue but it might be partially fixed at some point. Wow!

Comment 31

[Kohei Yoshino]

sicking: thanks for dropping by my Innovation Fair booth!

I tested the app again as some users reported it didn't work. The fast was, the resource paths in the manifest.appcache file were just wrong. This issue still exists. Sorry for the false alarm in my previous comment.

Comment 32

[Kohei Yoshino]

I have tested BzDeck for a while. Look like any cross-origin requests, not only XHR but also images and static JavaScript files, have been failing once the app cache is stored. I'm going to disable offline support in the app until this bug is resolved.

Comment 33

[Honza Bambas (:mayhemer)]

This is more a spec bug.

Comment 34

[Kohei Yoshino]

Good news. I can no longer reproduce the issue on the latest Nightly. I guess this has been fixed at some point, since the HTTP cache backend has been overhauled. Bug 1102172 might be similar. Thanks anyway, now I can re-implement my BzDeck app's offline support.

Comment 35

[Honza Bambas (:mayhemer)]

We will soon land bug 1285476, which will forbid serving appcached data in response to anonymous (cors) requests again.  Hence this bug will reemerge, but since appcache is a no longer maintained technology for us, we are not going to fix it.

Please migrate to Service Workers in case of problems.