682492 - Firefox sends If-Modified-Since with requests when the currently cached entity is not acceptable

Status: RESOLVED DUPLICATE of bug 1121447

(Core :: Networking: HTTP, defect)

Description

[theimp]

If Firefox has a request that it has already cached a response for, it should not send If-Not-Modified headers on a new request, if the response already in its cache cannot satisfy the request.

Obviously, the server will not know that Firefox wants something else, and will correctly respond with 304 Not Modified, and then Firefox will *incorrectly* use what was in its cache in the first place. Even, in fact, if the cached response was an error with no message-body component.


Summary:
* The If-Not-Modified header should not be used when the entity in the cache would not be suitable according to the other parameters of the request.
* The cache must not satisfy requests that the entity body is invalid for, even if it gets a 304 Not Modified response from the server when "revalidating".


Example:

Firefox requests a document in a certain format (For example, consider the early comments in Bug 676740; requesting a document that Firefox, correctly or not, expects to be a stylesheet, with Accept: text/css,*/*;q=0.1)

The server, detecting that this is silly, sends back a 406 Not Acceptable response; but with a Vary: Accept header, so that Firefox will come back when it wants the actual document at that URL, in a sensible format.

When Firefox wants the document again, it determines that what it has cached is not acceptable. It now wants:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

But the current entity has no Content-Type (since it has no body), and has a Vary: Content-Type header.

So, the request hits the network.

So far, this is all correct. But, when the request hits the network, it includes an If-Not-Modified header.

This is wrong, because Firefox doesn't simply want to know if the document has changed; it wants a document with the new parameters it has sent, which differ from what's in the cache: the currently cached entity cannot be used to satisfy the request.

The server cannot know that Firefox even has a cached copy, so the server cannot do anything about this condition. So…

Firefox receives a 304 Not Modified response (which isn't going to have a Content-Type header or message body either), and so then displays the cached response (406 Not Acceptable and with no message body), which it must not do because caches must not serve entities that are not valid for a request.

Comment 1

[theimp]

Sorry, I keep saying If-Not-Modified when I mean If-Modified-Since.

Comment 2

[theimp]

Further testing reveals that whether the existing view is replaced on refresh is somewhat dependent on the file extension!

Consider these test-cases:
http://203.59.75.251/Bug682492
http://203.59.75.251/Bug682492.cgi
http://203.59.75.251/Bug682492.php
http://203.59.75.251/Bug682492.aspx
http://203.59.75.251/Bug682492.html
http://203.59.75.251/Bug682492.html.php

In terms of response behavior, they differ only by extension; yet, for the files ending in .php only, refreshing the page (however done) does not cause the current view to be replaced with the cached entity (even though it gets an identical 304 Not Modified response)!

Is this a deliberate Firefox compatibility quirk?

The real bug, though, is the If-Modified-Since headers.

Comment 3

[theimp]

Attachment: Example PHP script that will generate the behavior of Bug 682492

Comment 4

[Boris Zbarsky [:bzbarsky]]

> The server cannot know that Firefox even has a cached copy,

Well, it can: it's getting an If-Modified-Since header.

But yeah, sending If-Modified-Since in combination with rerequesting due to Vary is a bit odd.  Patrick, what do you think?

Comment 5

[Patrick McManus [:mcmanus]]

(In reply to Boris Zbarsky (:bz) from comment #4)
> > The server cannot know that Firefox even has a cached copy,
> 
> Well, it can: it's getting an If-Modified-Since header.
> 
> But yeah, sending If-Modified-Since in combination with rerequesting due to
> Vary is a bit odd.  Patrick, what do you think?

I think a conditional request that was etag based would be ok, but doing variant selection with timestamps is kinda crazy :) (multiple variants will each have their own timelines).

So in the vary case we shouldn't send the if-modified-since.

In general to be conservative be probably shouldn't seek to conditionally revalidate 4xx level caches entries at all.

Comment 6

[Honza Bambas (:mayhemer)]

(In reply to theimp from comment #2)
> refreshing the page (however done) does not cause
> the current view to be replaced with the cached entity (even though it gets
> an identical 304 Not Modified response)!

We immediately expire 406 response, it means we set expiration time to 1979-01-01 00:00.  There for we never send If-Modified-Since header on next request and reload the page unconditionally.

It is non-sense to differentiate variations by date (last-modified).  However, not sure the idea is crazy or not, just what came to my mind right now: we might enhance the cache key of the varying request header values and have multiple entries for each URI that way.

Comment 7

[theimp]

> There for we never send If-Modified-Since header on next request and reload the page unconditionally.

Not so. Please see the examples.

Load:

http://203.59.75.251/Bug682492

Note that the page is retrieved twice due to Bug 676740. On the second response, a 406 Not Acceptable response overwrites the current page in the cache.

Then, refresh the page. A conditional request, with an If-Modified-Since header, is sent.

And if a 304 Not Modified response is received, then exactly what happens next depends on a factor, that seems so far to be the file extension.

> We immediately expire 406 response, it means we set expiration time to 1979-01-01 00:00.

Yes, I see that.

Nevertheless: the message body on the 406 Not Acceptable response is displayed conditionally on some factor, so far only the extension has been identified.

Load the following URLs in separate tabs:

http://203.59.75.251/Bug682492.html
http://203.59.75.251/Bug682492.html.php

Refresh each one. In any order.

The one that ends with ".php" has the current view (the current DOM) preserved; any other extension shows what's in the cache (which in this case will be the message body of the 406 Not Acceptable response).

Even though, preserving the current view, or updating the current view with an expired (4xx) response, is equally wrong:

RFC 2616: If a 304 response indicates an entity not currently cached, then the cache MUST disregard the response and repeat the request without the conditional.

Comment 8

[Masatoshi Kimura [:emk]]

> We immediately expire 406 response, it means we set expiration time to 1979-01-01 00:00.  There for we never send If-Modified-Since header on next request and reload the page unconditionally.
If the cache have an expired cache entry, client may send If-Modified-Since to revalidate the cache entry. If we will never use the 406 response, it should not be cached at all.