567365 - Cache-Control no-cache on https page disables history

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Alain Knaff]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.1.9) Gecko/20100330 Fedora/3.5.9-1.fc11 Firefox/3.5.9
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.1.9) Gecko/20100330 Fedora/3.5.9-1.fc11 Firefox/3.5.9

When visiting a https page with Cache-Control: no-cache, then history does not work: The page is reloaded when going back to it, and a POSTDATA dialog is popped up if such page was reached by a POST.

The problem only happens if the page is served by https, but not with http.

Note for the triager: This is not the usual case of bug #160144 (which is about Cache-Control: no-store, not no-cache), and comment #112 specifically asked to file a separate report for each case where firefox warns about re-submission.

URL: This can be seen "in the wild" in the admin interface of any mailman mailing list which is served over https.

Reproducible: Always

Steps to Reproduce:
1. Visit https://www.lll.lu/~aknaff/bug-reports/mozillaNoStore/
2. Click on the "no cache" button (this triggers a post to a page which returns Cache-Control: no-cache)
3. Click on continue.
4. Go back in history

Actual Results:  
A dialog pops up
"To display this page, Firefox must send information that will repeat any action (such as a search or order confirmation) that was performed earlier."
To add insult to injury, this dialog box is also modal!


Expected Results:  
The browser should just redisplay whatever it has in history. No-cache is about the cache, not the history, which are 2 different entities. In most other cases Firefox does correctly handle these 2 entities differently, as can be seen with the must-revalidate button:
1. You can go back in history to a Cache-control: max-age=0,must-revalidate page, and you see its state like it was when you first visited.
2. When revisiting the page (for example, by re-entering its URL, by following a link to it), a new page is fetched from the server (as expected)

Comment 1

[Boris Zbarsky [:bzbarsky]]

> which is about Cache-Control: no-store, not no-cache

Here's the relevant code in nsHttpChannel:

2101        else if (mLoadFlags & VALIDATE_NEVER) {
2102            LOG(("VALIDATE_NEVER set\n"));
2103            // if no-store or if no-cache and ssl, validate cached response (see
2104            // bug 112564 for an explanation of this logic)
2105            if (mCachedResponseHead->NoStore() ||
2106               (mCachedResponseHead->NoCache() && mConnectionInfo->UsingSSL())) {
2107                LOG(("Validating based on (no-store || (no-cache && ssl)) logic\n"));
2108                doValidation = PR_TRUE;
2109            }

So in fact, SSL+no-cache is just like no-store for purposes of history code.  And unfortunately, that behavior is needed to make existing bank sites secure (in the "someone who walks up to the computer after you click the logout link can't just go back through history to the site").  There have been existing bugs on this.

Comment 2

[Alain Knaff]

Thanks for posting the code, so now we know at least that it is very trivial to fix.

Can we at least make this configurable, for those who prefer a browser which is RFC 2616 section 13.13 compliant?

   User agents often have history mechanisms, such as "Back" buttons and
   history lists, which can be used to redisplay an entity retrieved
   earlier in a session.

   History mechanisms and caches are different. In particular history
   mechanisms SHOULD NOT try to show a semantically transparent view of
   the current state of a resource. Rather, a history mechanism is meant
   to show exactly what the user saw at the time when the resource was
   retrieved.

   By default, an expiration time does not apply to history mechanisms.
   If the entity is still in storage, a history mechanism SHOULD display
   it even if the entity has expired, unless the user has specifically
   configured the agent to refresh expired history documents.

This whole think is ridiculous, going to your cited bug 112564, most people think that the current behaviour is asinine, sorry for the French. If a bank blacklists firefox over this, just let them: there are enough other banks out there to chose from. And those who still want to keep such a bank know how to change their user agent.

Or even better: please post a list of such banks (are there still any around, after the 8+ years that this is open?), so that we can drive them out of business beforehand. The financial crisis helping, this should be easy nowadays.

Comment 3

[Boris Zbarsky [:bzbarsky]]

> Can we at least make this configurable

Sure, on a code level.  Whether that's a good idea is a policy decision, obviously; ccing Jason.

Comment 4

[WADA:World Anti-bad-Duping Agency]

FYI.
Following is document for bfcache. 
  https://developer.mozilla.org/en/Using_Firefox_1.5_caching
Sorry but I don't know spec was changed by newer Fx or not.

Comment 5

[Jesse Ruderman]

I think we should make this change.  Paranoid banks can and should use no-store; they'd be silly to block Firefox based on this change.

Quoting Alain Knaff from bug 160144 comment 160:

The argument "the page must be removed from history, because the web site 
asked for this" is weak in the case of invalidations which only happen on SSL.
Indeed, in that case nobody decided to make the page no-cache+SSL. Indeed, the
application programmer may have added no-cache due to other considerations
(such as frequently changing contents). The guy who installed it on a web site
may have installed it on an SSL site for other consideration (protection
against eavesdroppers on the wire) without knowing that the app itself also
sends no-cache. None of the two parties conciously decided to put no-cache and
SSL together. So, arguing that firefox's behaviour is just following somebody's
wishes not to have the page in history is ingenuous.

Comment 6

[Jason Duell]

I've got no objection in theory to allowing this to be configurable. 

Do we know if any other browsers behave as we do, or follow the RFC?

Comment 7

[Alain Knaff]

> I've got no objection in theory to allowing this to be configurable. 

Thanks.

> Do we know if any other browsers behave as we do, or follow the RFC?

See bug 160144 comment 160.

In summary:
- Konqueror follows the RFC
- Google Chrome is buggy in exactly the same way as Firefox (discards history on no-store or (no-cache + SSL))
- Internet Explorer 7 is buggy too, but in a slightly different way (does not discard history on no-store, but does on must-revalidate + max-age=0 + SSL). This is interesting, as actually history discarding on no-store would make some sense, whereas the other cases make much less sense.

In addition, I now performed the test with lynx, and it is also RFC compliant.

Comment 8

[Jason Duell]

Alain,

Would you mind checking Opera and/or Safari?  Thanks.

Comment 9

[Alain Knaff]

Opera (10.10) does not have the problem (tried: no-store, https+no-store, https+no-cache, ... both with GET and POST)

Unfortunately, I do not have access to a Mac right now, so I can't test on Safari. And extrapolating from Konqueror seems risky, as it is known that in many areas Safari has fixed bugs which are still present in Konqueror.

If somebody who has a Mac wants to test, here is the test URL:
http://www.lll.lu/~aknaff/bug-reports/mozillaNoStore/must-revalidate.cgi

Use http instead of https to test the https case.

Comment 10

[Shad Sterling]

Safari 4.0.5 (5531.22.7) on OS X 10.5.8

These were displayed without reload when returned to with the back button:
GET http://lll.lu/~aknaff/bug-reports/mozillaNoStore/none.cgi
GET http://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-cache.cgi
GET http://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-store.cgi
GET http://lll.lu/~aknaff/bug-reports/mozillaNoStore/must-revalidate.cgi
GET http://lll.lu/~aknaff/bug-reports/mozillaNoStore/missingLastModified.cgi
POST http://lll.lu/~aknaff/bug-reports/mozillaNoStore/none.cgi
POST http://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-cache.cgi
POST http://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-store.cgi
POST http://lll.lu/~aknaff/bug-reports/mozillaNoStore/must-revalidate.cgi
POST http://lll.lu/~aknaff/bug-reports/mozillaNoStore/missingLastModified.cgi
GET https://lll.lu/~aknaff/bug-reports/mozillaNoStore/none.cgi

These were reloaded without permission when returned to with the back button:
GET https://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-cache.cgi
GET https://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-store.cgi
GET https://lll.lu/~aknaff/bug-reports/mozillaNoStore/must-revalidate.cgi
GET https://lll.lu/~aknaff/bug-reports/mozillaNoStore/missingLastModified.cgi

These opened a window-modal confirmation dialog when returned to with the back button:
POST https://lll.lu/~aknaff/bug-reports/mozillaNoStore/none.cgi
POST https://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-cache.cgi
POST https://lll.lu/~aknaff/bug-reports/mozillaNoStore/no-store.cgi
POST https://lll.lu/~aknaff/bug-reports/mozillaNoStore/must-revalidate.cgi
POST https://lll.lu/~aknaff/bug-reports/mozillaNoStore/missingLastModified.cgi

It looks like Safari doesn't keep any https POST results, but keeps all https GET or http results, regardless of the headers.  That's even more bizarre.

Comment 11

[Kai de Leeuw]

(In reply to comment #9)
> Opera (10.10) does not have the problem (tried: no-store, https+no-store,
> https+no-cache, ... both with GET and POST)

In Opera, I found out today, you need to use must-revalidate to not keep history and refresh the page on using the back button. This only works in HTTPS however. It is explained in more detail here: http://my.opera.com/yngve/blog/2007/02/27/introducing-cache-contexts-or-why-the

So in my code (ASP.NET), when I tried to solve this problem to refresh the page on using back, I did like this:
this.Response.Cache.SetCacheability(HttpCacheability.NoCache); //ie
this.Response.Cache.SetNoStore(); //firefox
this.Response.Cache.SetRevalidation(HttpCacheRevalidation.AllCaches); //opera

Comment 12

[[e-mail]]

Weird. If I visit any of those test cases, then go offline, then go back, I always get the page from cache. In fact, even if I go back to get my custom error page, then go offline, then click reload, I still get the cached page.

Comment 13

[Alain Knaff]

For me (in 3.6.3), it still happens, unfortunately.

When I go to https://www.lll.lu/~aknaff/bug-reports/mozillaNoStore/, then click the "no-cache" button, then "continue", and then go back, the infamous POSTDATA dialog pops up.

With a patched version (see bug 160144), it no longer pops up.

Comment 14

[Boris Zbarsky [:bzbarsky]]

Alain, did you go offline like comment 12 said?

In any case, confirming per comment 6.  Jason, you want to just try shipping a fix in the betas and seeing what the reaction is?

Comment 15

[Alain Knaff]

I tried "going offline" (without the patch), and indeed that looks like an interesting workaround: even after the POSTDATA dialog box appears, I can click Cancel, then "Work offline", and then back again. And presto, the page is there. This is great news. Knowing this workaround, the bug has become much less annoying now (although it is still an eyesore...)

With the patch, I can go back to a no-store or no-cache+SSL page with or without the patch.

Comment 16

[Jason Duell]

> Jason, you want to just try shipping a fix in the betas and seeing what the reaction is?

Sure.

Comment 17

[Boris Zbarsky [:bzbarsky]]

OK.  So keep the no-store behavior, but stop using it for SSL no-cache?

Comment 18

[Shad Sterling]

Depends on bug 200208:  Clearly distinguish between history & cache

Comment 19

[Boris Zbarsky [:bzbarsky]]

No, it really doesn't.

Comment 20

[Alain Knaff]

Still broken in Firefox 4.0.1 .

Of course, in comment 15, I meant "With the patch, I can go back to a no-store or no-cache+SSL page with or without 'Work Offline' "

Comment 21

[Jesse Ruderman]

This bug frequently makes sites become slower when they adopt https. For example, developer.mozilla.org doesn't support bfcache right now because of this bug. I'm concerned that this bug may slow adoption of https.

Comment 22

[Eric Shepherd [:sheppy]]

This... explains some things.

Comment 23

[Boris Zbarsky [:bzbarsky]]

Actually, the bfcache and form state behavior is somewhat separate from this bug as filed.  This bug is about the network traffic aspect.  For bfcache and form state restoration, you want nsDocShell::ShouldDiscardLayoutState (which has basically the same logic).  And you probably want a separate bug (which should probably depend on this one; fixing that bug without fixing this one makes no sense).

Comment 24

[Alain Knaff]

Attachment: Fix for this issue (originally posted 2010-06-09)

I'm re-attaching this (originally posted on 2010-06-09 to bug 160144), just in case people might have lost track of it in the year since I've originally posted it.

According to my tests (see comment #13) this fixes the issue.

Any objections?

Comment 25

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

With the patch, how can a website get the same effect without using Javascript? How can we tell if a website is relying on the old behavior, or whether it wants the new behavior?

This patch definitely changes (breaks) the log out behavior of sites. I tried it out with Facebook and logging out of Facebook correctly prevented private information from being revealed with the "back" button because Facebook uses some Javascript to delete the entire history for the tab. (Oddly, it even deleted history for non-Facebook pages that were in the history ahead of facebook pages, which IMO we shouldn't allow. Is there a bug this?) But, when I tried it with noscript active, Facebook's log out behavior wasn't correct; pushing the back button allowed me to see information that requires me to be logged in.

A small issue with the patch: the patch should delete the comment immediately preceding this "else if" block, and it should remove the first (now redundant) log statement within the else if block.

Comment 26

[Alain Knaff]

Given the number of comments on the dozens of bug reports gravitating around this issue, I think it is safe to say that nobody really wants such anti-user behavior by websites. So, not only does Facebook break the back button on its own site, but it also manages to break it on any unrelated site visited before? And you actually want to keep it that way? Well, *I* don't.

If Facebook could read local files off my disk, it certainly would. That doesn't mean however that Firefox should allow this. And if a bug in Firefox' Javascript engine allowed this, this should be fixed ASAP too.

The browser should do the user's bidding, not the webmaster's. And with the current behavior it pleases neither the user, nor (most) webmasters.

Agreed on the cosmetic remarks about the patch: yes the comment and one of the log statements can be deleted, and possibly the patch might need some tweaking to apply to current versions of Firefox.

Comment 27

[Jesse Ruderman]

It's not "anti-user" to try to make "log out" do what users expect. We should probably provide an API for that. But that shouldn't block fixing this bug.

Comment 28

[Jesse Ruderman]

Filed bug 667257 for a targeted "log out" API.

Comment 29

[Jesse Ruderman]

Alain's patch looks like it belongs on bug 261312. I'd rather not drag that debate into this bug, because I think it will slow down fixing this bug.

Comment 30

[Boris Zbarsky [:bzbarsky]]

Jesse, what do you think this bug is about, exactly?  It looks like a duplicate of bug 261312 to me...

Comment 31

[Alain Knaff]

(In reply to comment 30)
>  It looks like a duplicate of bug 261312 to me...

I had to recheck twice too... It looks like bug 261312 is just about when the behavior is triggered by Cache-Control: no-store , whereas this bug (bug 567365) is about when the behavior is triggered is triggered by Cache-Control: no-cache + SSL.

My patch fixes both situations.

Comment 32

[Schmitz Christian]

post here, only to follow the thread. Thanks

Comment 33

[Alain Knaff]

(in reply to comment 32)

Thanks for your interest. As far as I know, if you vote for a bug, you also get automatic notifications of new comments. And the bug will rank higher in the stats if enough people vote for it!

Comment 34

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

note to self: browserID session management, etc should also be involved with us in the security discussion around this.

Comment 35

[Alain Knaff]

Still broken in 7.0.1

Comment 36

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

http://blogs.msdn.com/b/ieinternals/archive/2012/03/01/ie10-beta-consumer-preview-minor-changes-changelist.aspx:

  "Internet Explorer [10] now ignores no-cache on back/forward
   navigations, as other browsers do and RFC2616 allows.

    This also allows Conditional GET revalidation of no-cache resources.
    Use no-store to prevent resource reuse in forward/back navigations."

Comment 37

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

closing the sec blocking bug here as incomplete as no work has been done in over a year, but leaving the sec-review flag on this bug to indicate we still want to be involved if this moves forward. Please need-info when this bug is ready.

Comment 38

[Alhaitham Ibrahim]

Affected site

https://www.reddit.com/

Related discussion

https://www.reddit.com/r/firefox/comments/3csi9u/time_to_revisit_this_14_year_old_bug/

Comment 39

[Morgan]

(In reply to Alhaitham Ibrahim from comment #38)
This bug is driving me crazy. PLEASE fix it!

Comment 40

[Morgan]

This bug appears to be fixed in 40 beta (20150723165742)?

Comment 41

[Jesse Ruderman]

I don't think so, but Reddit did just fix their headers to not specify no-cache.

Comment 42

[Morgan]

(In reply to Jesse Ruderman from comment #41)
> I don't think so, but Reddit did just fix their headers to not specify
> no-cache.

Ah, my apologies for the mistake. My biggest issue regarding this bug was with reddit so note sure if I still feel this is a valid bug (for me). I can understand the why behind the functionality.

Comment 43

[Patrick McManus [:mcmanus]]

Chrome, as of today anyhow, will load the no-cache https page from bfcache when back is pressed. This is the STR from comment 0.

If we needed a tie breaker, this should suffice - we should loosen the no-cache semantics, but based on comment 36 (and others) we should definitely leave no-store in place.

jesse and bz are ni'd just in case they aren't cool with that.

Comment 44

[Boris Zbarsky [:bzbarsky]]

I can probably live with it, esp. if everyone else is doing it anyway.

Comment 45

[Patrick McManus [:mcmanus]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=20299e1955c6

Comment 46

[Patrick McManus [:mcmanus]]

Attachment: do not force validate on validate_never for no-cache/https

Comment 47

[Boris Zbarsky [:bzbarsky]]

Don't you need to fix the docshell bits too, if you want bfcache to work here?

Comment 48

[Patrick McManus [:mcmanus]]

(In reply to Boris Zbarsky [:bz] from comment #47)
> Don't you need to fix the docshell bits too, if you want bfcache to work
> here?

its true this doesn't do bfcache.. I guess we want that too.. (?) (this enables the http cache to work without sending a network request). I'll take a look at it. thx.

Comment 49

[Patrick McManus [:mcmanus]]

Attachment: allow bfcache for no-cache/https

Comment 50

[Patrick McManus [:mcmanus]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8e7566d3f614

Comment 51

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 8711865 [details] [diff] [review]
allow bfcache for no-cache/https

r=me on the docshell bits

Comment 52

[Jason Duell]

Comment on attachment 8711865 [details] [diff] [review]
allow bfcache for no-cache/https

Review of attachment 8711865 [details] [diff] [review]:
-----------------------------------------------------------------

Ship it!

Comment 53

[Patrick McManus [:mcmanus]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e73523d185a7a9b3a409f342daf1ba108f988b81
Bug 567365 - allow bfcache for no-cache/https r=jduell r=bz

Comment 54

[Patrick McManus [:mcmanus]]

Release Note Request (optional, but appreciated)
[Why is this notable]: webdev compatibility with chrome and ie
[Suggested wording]: Allow no-cache on back/forward navigations for https resources as other browsers do.
[Links (documentation, blog post, etc)]: none

Comment 55

[Pulsebot]

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/542310f3191a

Comment 56

[Carsten Book [:Tomcat]]

sorry had to back this out for test failures like https://treeherder.mozilla.org/logviewer.html#?job_id=20490375&repo=mozilla-inbound

Comment 57

[Patrick McManus [:mcmanus]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=67c8fbca1fa5

Comment 58

[Patrick McManus [:mcmanus]]

Attachment: allow bfcache for no-cache/https

Comment 59

[Patrick McManus [:mcmanus]]

Comment on attachment 8712237 [details] [diff] [review]
allow bfcache for no-cache/https

Review of attachment 8712237 [details] [diff] [review]:
-----------------------------------------------------------------

this update just removes some test coverage for the no-cache specific handling. afaict 112564 is all about no-cache even though it contains a comment about no-store (probly a copy and paste). 215405 contained tests for both and only the no-cache subtest was removed. there are no c++ changes in the interdiff.

Comment 60

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 8712237 [details] [diff] [review]
allow bfcache for no-cache/https

>--- a/docshell/test/chrome/bug112564_window.xul

Instead of removing this test, change its expectations to expect the https+no-cache thing to be be bfcached, please.

>+++ b/docshell/test/chrome/bug215405_window.xul

Similar here: please update the test expectations and comments instead of removing the test.

Comment 61

[Patrick McManus [:mcmanus]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5419ee54f221

Comment 62

[Patrick McManus [:mcmanus]]

Attachment: allow bfcache for no-cache/https

Comment 63

[Patrick McManus [:mcmanus]]

I figured we didn't need to have special cases tests to test what is now the default case.. but you're right -as long as they're already written just update them.

Comment 64

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 8712307 [details] [diff] [review]
allow bfcache for no-cache/https

Much better, thanks!  r=me

Comment 65

[Patrick McManus [:mcmanus]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5760541fd240b043b2f6361810e522d8652a56f2
Bug 567365 - allow bfcache for no-cache/https r=jduell r=bz

Comment 66

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/5760541fd240

Comment 68

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Added to Fx47 release notes.

Comment 69

[jjcervero]

Hi.  We have an https web, and after the last release 47, the Firefox navigator not recognize the instruction: <meta http-equiv="cache-control" content="no-cache" />.  Any help would be apreciated.

Thanks!

Comment 70

[Honza Bambas (:mayhemer)]

(In reply to jjcervero from comment #69)
> Hi.  We have an https web, and after the last release 47, the Firefox
> navigator not recognize the instruction: <meta http-equiv="cache-control"
> content="no-cache" />.  Any help would be apreciated.
> 
> Thanks!

Metas regarding caching in HTML content are not supported for a very very long time.  This is definitely not a 47 regression.  If you believe there is a Firefox bug, then please make sure it's not a problem on your site and potentially file a new bug best with a life test case and good description what's broken.  Thanks.