Closed Bug 1060082 Opened 10 years ago Closed 10 years ago

back button show "Document Expired" after POST, ONLY when cache is full

Categories

(Core :: Networking: Cache, defect)

Product:
Core
Component:
Networking: Cache
Version:
34 Branch
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1079789

People

(Reporter: fireattack, Assigned: mayhemer)

References

Details

(Keywords: regression)

Attachments

(1 file)

workaround
1.29 KB, patch
Details | Diff | Splinter Review
User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:34.0) Gecko/20100101 Firefox/34.0 (Beta/Release) Build ID: 20140828030205 Steps to reproduce: It's a private site (from my school) so you can't test it by yourself. But it reproducible under new profile and newest Nightly build. I will provide the details so hopefully someone could find a site like it: It's a SSL site (if matters), and after click one of some buttons, it will send POST request, and then bring you to another page. Actual results: If the cache is NOT FULL (either 350MB by default, or any numbers you set manually), you can "back" to the previous page (with buttons) by click the back button without any problem. but if the cache is FULL (I can easily reproduce it by setting cache max to 10MB, and open several heavy sites firstly), the back function doesn't work. It will show: Document Expired This document is no longer available. The requested document is not available in Nightly's cache. As a security precaution, Nightly does not automatically re-request sensitive documents. Click Try Again to re-request the document from the website. So I think the point is clear: It won't cache the history pages in this site ONLY WHEN your cache is full (or I should say, wont "Replace" old cache).
It might be related to Bug 200208, but my problem only happens when cache is full.
OK, I found a website where you can test: http://www.datdota.com/teams.php Why to reproduce: 1. Set your cache to very small like 10MB 2. fulfill your cache by opening lots of sites with pictures 3. Open http://www.datdota.com/teams.php 4. Change some options and click "submit" 5. When loaded, click back button. 6. "Problem loading page - Document Expired" will show. If not, try again from step 4.
(In reply to Benjamin Peng from comment #2) > 4. Change some options and click "submit" > 5. When loaded, click back button. > 6. "Problem loading page - Document Expired" will show. If not, try again > from step 4. Can't reproduce (5 MB cache). I have no Flash plug-in installed, and Google ads are blocked.
(In reply to [:Aleksej] from comment #3) That was with 2014-08-31-03-02-06-mozilla-central-firefox-34.0a1.ru.linux-x86_64. If it doesn't happen with very old builds, it would be useful to find when it started, e.g., using http://harthur.github.io/mozregression/
QA Whiteboard: [bugday-20140901]
Component: Untriaged → Document Navigation
Product: Firefox → Core
I just can't reproduce it either now. I will close this bug and see what's going on in the future. Thanks!
Status: UNCONFIRMED → RESOLVED
Closed: 10 years ago
Resolution: --- → INVALID
I can reproduce it again: you need to repeat go back and forward several times on the page I mentioned above to see it happen. I'm using 15MB cache, if it matter.
Status: RESOLVED → UNCONFIRMED
Resolution: INVALID → ---
Another guy with same problem. https://support.mozilla.org/zh-CN/questions/1019567 I can reproduce with his website, as well (but I need to POST twice then back).
Please some developers at least verify this bug?
Component: Document Navigation → Networking: Cache
Another user report: https://support.mozilla.org/zh-CN/questions/1019146 BTW, I believe I was vague on OP, so here is the correct way to reproduce it: 1 Set your cache to very small like 15MB 2 fulfill your disk cache by opening lots of sites with pictures 3 Open http://www.datdota.com/teams.php 4 Change some options and click "submit" 5 Do last step again (better do it more times to make sure the problem happening) 6 When loaded, click back button. 7 "Problem loading page - Document Expired" will show. If not, try again from step 4. 2-1 Reset your cache to a large number. 2-2 Redo 3~6 2-3 The problem in step 7 never happens
Assignee: nobody → honzab.moz
Benjamin, can you reproduce with Nightly [1] and a clean profile [2] ? [1] http://nightly.mozilla.org/ [2] http://kb.mozillazine.org/Creating_a_new_Firefox_profile_on_Windows
Flags: needinfo?(human.peng)
(In reply to Honza Bambas (:mayhemer) from comment #10) > Benjamin, can you reproduce with Nightly [1] and a clean profile [2] ? > > [1] http://nightly.mozilla.org/ > [2] http://kb.mozillazine.org/Creating_a_new_Firefox_profile_on_Windows Yes, I can reproduce it with Nightly, in a brand new profile.
Flags: needinfo?(human.peng)
This is not a cache2 regressions, I can reproduce with cache1 as well, just cache2 is more consistent in evicting POSTs so it's easier to reproduce. I'm not sure how to enforce keeping cache entries in the browsing history chain except using the forceValidFor for a reasonable amount of time and dropping it when windows go away (not sure how to respect session restore and undo close of a tab). The problem here is that we actually cannot ensure that VALIDATE_NEVER / LOAD_ONLY_FROM_CACHE when using bfcache will work (the entry will actually be found).
Assignee: honzab.moz → nobody
Status: UNCONFIRMED → NEW
Component: Networking: Cache → Document Navigation
Ever confirmed: true
There is no "bfcache" involved here. Just regular cache. Please don't confuse them. If bfcache were involved, we wouldn't touch necko at all. Past that, I don't think there's an issue here insofar as things _might_ get evicted from the cache. But why is a new load what's getting evicted, instead of something older?
There is no document navigation issue here, by the way: that's working as designed. Things that are in cache get shown from cache, things that are not in cache get the error page and an option to repost. The question, again, is why a load that just happened is not in the cache.
Component: Document Navigation → Networking: Cache
(In reply to Boris Zbarsky [:bz] from comment #14) > There is no document navigation issue here, by the way: that's working as > designed. Things that are in cache get shown from cache, things that are > not in cache get the error page and an option to repost. > > The question, again, is why a load that just happened is not in the cache. And the answer is that is simply got evicted according standard cache policies - entry expired and we are out of disk cache capacity. Necko cache does nothing wrong. However bfcache mechanisms sets the flags on the channel to load only from cache and also sets the post id on the channel so that it knows which entry to look for. So, this IMO should be fixed more somewhere on the docshell/navigation level. Necko provides tools to keep the entry alive (no to be evicted), but has not the information to do it on its own. Navigation history has to do it. That's why I talk about "bfcache".
(In reply to Boris Zbarsky [:bz] from comment #13) > There is no "bfcache" involved here. Just regular cache. Please don't > confuse them. If bfcache were involved, we wouldn't touch necko at all. > > Past that, I don't think there's an issue here insofar as things _might_ get > evicted from the cache. But why is a new load what's getting evicted, > instead of something older? Older cache load gets evicted. This happens on going back.
To be more precise: everything that occupies the (in my case) 4MB of the cache has a longer freshness lifetime than the previous POST, POSTs usually go away quickly since they are usually immediately expired. If you think we should keep POSTs longer despite the standard cache logic, let me know.
(In reply to Benjamin Peng from comment #1) > It might be related to Bug 200208, but my problem only happens when cache is > full. And the setting is standard?
> entry expired and we are out of disk cache capacity OK. So the point is that the cache is full of non-expired things, but this new load is set to expire immediately? > However bfcache mechanisms Please stop using the word "bfcache" when you mean "session history"? It really confuses me every time, and I'm sure it confuses everyone else who reads your comments... They are NOT the same thing. > sets the flags on the channel to load only from cache and also sets the post id on the > channel so that it knows which entry to look for. Correct. > So, this IMO should be fixed more somewhere on the docshell/navigation level. I don't think there's anything to fix here on the docshell side; we don't want to pin the cache entry or anything like that. > Necko provides tools to keep the entry alive We don't want to keep the entry alive at all costs here. It should be fine to evict it. I was just surprised that the second-most-recently-added entry was being evicted, but if that's the only expired thing around that makes sense. > If you think we should keep POSTs longer despite the standard cache logic, let me know. I think there may be value in keeping POSTs that were recently added to the cache even if they're expired, in favor of evicting something that hasn't been accessed in a long time.... It's a tough call.
Maybe we shouldn't consider POST responses as "expired" as long as there's a session-history entry which refers to its post-id.
That would pin them pretty much forever in some cases. We could do that already, by having the session history hold a cache token, instead of a cache key. But we don't want to actually force that behavior....
I'm not suggesting to pin things in the cache. I'm saying "don't consider it expired". We still (presumably) toss non-expired things out from the cache based on some LRU-like algorithm.
Please also check bug #1092752.
(In reply to Boris Zbarsky [:bz] from comment #19) > We don't want to keep the entry alive at all costs here. It should be fine > to evict it. I was just surprised that the second-most-recently-added entry > was being evicted, but if that's the only expired thing around that makes > sense. It doesn't make sense to evict expired entries first. It would only make sense if they cannot be re-validated and there's no instance in session history left. A non-expired cache entry is more likely to be still available by other means, so the cache is less essential.
FYI, this bug sill exists in newest Nightly. And for anyone who didn't waste time reading https://support.mozilla.org/zh-CN/questions/1019146 , there is a website can guarantee to reproduce this bug: https://startpage.com/ Two things: 1. make sure your disk cache is FULL 2. at least search twice before you hit "back" to get the bug
OK After mozregression, changes: Last good: 05-16 1st bad: 05-17 http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=58c5a3427997&tochange=2893f60d5903
Keywords: regression
not a regression per comment 18
Keywords: regression
See Also: → cache2enable
(In reply to Wayne Mery (:wsmwk) from comment #27) > not a regression per comment 18 What? Comment 18 said nothing and it's a regression because the window is already found?
Keywords: regression
If you mean the "non-standard" setting of cache size, it actually doesn't matter I did so just for quicker testing. The bug will happen when cache is full no matter what size of cache you set (includes default: auto).
Sorry, I mean comment 12 (I didn't read comment 18), which says not a regression of cache2 by may be tickled by it. But, if it is a clear regression it would be nice to identify the bug.
(In reply to Wayne Mery (:wsmwk) from comment #30) > Sorry, I mean comment 12 (I didn't read comment 18), which says not a > regression of cache2 by may be tickled by it. But, if it is a clear > regression it would be nice to identify the bug. Oh OK, that makes sense now :D In my testing I can 100% produce the bug with cache2 and 0% with cache1 but I only used mozregression to test different builds once so it may suffer the problem of lack of samples.
Assignee: nobody → honzab.moz
See Also: → 1079789
This bug has been haunting me ever since I upgraded to firefox 32, and it's still present in 35.0.1. It sure looks like a regression to me, since it never happened to me before, and then firefox 32 landed and then suddenly it started happening many times every day. Clearing my cache temporarily stops it from happening, but it just comes back again. It's quite easy to reproduce with startpage.com search results, perhaps because startpage uses https and post for its form submits. Just so nobody assumes it's caused by a cache directive from the web server, here are some sample headers from a startpage.com search. Note that the response doesn't contain any cache directives. POST /do/search HTTP/1.1 Host: startpage.com User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Referer: https://startpage.com/ Cookie: <REDACTED> Connection: keep-alive Content-Type: application/x-www-form-urlencoded Content-Length: 77 HTTP/1.1 200 OK Date: Mon, 09 Feb 2015 19:26:42 GMT Server: Apache Content-Language: en Content-Type: text/html Set-Cookie: <REDACTED> Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Transfer-Encoding: chunked In short, basic back/forward functionality has been broken on Firefox for nearly half a year, and it's really painful for those of us using an affected site regularly. If the problem is being ignored because nobody can reproduce it, I'd be glad to help with testing here. I reproduce it many times per day.
Forest, if you are interested in helping, I'm curious if bug 1079789 has an effect on this bug too. Please install a try build from http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/honzab.moz@firemni.cz-61ab7175c859/ according to your platform and check. Rather use a new profile or backup your current one before running this try build.
Flags: needinfo?(mozilla)
(In reply to Honza Bambas (:mayhemer) from comment #33) > Please install a try build from > http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/honzab.moz@firemni. > cz-61ab7175c859/ according to your platform and check. I created a new profile, copied my cache files into it, and tested with Firefox 35.0.1 from Ubuntu Trusty vs. the following build: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/honzab.moz@firemni.cz-61ab7175c859/try-linux64/firefox-38.0a1.en-US.linux-x86_64.tar.bz2 Firefox 35.0.1 from the Ubuntu repo: - When my cache was full, navigating back to startpage.com search results produced the "Document Expired" message described in this bug. - When my cache was freshly emptied, navigating back to startpage.com search results produced the expected page almost instantly. The Live HTTP Headers extension confirmed that going back did not reload the page from the server. firefox-38.0a1 from your ftp site: - When my cache was full, navigating back to startpage.com search results produced the expected page, but it took a few seconds. The Live HTTP Headers extension confirmed that going back reloaded the page from the server. - When my cache was freshly emptied, I got the same behavior. The Live HTTP Headers extension confirmed that going back reloaded the page from the server. It was as though the cache was not used at all.
Flags: needinfo?(mozilla)
Just to avoid confusion: When testing with a freshly emptied cache, I first cleared my cache, and then initiated a new startpage.com search and navigated away from it and back again.
Attached patch workaround — — Splinter Review
I'm running with the attached workaround without problems. A real solution could evict expired entries that are very unlikely to revalidate or to be reused unrevalidated, but that isn't necessary. They should go away soon enough by the normal frecency algorithm like any other unused entry.
Depends on: 1136897
Status: NEW → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → DUPLICATE
(In reply to Honza Bambas (:mayhemer) from comment #37) > *** This bug has been marked as a duplicate of bug 1079789 *** Are you sure it's a duplicate? As a reminder, the build you had me test did not seem to restore the expected behavior: it caused page reloads from the server instead of from the cache.
(In reply to Forest from comment #38) > (In reply to Honza Bambas (:mayhemer) from comment #37) > > *** This bug has been marked as a duplicate of bug 1079789 *** > > Are you sure it's a duplicate? As a reminder, the build you had me test did > not seem to restore the expected behavior: it caused page reloads from the > server instead of from the cache. Goal of this bug was to make back/forward work again, it's now fixed. That we are reloading the page is because there is no-store (or no-cache + https) in the post response. It's not a bug.
(In reply to Honza Bambas (:mayhemer) from comment #39) > Goal of this bug was to make back/forward work again, it's now fixed. That > we are reloading the page is because there is no-store (or no-cache + https) > in the post response. It's not a bug. But there isn't a no-store or no-cache in the response. See the header fields I posted in Comment 32.
I do see them when I'm testing.
Interesting: I just checked again, and startpage is now including Cache-Control in its responses. That's new. I wonder if the change was a response to the Firefox bug. *shrug*
Why this bug marked as "resolved"? I still have the problem on https://startpage.com with newest nightly.
(In reply to Benjamin Peng from comment #43) FWIW, there is bug 1138291.
And also bug 1136897, which is a continuation of this one. This bug applied to entries that were active (loaded in memory and used). There is a limited size pool of these active entries. And when we needed more space in it, we were purging and dooming (removing also from disk) expired entries. That bug also prevents dooming expired disk entries when we are running out of disk space. Yes, there is another layer bellow that manages the disk cache capacity and keeping under a certain limit. That bug landed just today. If you still can reproduce with Nightly having bug 1136897 fixed, please open a new bug, since the cause is probably different this time. This bug has been closed. Thanks.
Never mind, the problem on https://startpage.com *now* (it wasn't) is because the site changes behavior on its own..Chrome returns same warning (Error code: ERR_CACHE_MISS).
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: