1472482 - Refreshed page still downloads old cached versions of its links

Status: NEW

(Toolkit :: Downloads API, defect, P3)

Description

[LorenAmelang]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Build ID: 20180621125625

Steps to reproduce:

Downloaded 
https://www.nirsoft.net/utils/browsinghistoryview-x64.zip
from 
https://www.nirsoft.net/utils/browsing_history_view.html
a few days ago. Page author has since updated the file. When I "download" it now I get the old version - even though I see the new version of the page listing the link. The Library -> Downloads view seems to show each new download as a fresh request, no mention of grabbing it from cache. (The file is too small to create an obvious wait!)



Actual results:

I refreshed the page, and still get the old download file. I read a lot of old bugs, and tried Shift -> RightClick -> Save link as...  which gets the new file. After that the regular click also gets the new file. 

This has wasted way too much of my time and the file author's time...  


Expected results:

How are users supposed to know this? At the very least, I would suggest that if a user explicitly refreshes a page, everything to do with the old page version should be removed from the cache, including downloads linked from it. 

As 
https://bugzilla.mozilla.org/show_bug.cgi?id=138117
said thirteen years ago:
"The user agent is not fetching the file that is on the server, and its not
informing the user that it is not doing what they requested.  Whenever a browser
doesnt perform exactly like wget (barring bugs of course), its breaking semantic
transparency; which is ok, but it should only do this with good cause (i.e. the
user has specifically requested this e.g. user pref.), and it should keep the
user in the loop."

A note in the Library -> Downloads view that the "download" came from cache instead of being freshly requested might help, but most people don't look there. A warning in the initial "Opening <file>" dialog that a local copy will be opened, instead of a new download, might get more notice. A selection to request a fresh download should accompany that!

Comment 1

[mkm]

Please increase the priority of this bug! It is not acceptable for a browser to just randomly cache stuff and never even ask the origin server if it might have been updated!

I have just spent a couple of hours trying to fix a phantom "only in CI" bug that was actually only Firefox not downloading the modified file from the repository, even though there was an ETag present in the cached response, so up-to-date checking would have been quick and easy.

Additionally, Firefox made it unnecessarily hard to diagnose this problem by showing misleading information in the network tab. I was believing the "200" status code (a lie, since there was no actual request made at that time), when I should have been scanning all available columns to find one named "Transferred", which would have shown that the request was "cached".
In hindsight, the "Date" header and the Response/Request header sizes of 0B would have hinted at this as well, but this is still no excuse. What good is the network tab if I cannot even trust the status code?

Comment 3

[Marco Bonardo [:mak]]

It sounds like a cache bug, for which the server may not be sending the appropriate headers.
It may be a longstanding problem we have, I see bug 138117 that is 17 years old and a bunch of bugs similar to this one are duplicated to it.

Unfortunately this may fall in the category of expensive fixes for niche cases, that means high cost for small benefit, that delays the fix until someone from the community chimes in.