Closed Bug 658969 Opened 13 years ago Closed 8 years ago

Bad behavior when cache max size set to 0 MB

Categories

(Core :: Networking: Cache, defect)

Product:
Core
Component:
Networking: Cache
Version:
2.0 Branch
Platform:
x86_64
Windows 7
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: mozilla_bugzilla, Unassigned)

References

(
URL
)

Details

User-Agent:       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Build Identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1

This happened to me twice since the Fx 4.0.1-update. I was running the beta's for Fx4 since december 2010 and installed the final straight after it was released. Never encountered this particular problem before.

I installed the Fx4 final after uninstalling all other versions of Fx or Minefield I had, and I started from scratch with a clean profile. I have not experienced any other problem crashes, hangups or other problems.

I'm unable to reproduce, but the symptoms point to something quite critical. That's why I will report it here:

After using Fx4 for some time, (say, a day without quitting) the browser suddenly starts to:
- produce 'bad' results for webpages such as displaying only two lines of random CSS-code from the site;
- OR displaying only a fraction of the (rendered) HTML of the page;
- OR, display nothing but a blank page;
- OR, display an error saying it was unable to interpret the received data (sorry, forgot to write it down). I think it was something like 'bad data' or 'malformed request'. This happened to me when using Google with live search. If I encounter this again I'll write down the message.

In all the above situations, refreshing will simply return the same results. If the page was blank, refreshing will return a blank page. If I got a random string of CSS, I will see that same string after refreshing.

This all seems to be happening after some time of using the browser for standard stuff like Youtube, Google, Google docs, gmail, some news sites or blogs.

If it happens, all menu's and UI elements are still responding. I can just close windows and quit Fx, then open it again to make it all work. If I leave Fx open and use another browser to check on the sites that are borking on Fx, the other browser shows them just fine.

To my surprise, Thunderbird seemed to suffer from a very similar problem, at the very same time! It wouldn't show me e-mails anymore, or return search results. It would still display the list of mails and inboxes.

Does this sound familiair to anyone? I have no idea what to do to provide better info, and I'm also restricted in the time I can spend on solving this.

Reproducible: Didn't try

Steps to Reproduce:
I haven't managed to find anything that could lead me to reproducing this. It appears to be happing at random, after some hours, maybe a day, of using Firefox.



My extensions for Fx4 are
DOM inspector
(Video) DownloadHelper
LogMeIn Remote Access Plugin
Also, Java Console 6.0.20 and .21 are listed
Using default skin.

I have not installed any extra extensions for weeks. Certainly not since 4.0.1 came out.

I'm using Windows 7 Ultimate 64-bit with all updates, a proper virusscanner (ESET NOD32) with daily updates. Got 4 gigs of ram, not much software running: Skype, PuTTY, a few (file) explorer windows
Forgot to complete my last sentence:
[...] not much software running: Skype, PuTTY, a few (file) explorer windows, Thunderbird, Crashplan, Tweetdeck. Sometimes Photoshop or InDesign. Last time this occured the system wasn't under load.
When this happens, can you check if your disk (where your Firefox profile and cache is located) is full? Or if you go over the FF disk/memory cache limit (see about:cache). Can you tell us how much memory FF is using at that time (about:memory)?
Version: unspecified → 4.0 Branch
I'll look up that info the very next time I run into it. The disk full-scenario doesn't seem likely to me at this point but I will check for that as well.
Just experienced pictures not completely loading, ending with some garbage like when you have a corrupted JPG. I checked my disk and indeed: the disk that has my profile was full. Free space on that disk was 1 GB this morning (usually it's more) - I suspect something strange is happening when streaming the webcam of a local radio station.

Just in case, the info on disk en memory:
Memory mapped: 683,671,552
Memory in use: 385,650,576
"Other info" showed 463,216,640 committed, 2,703,360 dirty, 562,679,808 privatebytes and 709,582,848 workingset to name a few.

Memory cache device:
Number of entries: 7524
Maximum storage size: 30720 KiB
Storage in use: 27604 KiB
Inactive storage: 27623 KiB

After closing Fx4, the disk returns to having 1 GB of free space. Just closing the window that has the webcam doesn't solve it.

In that case, steps to reproduce on my machine:
- Surf to http://www.3fm.nl/socialradio/app
- View webcam for ~2 hours to fill up 1 GB of space
- Random things will start occuring when the disk is full

I'm entirely sure the problem of not releasing the space on disk is a Fx4 problem. ESET NOD32 has caused a similar problem to me when I was still using 3.x.x. I'll check tomorrow and ask them for advice.

However, I would suggest, since the effects are kinda random:
- Better handling of a 'disk full' scenario. Perhaps display an errorpage when there's less than 100 MB available on the profile disk with an override button?
Sorry, minor but crucial error:
"I'm entirely sure [...]" -> I'm NOT entirely sure [...]".

As in, I'm not sure if Fx4 does something that upsets ESET NOD32 specifically, OR that it's a problem for other virusscanners as well, OR that NOD32 gets spun up on this entirely by itself.

Since I have had experiences with something similar I will contact ESET on this first.
Thanks, this may move us forward. It it quite possible that the webcam feed is stored into one giant file inside Firefox cache and fills your disk. You didn't include the Disk cache device stats, please post it.
Moving to a better component.
Component: General → Networking: Cache
Product: Firefox → Core
QA Contact: general → networking.cache
Version: 4.0 Branch → 2.0 Branch
...post it when it happens again. We want to see what limit you set on the disk cache and if Firefox is storing more than that.
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:7.0a1) Gecko/20110621 Firefox/7.0a1

Martin, can you please add the Disk cache device stats as aceman suggested in Comment 6?

Thanks!
Terribly sorry for my late response. Lot's of things to do.

I can be short on what limit I set on my disk cache: 0 MB.
That must be why there where no disk cache device stats on about:cache.

The reason I have it set to 0 MB is as follows:
I do a lot of HTML/CSS webdevelopment work. While using previous versions of Firefox (2.x, 3.0, 3.6), I sometimes ended up with Firefox serving me a cached version of a document after I had made changes. This lead to several cases which cost me a lot of time searching for what went wrong.

Firefox didn't have options to disable the cache but it allowed me to set it to 0, which I figured would practically have the same effect. I have set my cache to 0 MB since then to make sure I didn't end up viewing an old document again. This setting has worked very well for me all those years. Never experienced any problems.

Is this specific setting causing my problems? Do you think a setting of 0 MB is a legit value? Or should users not be able to set a disk cache of 0 MB?
Good question if it is legit. However, there is a separate pref in about:config called 'browser.cache.disk.enable'. Try setting that to false and see if it makes a difference (doesn't fill the disk). The webcam on the testpage seems to use Silverlight plugin. So even if FF is not caching anything, the plugin may do its own caching and doesn't obey FF settings. I can't confirm it myself, because I have Silverlight 3, but the video still does not play. Can you see where the file of the webcam stream is growing? Is it in the Firefox cache folder? Or is it somewhere else, like system/user temporary folder or some silverlight specific folder?
URL: http://www.3fm.nl/socialradio/app
I was unaware of that setting: I will test the results of setting that next week.

The temporary files created are not in de Fx cache. They are created in \Windows\Temp. The names are HTT[4 hex chars].tmp. They vary in size from less than a single MB to 1GB+. The files cannot be removed when Fx is open. Windows 7 will refuse to remove them.

Other strange things occur with those specific files, for example, the files sometimes change icon from the Skype-icon to Excel-icon and then back again. Then to a icon with a lock on it, then back to something else.

I'm pretty sure this is somehow caused by a combination of Fx, Silverlight and NOD32. I forgot to report this to NOD32 after comment #4 but I will report this to them today including this thread and silverlight details and see what they come up with.

Summery: Fx doesn't seem to be a likely cause of the disk filling up, but if the disk does fill up then it behaves strange with a disk cache set to 0 MB. I will test with browser.cache.disk.enable set to 0 and see if the result is the same or different. Will report if NOD32 has anything to say on the problem.
> behaves strange with a disk cache set to 0 MB. I will test with 
> browser.cache.disk.enable set to 0 

I suspect that setting disk cache max size to 0 will not stop streaming files from using the disk cache, when it's set to be enabled overall. Setting browser.cache.disk.enable to 0 may give you better results.

(Yes, there's no logical difference between the two: perhaps we should add special detection logic to equate max size = 0 with disabling disk cache).

There's still a chance the media cache will store at least some stuff on disk anyway (roc?) but it shouldn't be 1 GB as far as I know.

So, Martijn, let us know how browser.cache.disk.enable=false works for you.
Summary: Firefox stops loading pages or produces very weird/incomplete results → Bad behavior when cache max size set to 0 MB
You can set media.cache_size to 0 or something small, but some data will still be stored while HTML media elements are active.
Guys, this seems to be Silverlight plugin doing the caching/storing the file (comment 11). Maybe Firefox cache settings have no effect on it. We need the original reporter to confirm this. And if that is the case, is Firefox able/intended to somehow force the cache limit on plugins? Probably not.
It took me longer than expected to test and I'm also still working this out with Eset support.

What I've tested and what my interpretations are:
- The problem with files appearing in \Windows\Temp is purely a Eset NOD32 issue and I hope to get them to solve that. If I simply shut down NOD32 Real-time protection, no files are created. That leads me to conclude that this isn't an issue in Firefox, nor in any of the plugins like Flash or Silverlight. I also think it is safe to assume that the creation of these files is not influenced by Firefox cache settings, but I will test that.

- A few weeks ago I updated to Fx5, build identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
I didn't make changes to my setup so my disk cache is set to 0 MB, but browser.cache.disk.enable is 'true'. Media.cache_size is the default 512000.

Over the last weeks I had several moments where my disk was full, but this time Firefox responded much better: no problems with 'chunks' of pages being displayed or pages not rendering properly. All pages rendered as I expected them to render.

I did notice YouTube will start to throw 'an error occurred, try again later' when my disk is full. Other than that, I see no signs of strange Firefox behavior when the disk is full on version 5.0. I don't know what changes fixed this issue.

Looking at these findings, do you still want me to test the behaviour with browser.cache.disk.enable = false and/or media.cache_size? What particular circumstances would you like me to test?

For the record: I have no problems with media like sound/video/animation being cached. It's just the cache for HTML/CSS that I felt worked better disabled. See #9.
Good to hear that. Maybe the memory cache got better and is kicking out old files and storing full copies of new files downloaded. But I have not looked if there were any relevant changes.

Yes, try browser.cache.disk.enable=false as that is the official means of disabling the disk cache.
However, if you just have the size set to 0, do you see the disk cache in about:cache? Is its block there with size:0 and used:0 or is it completelly missing?

Yes, youtube flash probably still want to store parts of the videos on disk and fails.
With just the cache size set to 0 MB, I don't see any disk cache summary in about:cache. The only two blocks I see are Memory cache device and Offline cache device.

With browser.cache.disk.enable = false, I don't see the disk cache summary either, independent of wether browser.cache.disk.capacity is set to 0 or higher. I would recommend showing the block even when cache size is set to 0, and display 'disabled' when browser.cache.disk.enable = false.

For the record, setting browser.cache.disk.enable = false doesn't influence the creation of files in \Windows\Temp at all. I just tested that just to be sure.
I will leave the setting like that to see if it works as expected.
At this moment it seems that both settings, (1) browser.cache.disk.enable = false and (2) setting the cache size to 0 MB, behave the same. In both cases Firefox 5 just continues to work without problems unless a website requires media to be cached like YouTube or Vimeo. So that's good, or expected behaviour.

The remaining question is then wether or not setting the disk cache to 0 MB should automatically trigger browser.cache.disk.enable to false, or leave it true and just set disk cache = 0 in the configuration.

Concerning the issue I reported to Eset, this was their response:
"There is no way to modify the behavior of cleaning Web protection temp files. If any web page is creating a lot of large files there (e.g. YouTube), you might want to add it to Addresses excluded from web scanning.

We have forwarded the case to the dev team, so they could come up with a solution on the long term."

Just for others who may be experiencing this issue and want a fix, here's a thread about this issue on the Wilders Security Forum with some suggestions: http://www.wilderssecurity.com/showthread.php?t=260611
To clarify; by expected behaviour I mean the webpages usually shows an error by themselves if a video can't be cached or loaded because of a full disk. The error message itself isn't referring to any problems with the disk of course.

I don't think it's possible to create better handling for that without over-doing things.
That it probably the fault of the flash video player application that it doesn't provide meaningfull error messages. Mozilla can't do anything about that (I don't know whether plugin disk writes go through Mozilla APIs, but probably not). The author of the flash video player must fix that.
(In reply to Martijn from comment #20)
> To clarify; by expected behaviour I mean the webpages usually shows an error
> by themselves if a video can't be cached or loaded because of a full disk.
> The error message itself isn't referring to any problems with the disk of
> course.
> 
> I don't think it's possible to create better handling for that without
> over-doing things.

meaning wontfix?
Flags: needinfo?(mozilla_bugzilla)
(In reply to :aceman from comment #21)
> That it probably the fault of the flash video player application that it
> doesn't provide meaningfull error messages. Mozilla can't do anything about
> that (I don't know whether plugin disk writes go through Mozilla APIs, but
> probably not). The author of the flash video player must fix that.

Yes, agreed.

(In reply to Wayne Mery (:wsmwk) from comment #22)
> (In reply to Martijn from comment #20)
> meaning wontfix?
For that particular part of this report, yes.

I believe only this remains:

(In reply to Martijn from comment #19)
> The remaining question is then wether or not setting the disk cache to 0 MB
> should automatically trigger browser.cache.disk.enable to false, or leave it
> true and just set disk cache = 0 in the configuration.
Flags: needinfo?(mozilla_bugzilla)
changing one pref should not change the other... doing so creates odd push/pop behavior in code that changes prefs.
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.