1440986 - Appcache folder wrong in about:cache when browser.cache.disk.parent_directory is used.

Status: RESOLVED WONTFIX

(Core :: Networking: Cache, defect, P5)

Description

[Dr B W Darvell]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0
Build ID: 20180206200532

Steps to reproduce:

open  about:cache?storage=&context=
refresh does not change it


Actual results:

memory
Number of entries: 	2266
Maximum storage size: 	32768 KiB
Storage in use: 	21952 KiB
Storage disk location: 	none, only stored in memory

disk
Number of entries: 	2266
Maximum storage size: 	32768 KiB
Storage in use: 	21952 KiB
Storage disk location: 	none, only stored in memory

appcache
Number of entries: 	136
Maximum storage size: 	512000 KiB
Storage in use: 	4827 KiB
Storage disk location: 	R:\FFcache\OfflineCache



Expected results:

1.  R:\FFcache\OfflineCache does not exist. R: is a RAM disk (10 GB), created from scratch at each reboot (no save and reload).  The entries which show are all old, ranging from 
2017-10-26 19:24:15 	Expired Immediately 	
to
2018-02-04 16:17:12 	Expired Immediately 	

To my mind, none should appear (the system has been rebooted a number of times since that last date), but there is no proper cache folder being created anyway.  Is this an oversight - is nothing really being cached as it should?
Where are these old items being stored?  Why do they reappear?  How can this be cleaned up?

2.  What is present is:
R: FFcache -
....... cache2
...............doomed
...............entries

- all of which are completely empty apart from the folders themselves.  There is nothing else.  These entries appear to be useless, but obviously recreated each time.
That structure (also empty) also appears in the original hard-drive location, i.e. left over from before I created the RAM disk (in the hope that things would be speeded up a lot).

I can find no other active cache locations on my system (the names might be obscure and not involve 'cache', of course).

3.  The "disk" entries in the report list are the same as the "memory" entries - which makes no sense.

I can provide all relevant config settings, of course, if these are identifed for me.

Note that I do not see any operational fault as such - FF is (apparently) working nicely.  But, it is plain that cacheing is not working as it appears it should.

Thanks,

BWD

58.0.2 (64-bit)  Win Pro 7  54 GB RAM, Xeon 8 cores.

Comment 1

[u408661]

Honza, can you take a look at this?

Comment 2

[Honza Bambas (:mayhemer)]

Not right now.

Comment 3

[Honza Bambas (:mayhemer)]

Note that we report what's stored in memory cache being contained in the disk cache as well, because internally this is how the things work - querying disk cache also looks into memory cache.  That is by design, but you are not the first to complain about how it's presented in about:cache.  I have no immediate plans to change it, tho.

Anything related to appcache not being a serious functionality issue or a security/stability issue is an automatic wontfix.

Comment 4

[Dr B W Darvell]

Hi,
TFTR
Disk-memory report duplication: OK, no problem.  That is minor, it would seem.

As for 
Storage disk location: 	R:\FFcache\OfflineCache 
under "appcache": why does this get reported at all when it does not exist?  Why are all the entries defunct?  And where did those entries come from anyway, after rebooting?  I do not know whether those entries are capable of holding sensitive information, but I cannot delete them if I cannot find them - which is what I would like to do.  Where are they?

Then, the completely empty folder structure in R: - why is this created at all (and in the original profile)?

I understand the idea of "if it ain't broke ... ", but unless I have suffcient memory that the (ram)disk is not required at all - which would fine, it does not look as though the cache is working as intended.  If so, that is broken.

Thanks,

BWD

Comment 5

[Honza Bambas (:mayhemer)]

(In reply to Dr B W Darvell from comment #4)
> Storage disk location: 	R:\FFcache\OfflineCache 
> under "appcache": why does this get reported at all when it does not exist? 

Probably because it would be created there, but I'm puzzled why there are entries reported at all.  Also, this could be a bug - we may report the directory wrong.  How are you directing firefox to keep the cache at this folder?

note that you can delete appcache with shift-ctrl-del ("Clear All History") and pick "Offline Website Data".

> 
> Then, the completely empty folder structure in R: - why is this created at
> all (and in the original profile)?

This is the default behavior for web content cache.  We always ensure the folder structure is there, regardless if you disable disk cache (presuming you did).

Comment 6

[Dr B W Darvell]

I think there is a bug w.r.t. R:\FFcache\OfflineCache - reporting is at best misleading, certainly unhelpful because the cache is elsewhere on another disk.  Potentially a security risk?

The clear history move worked.  Was that a failure to update correctly after a crash, perhaps?
(Never having used that before, I did not realize it was selective and granular.)


From prefs.js, modified entries that include "browser.cache"):

user_pref("Browser.cache.memory.capacity", 32768);
user_pref("browser.cache.disk.capacity", 358400);
user_pref("browser.cache.disk.enable", false);
user_pref("browser.cache.disk.filesystem_reported", 1);
user_pref("browser.cache.disk.parent_directory", "R:\\FFcache");
user_pref("browser.cache.disk.smart_size.first_run", false);
user_pref("browser.cache.disk.smart_size.use_old_max", false);
user_pref("browser.cache.frecency_experiment", 2);

I had tried first redirecting the cache to the RAMdisk, but then I disabled the disk thinking it would be faster in straight RAM, but then left the settings.  I would have thought that ...disk.enable >> false would prevent other actions, so that the directories would not get created, but if the default is to create the folder, then OK.

Incidentally, if FF is restarted (no reboot), do I presume correctly that the cache has been lost?  If I enable the disk, does that then work in parallel and so be available for restart?

Thanks,
BWD

Comment 7

[Honza Bambas (:mayhemer)]

(In reply to Dr B W Darvell from comment #6)
> I think there is a bug w.r.t. R:\FFcache\OfflineCache - reporting is at best
> misleading, certainly unhelpful because the cache is elsewhere on another
> disk.  Potentially a security risk?

I don't think so.  We jsut take the folder path from a bad place, no intention to fix it (appcache)

> user_pref("browser.cache.disk.parent_directory", "R:\\FFcache");

that's what I thought, thanks.

> Incidentally, if FF is restarted (no reboot), do I presume correctly that
> the cache has been lost?  

no idea what you mean, but presuming you disabled disk cache, then obviously yes, lost after restart.

> If I enable the disk, does that then work in
> parallel and so be available for restart?

no idea what you mean

Comment 8

[Honza Bambas (:mayhemer)]

appcache issues are wontfix and this is non-standard setting.

Comment 9

[Dr B W Darvell]

re: parallel: which cache takes precedence? Is there any point in having both?

Other points - OK (untidy, but ...)

Thanks.

Comment 10

[Dr B W Darvell]

I tested re-enabling the disk cache.

R:\FFcache\cache2\entries now has many items:

memory
Number of entries: 	2341
Maximum storage size: 	32768 KiB
Storage in use: 	22660 KiB
Storage disk location: 	none, only stored in memory

disk
Number of entries: 	13963
Maximum storage size: 	358400 KiB
Storage in use: 	280110 KiB
Storage disk location: 	R:\FFcache\cache2 

so there does seem to be a point that memory-only does not cover.
(There is a discrepancy between what is shown in this report for the number of entries and what is actually on the disk.)

But now there is also:
R:\FFcache\Cache
with subfolders \0, \1, ... \F, all of which are quite empty.
and it also has _CACHE_MAP_ (276 bytes), and _CACHE_001_ ... 3_ (4 MB each), but these are data-empty files. 

R:\FFcache\_CACHE_CLEAN_ has also appeared, 1 byte: "1".

I accept your approach that if it ain't broke it is not worth spending time on, but it strikes me that some help on all of this would be useful in notes somewhere.  Guessing and testing is at best erratic.

Comment 11

[Honza Bambas (:mayhemer)]

So, now appcache is in the folder as specified with browser.cache.disk.parent_directory?  Interesting.  Not sure what is or was wrong, but makes this but even more WONTFIX then for me.

Maybe I should add a note to about:cache context view that memory entries are also contained in the disk entries listing (that it's projected into it).

Comment 12

[Dr B W Darvell]

No, that is still shown as 
   Storage disk location: 	R:\FFcache\OfflineCache 
- which still does not exist.
I did find one site that puts content under that heading (an Outlook webapp).  It will interesting to see whether that cache is emptied later, but the fact is it is not a visible file or folder, and absolutely not where it is claimed to be.

Oddly, I have just found a hdisk "Offline" cache in the relevant profile.  This is multilevel w.r.t. folders, but it has just three dates represented on contained files: March 25, Oct 26 and Nov 25 last year.  All defunct, I would guess.

The help that would be beneficial would be for how the various caches and cache settings are supposed to work.

Comment 13

[Honza Bambas (:mayhemer)]

I'm sorry, but I really don't follow the expectations and results you are facing here, there are too many confusion now.


Can you please very simply and clearly state what is the problem you are facing, what you do, what is it you expect to be the outcome and what actually happens you believe is an unexpected behavior?

I will then try to reproduce and potentially, if found necessary, fix it.

Thanks.

Comment 14

[Dr B W Darvell]

All I am saying is that the cache system makes no sense to an ordinary user, and especially if trying to optimize settings.
- folders that are stated that do not exist
- cached material that is not where it is said to be but hidden
- cached material that is not expunged when expired, and indeed lasts very much longer than makes sense
- creation of numerous supposed cache folders that are unused
- inconsistency in reporting

None of this appears to affect function, you say, therefore is not broken, therefore wontfix.

The lack of information (that I can find, at any rate), on how the cache system is supposed to work and how to optimize settings, given resources to hand, is unhelpful and could easily be provided.

So, the problem is that none of it makes sense, and that what I expect is clarity.  Tidy, logical, consistent.

Apologies if this is too low level.

Thank you.

Comment 15

[Honza Bambas (:mayhemer)]

(In reply to Dr B W Darvell from comment #14)
> All I am saying is that the cache system makes no sense to an ordinary user,
> and especially if trying to optimize settings.
> - folders that are stated that do not exist

if it's only appcache, then it's wontfix

> - cached material that is not where it is said to be but hidden

you need to be way more detailed and clear so that I understand what you mean with this

> - cached material that is not expunged when expired, and indeed lasts very
> much longer than makes sense

expired content is not removed from the cache since under certain conditions we do use it (I once changed the cache auto eviction behavior to remove expired content and I broke number of things and expectations) ; also, I don't understand why that would be a problem

> - creation of numerous supposed cache folders that are unused

again, be more clear which, where, why

> - inconsistency in reporting

yes, projection of memory only cache entries to disk cache entries is confusing, I'll file a bug to add a note to about:cache to make it clear, as stated above

> 
> None of this appears to affect function, you say, therefore is not broken,
> therefore wontfix.
> 
> The lack of information (that I can find, at any rate), on how the cache
> system is supposed to work 

it's a complex system.  our main goal is to be in parity with specifications, which we are (or at least are very very close to)

how the internals work is a complex thing, not sure what information you are looking for, but I could find some docs for you

> and how to optimize settings, given resources to
> hand, is unhelpful and could easily be provided.
> 
> So, the problem is that none of it makes sense, and that what I expect is
> clarity.  Tidy, logical, consistent.
> 
> Apologies if this is too low level.
> 
> Thank you.

Comment 16

[Dr B W Darvell]

1.  If FFcache\OfflineCache is only Appcache, then OK - ignore.
2.  Contents of appcache, which evidently is to be ignored.  It is not obvious that selective deletion can be applied because it is not obvious that the cache exist - no ordinbary user would be aware that potentially sensitive material is being stored indefinitely.
3.  expiry:  OK, but it does leave litter, and it is not apparent that it is always safe to delete.  
4.  Unused: Cache\0, etc., cache2\doomed.  If these are contingent on some condition or other, it is not obvious. EVen so, it does seem pointless to create a large structure for no reason.  It is also inconsistent: what is on the fixed disk has an extra level of many subdirectories that have not appeared (yet?) in the ramdisk version.
5.  Help: just guidance on what the caches are used for (there is clearly a difference), what priorities exist (if any), how much space is needed or sensible; what are the downsides - can a cache be too large? I do not need to know the internals, jsut the how the visible is meant to work.

As your remark about expired content makes clear, there is a lot of baggage in the system, that may well be interrelated, as things have been fiddled with over time, without really following through and cleaning up the programming débris.
I'll grant that there may well be mnore pressing matters, but as you indicate complications can ensue.

As for specs, I cannot comment, and I do appreciate that the system is very complex - impressively so, in fact.

Thanks.