20MB regression in application 'Data' size after first startup

RESOLVED WONTFIX

Status

()

RESOLVED WONTFIX
6 years ago
3 years ago

People

(Reporter: pretzer, Assigned: dougt)

Tracking

({regression})

21 Branch
All
Android
regression
Points:
---

Firefox Tracking Flags

(firefox20 unaffected, firefox21+ wontfix, firefox22 wontfix, firefox23 wontfix, fennec21+)

Details

(Reporter)

Description

6 years ago
I noticed a regression between FF20 (Beta) and FF21 (Aurora) regarding the size of the 'Data' Firefox generates after first startup, which is visible via the 'Application manager' in the Android settings.

STR:
1) Go to Android settings --> Application manager
2) Select Firefox Beta
3) Select 'Clear data'
4) Start Firefox Beta
5) Go back to the Application manager and check the value under 'Data' (Megabyte)
6) Do the steps above for Aurora as well and compare the values

These are the values I get on my device (Galaxy S2, Android 4.1.2):
Firefox 20 (Beta) --> 1.86 MB
Firefox 21 (Aurora) --> 21.66 MB
Firefox 22 (Nightly) --> 21.64 MB

This suggests an approximate regression of 20MB between FF20 and FF21. I'm not sure if this data is intended/needed or if it is just a bug, but considering that Firefox is increasingly extending its focus to lower end phones this seems like a pretty big deal.
On my LGE Nexus 4 (Android 4.2.2)

* Firefox 20 (Beta) → 1.78 MB
* Firefox 21 (Aurora) → 9.66MB
* Firefox 22 (Nighlty) → 9.54MB
Pulling the profiles of Nightly (http://cl.ly/image/1J2R2L0G0s2x) against Beta (http://cl.ly/image/2l0I0C0W0p1a), I only see a difference in determined cache size:
Looks like maybe it's safebrowsing related? CC'ing gcp
Keep in mind that the Cache size depends on the amount of space available on the device ("smart sizing"). But that behavior should be the same, for a given device, since 14 (see bug 745340).
Actually, cache smart sizing determines the maximum size of the disk cache -- but these measurements are taken before any significant page loads, so I would expect the cache to be mostly empty, and its size dominated by the pre-allocated block files, _CACHE_001_, etc. Maybe the size of the block files changed between 20 and 21?
>Looks like maybe it's safebrowsing related?

The screenshots AaronMT posted clearly show that that is not the case...
There is a change in the disk cache block files found in <profile>/Cache.

On Firefox 20, initial block files look like this:

-rw------- app_139  app_139     16538 2013-03-26 09:30 _CACHE_001_
-rw------- app_139  app_139      4096 2013-03-26 09:30 _CACHE_002_
-rw------- app_139  app_139      5978 2013-03-26 09:30 _CACHE_003_
-rw------- app_139  app_139       276 2013-03-26 09:30 _CACHE_MAP_

On Firefox 22, initial block files look like this:

-rw------- app_123  app_123   4194304 2013-03-26 09:30 _CACHE_001_
-rw------- app_123  app_123   4194304 2013-03-26 09:30 _CACHE_002_
-rw------- app_123  app_123   4194304 2013-03-26 09:30 _CACHE_003_
-rw------- app_123  app_123      8468 2013-03-26 09:30 _CACHE_MAP_

I'm pretty sure this is intentional, but I haven't found a bug or changeset.
Re comment #7, that matches what I'm seeing in my pulled profiles too
(In reply to Geoff Brown [:gbrown] from comment #7)
> I'm pretty sure this is intentional, but I haven't found a bug or changeset.

See https://bugzilla.mozilla.org/show_bug.cgi?id=816642#c5
Blocks: 816642
(Reporter)

Comment 10

6 years ago
So is there anything we can do about this or do we have to live with it? Did anyone contact the Snappy/Performance team or the assignee of bug 816642 about this?
tracking-firefox21: --- → ?
CC'ing :michal and :taras. Michal, I'd like to understand better how bug 816642 affects Fennec. It looks like based on bug 816642 comment 0 slow disk performance on Android was one of the causes the patch was landed in the first place, but is there some heuristic that limits how much of the available disk space it uses up? Eating up 20 MB on lower end phones seems significant.

As an aside, it looks like bug 816642 never actually got an r+ on the bug but landed with r=jduell. Please make sure all reviews are recorded on the bug, even if they happen elsewhere like on IRC.
Flags: needinfo?(michal.novotny)
tracking-firefox21: ? → +
Doug, assigning to you to get an assignee
Assignee: nobody → doug.turner
tracking-fennec: ? → 21+
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #11)
> CC'ing :michal and :taras. Michal, I'd like to understand better how bug
> 816642 affects Fennec. It looks like based on bug 816642 comment 0 slow disk
> performance on Android was one of the causes the patch was landed in the
> first place, but is there some heuristic that limits how much of the
> available disk space it uses up? Eating up 20 MB on lower end phones seems
> significant.

Preallocation of the cache blockfiles eats only 12MB. Given that minimum cache size on Android is 10MB regardless the available space, I think this is OK.


> As an aside, it looks like bug 816642 never actually got an r+ on the bug
> but landed with r=jduell. Please make sure all reviews are recorded on the
> bug, even if they happen elsewhere like on IRC.

That's not true. Patch v1 (#687065) has r+ from Jason. The final patch differs only in that fallocate() is disabled on MacOSX due to Tp5 regression. Such change doesn't need a new review.
Flags: needinfo?(michal.novotny)
(In reply to Michal Novotny (:michal) from comment #13)
> Preallocation of the cache blockfiles eats only 12MB. Given that minimum
> cache size on Android is 10MB regardless the available space, I think this
> is OK.
> 

If I'm understanding correctly, you're saying that the cache blockfiles should take up 12MB, and the cache will live inside this space. So the startup disk-usage regression should be 12MB, and as the cache fills up, the "regression" should shrink because the used blockfile space will be similar to the space used on disk without the blockfiles. Am I understanding that correctly? If so, why is there a 20MB regression noted in comment #0? I would expect that to be only 12MB.

> That's not true. Patch v1 (#687065) has r+ from Jason. The final patch
> differs only in that fallocate() is disabled on MacOSX due to Tp5
> regression. Such change doesn't need a new review.

Ok, but please still mark the new attachment as having an r+ carried over from the previous patch for clarity.
Doug, still looking for an assignee
Flags: needinfo?(doug.turner)
Summary: the test is flawed.

preallocation: 
In prior releases we sparsely allocated _CACHE_* block files which used less space for a particular run with a new profile. However they would eventually fill up on real clients anyway. The difference is that now the files are packed together, so you don't get really shitty IO patterns on read/write(eg less slow).

I expect that this will be fixed in the new cache design once we get rid of block files.
I don't disagree, but urge us to keep in mind that a certain class of user will still be alarmed/disappointed, taking the perspective "I just installed this browser and it's using X MB of data even before I have done anything!"
(Assignee)

Comment 18

6 years ago
Geoff, your team needs to make this trade off.  Please let us know what you want to do.
Flags: needinfo?(doug.turner)
(In reply to Taras Glek (:taras) from comment #16)
> Summary: the test is flawed.
> 

Taras clarified on IRC that the "test" he is referring to is "starting the browser for the first time and checking the disk usage". I agree that this space would fill up anyway after some usage, so I don't have a problem with it getting pre-allocated to improve performance. However my question in comment 14 (why 20 MB instead of 12 MB?) is still unanswered. I'm ok with leaving the behaviour as-is as long as we can account for all of the space being pre-allocated. (gbrown also pointed out that in comment #1 Aaron saw a ~8MB regression rather than the ~20MB regression that Peter reported, so maybe this depends on device?)
(In reply to Kartikaya Gupta (email:kats@mozilla.com
> However my question in
> comment 14 (why 20 MB instead of 12 MB?) is still unanswered. I'm ok with
> leaving the behaviour as-is as long as we can account for all of the space
> being pre-allocated. (gbrown also pointed out that in comment #1 Aaron saw a
> ~8MB regression rather than the ~20MB regression that Peter reported, so
> maybe this depends on device?)

There are other differences in data files and file sizes between 20 and 21. The most notable is res/fonts, empty in 20 but populated in 21:

shell@android:/data/data/org.mozilla.firefox_beta # ls -l res/fonts
-rw-rw-r-- app_139  app_139   1672660 2013-04-15 16:11 CharisSILCompact-B.ttf
-rw-rw-r-- app_139  app_139   1664448 2013-04-15 16:11 CharisSILCompact-BI.ttf
-rw-rw-r-- app_139  app_139   1690628 2013-04-15 16:11 CharisSILCompact-I.ttf
-rw-rw-r-- app_139  app_139   1724344 2013-04-15 16:11 CharisSILCompact-R.ttf
-rw-rw-r-- app_139  app_139    224592 2013-04-15 16:11 OpenSans-Bold.ttf
-rw-rw-r-- app_139  app_139    213292 2013-04-15 16:11 OpenSans-BoldItalic.ttf
-rw-rw-r-- app_139  app_139    212896 2013-04-15 16:11 OpenSans-Italic.ttf
-rw-rw-r-- app_139  app_139    222412 2013-04-15 16:11 OpenSans-Light.ttf
-rw-rw-r-- app_139  app_139    213128 2013-04-15 16:11 OpenSans-LightItalic.ttf
-rw-rw-r-- app_139  app_139    217360 2013-04-15 16:11 OpenSans-Regular.ttf

The total size of those files is 8055760 bytes. See bug 831354.

There are additionally about 2 or 3 MB of other files in a typical Firefox 21 profile.

I think it makes sense for Android to report 21 MB to 23 MB of data usage for Firefox 21+: 12 MB cache, 8 MB fonts, 2 or 3 MB other.

If I delete the cache files, Android reports 9 to 10 MB of data. This likely accounts for Comment 1: If :aaronmt's cache was not allocated yet, he would have seen numbers like that. It seems like the cache is sometimes allocated immediately on first run, and other times, it doesn't happen until you visit a page.

I am satisfied that the increase in data usage is by design and bounded. I don't have any outstanding concerns.


Closing as WontFix to emphasize that there was a 20 MB regression here but we are willing to live with that given the additional functionality and performance.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WONTFIX

Updated

6 years ago
status-firefox21: affected → wontfix
status-firefox22: affected → wontfix
status-firefox23: --- → wontfix
You need to log in before you can comment on or make changes to this bug.