Firefox (Gecko) uses large amounts of data when backgrounded with certain pages open

RESOLVED FIXED

Status

()

defect
RESOLVED FIXED
5 years ago
2 years ago

People

(Reporter: nicolas.gruel, Unassigned)

Tracking

(Depends on 1 bug, Blocks 1 bug)

29 Branch
All
Android
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(fennec-)

Details

Attachments

(2 attachments, 1 obsolete attachment)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0 (Beta/Release)
Build ID: 20140429162653

Steps to reproduce:

My phone was using my data plan and I forget it under my bed. The next day I received some message from my provider saying that I used all my data plan. The investigation to understand how I did it show to me that firefox used 1.39Gb of data in a day when under the bed... I looked at the history and I used it only to go to see one webpage. It is a very very bad surprise as you can imagine. I am sorry, I liked firefox, but until I do understand more about this problem I will not use it anymore on my cell phone. It would be too much expensive. I attach a screenshot of my data usage showing the problem. I am ready to help as much as I can. 



Actual results:

Firefox used 1.39GB of data.


Expected results:

No data downloaded since I was not using the phone.
Sorry to see this, something is definitely not right.

> I am ready to help as much as I can.

Thanks.

Did you have happen to have Sync setup?

> I looked at the history and I used it only to go to see one webpage

Which page was this? Was there any streaming media? 

Also, tip for the future: In Android you can set your limit as you did, but if you head into the advanced settings there you can toggle to disable cell data when your limit is met.
Wow, this is horrible and should never happen. Can you answer a few questions to help us figure it out?

1. Did you have Firefox Sync setup? If so, was it recent, or has it been working fine for a few day?
2. Can you remember what webpages might have been open? I'm wondering if the page(s) could have been updating dynamically in the background.

Thanks for any info you can give.
also, were any add-ons installed?
Hello,

thank you to look at it.

I do not have sync setup on my phone.

The webpage I looked in the morning was:

http://www.lefigaro.fr/le-scan-sport/2014/06/07/27001-20140607ARTWWW00098-fernandao-meurt-dans-un-accident-d-helicoptere.php

this is the only webpage I can see in my history for the day.
As for the add-ons I had:

- open link in zombie tab : 0.0.6
- https-everywhere : 3.5android.0
- ghostery : 5.2.1
- adblock plus : 2.6.3

All of them I downloaded from mozilla and used them for months without any problem.
Thanks. We're inspecting the network requests from that site to see if something is dumb going on.

Do you happen to have Flash for Android installed too?
No I do not have flash installed.
I tried loading the page with a mobile user-agent in my desktop browser and a few times I saw an ad request for a video/flv and a number of requests tallying up to ~15MB but nothing crazy. I wonder if it's possible that Gecko kept sending requests here for some reason? Maybe an onStop() was never called?
tracking-fennec: --- → ?
Posted image Sample data usage details (obsolete) —
Does your version of Android allow you to click on applications for more information in the Usage screen? For example, the details screen on my Nexus 4 (attached) shows both "Foreground" and "Background" usage, which could help us better determine which part of the browser was using the data.
Yes, most of the data came from background. See attaachment.
Posted image data usage
tracking-fennec: ? → +
Status: UNCONFIRMED → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1024599
I disagree that this is a duplicate - bug 1024599 talks about data usage of a tab (foreground data) but, according to the image in comment 11, this is 1.38GB of background data usage and only 14MB of foreground usage.
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: DUPLICATE → ---
(In reply to Michael Comella (:mcomella) from comment #13)
> I disagree that this is a duplicate - bug 1024599 talks about data usage of
> a tab (foreground data) but, according to the image in comment 11, this is
> 1.38GB of background data usage and only 14MB of foreground usage.

Then bug 1024599 needs to be more explicit. In triage today, the group decided that this bug can not be fixed as a one-off issue. The only way to keep this from happening again is to add some monitoring and a kill switch.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1024599
I am sorry but I don't think that this bug is solved. I am completely aware that it is probably really difficult to solve it with so little information but I am pretty sure that if this behavior rehappen and even become generalized the firefox brand will suffer a lot. As you can imagine, this bug cost me a little bit of money, I was lucky in a sense because it happen in my country and not when I was travelling. I was keeping it mainly to help the developpers to isolate the problem if possible but since it seems that it is not consider as a problem I will take no chance and I will remove it completely.
I agree that this bug is not solved, but there is nothing actionable here that we can do to diagnose the issue or resolve this particular bug other than attempting to reproduce the issue. The fact of the matter is that we have no monitoring system in-place to help detect and prevent issues like this. It's unfortunate, but that is why we opened a few bugs listed above. What we can try and do here is try to replicate the issue with what information we have. 

In the case here, you provided a possible troublesome URL. I tried leaving the URL up with Firefox overnight with my device idle, plugged in, on Wi-Fi and my background data usage for Firefox never exceeded 10MB by simply loading that site. It's possible that I didn't receive the same ad-content as you did or the same resources delivered that you did.

What we should be doing is reporting on examples where data usage begins to spike, hook Gecko up to the network monitor in the remote debugger and look at what's being requested and why.
(In reply to Mark Finkle (:mfinkle) from comment #14)
> Then bug 1024599 needs to be more explicit. In triage today, the group
> decided that this bug can not be fixed as a one-off issue. The only way to
> keep this from happening again is to add some monitoring and a kill switch.

What do you mean by more explicit? Bug 1024599 is a separate problem - limiting foreground data usage does not solve this background data problem. If this bug is going to be closed, it should be WONTFIX'd.

Though we can do more than add a monitoring service: we can audit all of our background services that touch the network for obvious bugs (e.g. HealthReportUploadService, Telemetry, etc.), or perhaps provide an option to allow background services to upload only on WiFi (with a prompt on first run?). In my opinion, there is some worthwhile investigation to be done here.

Nicolas, in Settings -> Mozilla, under the Data Choices header, what options do you have enabled?
Status: RESOLVED → REOPENED
Flags: needinfo?(nicolas.gruel)
Resolution: DUPLICATE → ---
There is no way for us to get the data from the past. The bugs mentioned in above are research based on this bug but unless we fix those and the user has this happen agian there is nothing that will move this bug to a fixed state. It is horible that this happened! Keeping an unactionable bug open is not serving any use either.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → INCOMPLETE
(In reply to Michael Comella (:mcomella) from comment #17)

> What do you mean by more explicit? Bug 1024599 is a separate problem -
> limiting foreground data usage does not solve this background data problem.
> If this bug is going to be closed, it should be WONTFIX'd.

Bug 1024599 is really about background browser tab usage and I added a comment + changed the summary to reflect that.

> Though we can do more than add a monitoring service: we can audit all of our
> background services that touch the network for obvious bugs (e.g.
> HealthReportUploadService, Telemetry, etc.), or perhaps provide an option to
> allow background services to upload only on WiFi (with a prompt on first
> run?). In my opinion, there is some worthwhile investigation to be done here.

That is a great idea. Can you file a bug to audit background services?
 
> Nicolas, in Settings -> Mozilla, under the Data Choices header, what options
> do you have enabled?

This would be great information to have.
Nicolas, in Settings -> Mozilla, under the Data Choices header, what options do you have enabled?

I have nothing. No telemetry, no crash reporter and no firefox health problem. I am turning them on now but since I will use firefox just seldomly now that will not be a great use.
Flags: needinfo?(nicolas.gruel)
Nicolas, what version of Firefox were you running? You can find this in Settings -> Mozilla -> About Firefox, just under the Firefox title.
(In reply to Mark Finkle (:mfinkle) from comment #19)
> That is a great idea. Can you file a bug to audit background services?

bug 1026779.
Are we clear how Android divides foreground and background data?

I would have assumed that "background" = "activity is not in foreground".

If anyone has an alternative perspective, it'd be great to get that written down so we can look at things from the right direction.


Without Sync set up (Comment 4), assuming release channel (=> no updater), and FHR etc. disabled (Comment 20), that points the finger at Gecko, rather than what we think of as "background services".

(We're quite confident that disabling the data reporting items means they do no work.)

Assuming that "background" just means "the activity wasn't foregrounded", this data usage could be:

* ABP updates.
* Ghostery tracker updates.
* Safebrowsing fetches.
* Add-on update checks.

One hypothesis is that overnight Gecko was finally alive long enough to run a bunch of pending tasks.

Another, of course, is that the page itself was doing lots of work, perhaps due to an error in the page JS. 

1.4GB over 8 hours is about 55KB/sec, and the graph implies approximately linear usage, then a stop. That would rule out most of our background services, but is within the realm of possibility for a page.
The version what it is now in my phone is 30.0.
(In reply to Richard Newman [:rnewman] from comment #23)
> Are we clear how Android divides foreground and background data?
> 
> I would have assumed that "background" = "activity is not in foreground".

Android lets you disable background data on apps so, since I assume they'd use equivalent terminology here, we can disable background data and test which functionality is disabled.
(In reply to Michael Comella (:mcomella) from comment #25)
> we can disable background data and test
> which functionality is disabled.

http://i.imgur.com/NEYryXa.gif
This is not resolved: http://imgur.com/zzzK8Wb
I am going back to Chrome, it never did anything like that. I was not downloading anything, just browsing random websites...
(In reply to rkj from comment #27)
> This is not resolved: http://imgur.com/zzzK8Wb
> I am going back to Chrome, it never did anything like that. I was not
> downloading anything, just browsing random websites...

Do you have Sync/Firefox Account set up?

Do you know which sites you were browsing?

Did you have any add-ons installed?
Flags: needinfo?(rkj)
I have Firefox sync enabled on rkj@go2.pl, no addons as far as I know.
Website on those days (from history): 
dictionary.reference.com/browse/event
en.m.wikipedia.org/wiki/Bahrain
en.m.wikipedia.org/wiki/Mystery_Spot
m.wikihow.com/Pronounce-Buenos-Aires
money.usnews.com/money/personal-finance/mutual-funds/articles/2012/06/18/some-401k-plans-let-you-take-the-wheelif-you-dare
myhealthonline.sutterhealth.org/
www.azlyrics.com/lyrics/ledzeppelin/stairwaytoheaven.html
www.berkeley.edu/news/berkeleyan/1998/0909/spot.html
www.foxbusiness.com/personal-finance/2011/09/23/pros-and-cons-self-directed-401k/
www.google.com/flights/
www.google.com/flights/gwsredirect
www.google.com/search
www.grammarbank.com/question-tags.html
www.howjsay.com/mobile.php
www.moddb.com/mods/stargatespaceconflict/news/full-interactive-campaign-announced
www.pcgamer.com/fantasy-ground-steam-dungeons-dragons/
www.pcgamer.com/gaming-mouse-myths-busted/
www.pcgamer.com/the-best-gaming-mouse/
www.sandlotscience.com/MysterySpots/Mystery_Spots_1.htm
Flags: needinfo?(rkj)
This exact thing has just happened to me - 1.01GB of background data in only 1 day. I certainly will not me using firefox on android again and you should really try to fix this issue - it cost me €15 and I do not have that money right now!
This is shitty, we should do something about it. Some ways to move forward:
  1) comment 25: figure out what would be considered "background data" by Android so we know which services to look at
  2) Audit our background services (it'd help to do 1 first)
  3) Add telemetry probes to a) know if this is a widespread problem and, if possible, b) identify how much data we use when running each service.
Status: RESOLVED → REOPENED
tracking-fennec: + → ?
Resolution: INCOMPLETE → ---
re comment 31, note comment 23 which largely rules out our background services.

Bob, a few questions to help us debug this issue:
  * Which version of Android are you running? See 3-dot menu -> Settings -> Mozilla -> About Nightly, it should be listed under the "Firefox" name (likely version 40-something)
  * Do you have Firefox Sync or Firefox Accounts enabled?
  * Do you have any addons installed? If so, which ones?
  * Under 3-dot menu -> Settings -> Mozilla, which items do you have checked under "Data choices"?
  * Were you using Firefox during the day when this happened? If so, which pages were you browsing?
Flags: needinfo?(vetatemypet)
Let's make sure this is still an issue. Has this happened again recently? Maybe we should file a separate bug about setting more measurement tools in place to understand what's going on if this happens again?
tracking-fennec: ? → -
(In reply to :Margaret Leibovic from comment #33)
> Let's make sure this is still an issue. Has this happened again recently?
> Maybe we should file a separate bug about setting more measurement tools in
> place to understand what's going on if this happens again?

Sorry, I missed comment 30! Let's fix this.
tracking-fennec: - → +
Margaret, can we get an assignee here?
Flags: needinfo?(margaret.leibovic)
What are we fixing here? We don't know how the data is being used. Websites could easily be consuming the data, could be sync could be something else.
(In reply to Kevin Brosnan [:kbrosnan] from comment #36)
> What are we fixing here? We don't know how the data is being used. Websites
> could easily be consuming the data, could be sync could be something else.

Let's talk to the network folks and see if we can track the data packets in some sort of stats rollup. If so, we could use that and the "application-foreground" / "application-background" notifications to send Telemetry histograms on the amount of networking in foreground and background.
If we can determine that Gecko is doing network while Firefox is in the background, we could try to shutdown the connections after a quota. Android/Java networking might already do something like that - limiting data while in background or screen off.
Finkle, re bug 1212466 comment 7, did you try that approach and check your data use? I wonder if they are related.
Flags: needinfo?(mark.finkle)
(In reply to Michael Comella (:mcomella) from comment #39)
> Finkle, re bug 1212466 comment 7, did you try that approach and check your
> data use? I wonder if they are related.

I did not, but I am running the same test suing the latest Nightly (without SSDP triggering in background) and will check. This might be related to bug 974207 as well.
Flags: needinfo?(mark.finkle)
Assignee: nobody → michael.l.comella
Flags: needinfo?(margaret.leibovic)
Is there any update on this bug?  I just got hit with 2Gb of background data usage over the last 48 hours.  I am uninstalling Firefox.  Also, I don't think its at all related to bug 974207 mentioned above.  Until you figure out what is wrong with Firefox, I think you need to put in some hard limits to shut it down if background data usage ever exceeds foreground data usage.  This is affecting people's pocket book.
(In reply to brian.m.kessler from comment #41)
> Is there any update on this bug?  I just got hit with 2Gb of background data
> usage over the last 48 hours.  I am uninstalling Firefox.  Also, I don't
> think its at all related to bug 974207 mentioned above.  Until you figure
> out what is wrong with Firefox, I think you need to put in some hard limits
> to shut it down if background data usage ever exceeds foreground data usage.
> This is affecting people's pocket book.

Thanks for your report, we definitely care about fixing this. We need to figure out what this network activity actually is.

Are you using the current release version of Firefox? Have you opted in to sending telemetry data? Bug 1249373 is filed about that issue in particular.
Flags: needinfo?(brian.m.kessler)
Also do you have sync set up? If so, do you have a large number of bookmarks?
Hi all,

I experienced the same problem today where 150MB were used in background without doing nothing.

After a little research I found this post. What is strange is that the only thing I have been looking at was also lefigaro.fr (http://www.lefigaro.fr/flash-actu/) so it has to be related...

See a screenshot of my data consumption : http://postimg.org/image/duhz2yu0f/

Anyway, it's a pretty bad surprise as I now went over my limit and I'll have to pay for that.

Even if we are able to define that lefigaro.fr is the troublemaker, it's not really normal that so much data can be used in the background.

Thanks for your time.
And on my side, I don't have sync set up.
Thanks for the datapoint, Flo.

Sounds like Comment 37/Comment 38 are the place to start, particularly checking lefigaro.fr.
tracking-fennec: + → ?
OS: Linux → Android
Hardware: x86_64 → All
Summary: firefox use 1.4Gb of data when doing nothing → Firefox (Gecko) uses large amounts of data when backgrounded with certain pages open
(In reply to :Margaret Leibovic from comment #43)
> Also do you have sync set up? If so, do you have a large number of bookmarks?

No sync or bookmarks.
Flags: needinfo?(brian.m.kessler)
So http://www.lefigaro.fr/flash-actu/ is an application that provides up to date news. Translated it is 'News Flash'. On first load it consumed ~5 to ~7  MB. The site has a meta refresh header, every 15 min it will repull the entire site. Every 30ish seconds it will ping a tracking gif (0.04 KB) and js (0.02 KB). This seems to be a case that the site is not designed for mobile users.
The background refresh issue will be fixed with bug 518805
Depends on: 518805
I'm not actively working on this. I expect it to get assigned during Thursday triage.
Assignee: michael.l.comella → nobody
This bug has a lot of comments and it's not clear what action can be taken to resolve this.

I filed bug 1253346 as a new meta bug to track our efforts to reduce background data, and we should file smaller bugs that block that when actionable items appear.

If anyone in this bug has testcases of websites that are causing problems, please file a new bug detailing that testcase.
Status: REOPENED → RESOLVED
tracking-fennec: ? → -
Closed: 5 years ago3 years ago
Resolution: --- → INCOMPLETE
The specific case mentioned in comment 44 and possibly comment 2 should be resolved in Firefox 47 by bug 518805.

If you have come to this bug because of data usage please file a new bug using this template https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox%20for%20Android&component=General&blocked=1253346 with your best recollection of what you did (what sites were open) before this occurred.
Resolution: INCOMPLETE → FIXED
Stale request
Flags: needinfo?(vetatemypet)
You need to log in before you can comment on or make changes to this bug.