Closed Bug 1013373 Opened 7 years ago Closed 6 years ago

[ZTE Open C v1.2] - If an app is left open data and battery are used up

Categories

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P2)

x86
macOS
defect

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: guigs, Unassigned)

Details

(Keywords: perf, power, Whiteboard: [c=power p= u= s=][Power])

Description: A user has reported that if they add a sim card to their device and leave the email app running on automatically sync, it eventually kills the battery and uses all the data.
https://support.mozilla.org/en-US/questions/1000085

There may be testing on this already, but is there a timeout or check on the device that may warn the user after inactivity that this can happen to them? Or maybe a recommended usage we can add to the knowledge base?
Whiteboard: zteopenc
Can we do some testing around what's described above to see if we can reproduce the high battery usage problem?
Component: Gaia::System → Gaia::E-Mail
Keywords: perf, power, qawanted
The email app tends to fail inactive.  If we're busy we'll usually get ourselves killed.  The stupid spinny refresh thing will keep spinning, but that shouldn't use power if the email app is in the background.  (At least I hope!  Man, that would be embarassing.)  The thing about email breaking is probably that periodic sync greatly increases the chance we'll break and fail to close ourselves and then by definition when you next go into the email app, we will be broken.

Network-wise, FxOS already tracks the activity of all apps in http://dxr.mozilla.org/mozilla-central/source/dom/network/src/NetworkStatsDB.jsm but does not currently expose any UI for the data that I'm aware of.  Of course, the trick is that presumably if the stats app doesn't know about the data usage, then the stats DB probably doesn't know about it either.  But there could be bugs *anywhere* in the process.

We could have the reporter pull the IndexedDB database for their network stats and provide it to us and we could see if there's anyone to blame.

My process for doing this on my trunkish buri was to:
- "adb shell ls /data/local/storage/persistent/chrome/idb/"
...
3249156127nsetta_ts
3249156127nsetta_ts.sqlite
3249156127nsetta_ts.sqlite-journal
...

- I looked for the anagram of netstats which is that one above.  It looks like our name generating algorithm is consistent because I see the same file-names on my v1.3T tarako that I see on my trunk-ish buri.

- In some directory I want the files:
  - "adb pull /data/local/storage/persistent/chrome/idb/3249156127nsetta_ts.sqlite"
  - "adb pull /data/local/storage/persistent/chrome/idb/3249156127nsetta_ts.sqlite-journal"
  - that thing without a file extension is a directory that holds blobs.  This API does not use blobs, so you don't need to try and pull it/its contents, but you will want to "mkdir 3249156127nsetta_ts" locally to make indexeddb happy possibly.

- to figure out the app id mapping without capturing other data:
  - "adb shell ls /data/local/storage/persistent"
  - note: one would want to potentially censor URLs of installed apps... we only need to know the ID's of apps we want to accuse.  And even then the reporter could actually not tell us the id's and we could just tell them what app seems to have used up all the data.

I use my updated version of :bent's IndexedDB browser to quickly look into the db (https://github.com/asutherland/idbbrowser/tree/update-paths), but you could also do other things to look at it, I suppose.
- Go to File, go to "open", browse to the place the two files are stored.
- Expand the "net_stats" database that got added
- go down to net_stats_store and then "network" if you care about wi-fi versus cellular.  Look at "serviceType" if you don't care about cellular/wi-fi. (I think).
(In reply to Andrew Sutherland (:asuth) from comment #2)
> I use my updated version of :bent's IndexedDB browser to quickly look into
> the db (https://github.com/asutherland/idbbrowser/tree/update-paths), but
> you could also do other things to look at it, I suppose.

And by this I mean that you want to download and install https://github.com/asutherland/idbbrowser/raw/update-paths/idbbrowser.xpi for it to work.  The version on AMO does not work with recent versions of Firefox.
Andrew, can you comment on the email app's behavior on network usage and power usage.  With regards to power, please make sure that the email app doesn't hold any wake locks directly or indirectly when it isn't actively doing anything.  So between the polls, please make sure you release handles on any resources that can keep a wake lock around (e.g. sockets).

We have tools for measuring power usage: http://mzl.la/1fKE3jy  and here's a post on idle wake ups and wake locks: http://bit.ly/1sVmmy9

Please contact :jhylands to get access to the power hardware.  There is hardware out in the wild that you can probably track down and borrow.
Flags: needinfo?(bugmail)
Priority: -- → P2
Whiteboard: zteopenc → [c=power p= u= s=]
As indicated in comment 2, I don't think the email app did this.  We will take the wifi wake lock when performing periodic sync, but we cap that timeout at 45 seconds.  Taking it for the full 45 seconds every time will add up if the sync interval is 5 minutes (unknown in this case), but per the user, something was using a ton of data.  That was almost certainly not email since our failure mode in this case is very likely for us to stop generating network traffic entirely.  gmail is accessed via TCPSocket which has the network stat tracking.

And then mainly in the support request is the line: "I can't say for sure which app is the culprit but it may be Grooveshark."  Since grooveshark is a music streaming service and they may not have optimized their app for Firefox OS, I could see it being grooveshark's fault, although I have not done the legwork to find out what grooveshark would be doing that could bypass the tracking stats.

I'd left this bug open in case the reporter could pull their netstats DB so I could better determine who was using up the bandwidth.
Flags: needinfo?(bugmail)
Keywords: qawanted
This seem still to be an issue on 1.3.0.0 using a FFOX_EU_EBAY_OPENCV1.0.0B04
Email app used all my data. It only stopped when I changed the setting wrt to checking for new mail to 'manually'.
I pulled the 3249156127nsetta_ts.sqlite from the device but I cannot find the mentioned net_stats_store. Would you like me to provide the file to you via email? Does it contain any personal information that would have to be scrubbed first?

This is what it says in the database table:
"net_stats"	"5"

This is from object_store:
"2"	"0"	"net_stats"	",appId,network,timestamp"
"3"	"1"	"net_alarm"	"id"

The rest sees to reference blobs? Is this the right file?
Whiteboard: [c=power p= u= s=] → [c=power p= u= s=][Power]
asuth, is this bug still relevant?
Flags: needinfo?(bugmail)
(In reply to Nicholas Nethercote [:njn] from comment #7)
> asuth, is this bug still relevant?

I don't believe so.  If this wasn't grooveshark, some device(s) also shipped with some type of messaging app that was extremely wasteful with alarms/etc., and we definitely haven't seen many bug reports about email doing things like this.  Plus we added various failsafes including automated connection closing, etc.
Status: NEW → RESOLVED
Closed: 6 years ago
Flags: needinfo?(bugmail)
Resolution: --- → WORKSFORME
My comment still stands. I did not install Grooveshark nor did I install any messaging apps that could be likely culprits. I'm open to supplying you with any data you might need to investigate this issue.

However, this might be resolved in newer versions. I haven't used the phone since July but Wikipedia tells me that there was a new release in August, i.e. 1.4

My bug report was still on version 1.3
Flags: needinfo?(bugmail)
I have the ability to build a battery harness for the ZTE Open, so I can possibly (with build assistance) test power consumption patches.
As far as I know, the latest version of B2G that works on the original ZTE Open is v2.1s (v2.2 and v2.2r don't compile, and master seems to create a system image that's too big to be flashed onto the phone), so it might be too late for that.
(In reply to David Treumann from comment #9)
> My comment still stands. I did not install Grooveshark nor did I install any
> messaging apps that could be likely culprits. I'm open to supplying you with
> any data you might need to investigate this issue.

For grooveshark I was referring to the support thread at https://support.mozilla.org/en-US/questions/1000085 that spawned this bug where grooveshark was explicitly namechecked as a potential cause.  Also some devices actually shipped with a buggy/aggressive messaging app on them; users didn't have to install it.  But almost certainly not the ebay device.

> However, this might be resolved in newer versions. I haven't used the phone
> since July but Wikipedia tells me that there was a new release in August,
> i.e. 1.4
> 
> My bug report was still on version 1.3

I very much appreciate your willingness to help track this down further!  There have been a number of other enhancements made to the sync process through version 2.2 that mean reproducing on an old version is unlikely to be helpful.  (Specifically, we replaced many of the underlying email libraries in v2.1 and timezone-related mismatches with internaldate that could cause churn were fixed in v2.2 with bug 886534, plus some bisection related fixes when there are a lot of messages.)

Having said that, knowing more about your email scenario can likely help me test/repdroduce locally so this is not a problem going forward.  If you could elaborate on:
- How much data are we talking about getting used up?  100 Megs, 2 gigs, etc.?
- What periodic sync setting you had enabled.  (5 minutes?)
- What server(s) you were using?  Your email address you are using for bugzilla is gmail, was it just a gmail account?  (I'm interested in gmail versus dovecot versus courier, etc.  Just knowing the server type is enough, or if you don't know the server type, I can probe the server given just the mail server's domain.)
- Along those lines, what account types were you using?  By default, gmail uses IMAP, but many people tried to manually use ActiveSync with gmail.
- What timezone were you in from the perspective of the device and of the mail server?  (Especially if you use gmail, they have a timezone setting in their web UI at least for calendaring.)
- About how many new messages would you get each day in your inbox?
- Do you practice inbox zero or otherwise aggressively archive/delete incoming messages?
- When you'd open the email app and you had periodic sync enabled, would the new messages already be there, or did it seem like email was only fetching them now that you opened it?
- Were there other apps on the phone that you might be using while email was trying to background sync/etc. FxOS will kill apps when there's not enough memory.  If we were looking at a pathological case where email would get killed during periodic sync, that could explain a lot.  But if you only ever used the phone for email, that's also very useful to know.
- Did you have any problems with the phone's screen turning on in your pocket, etc.  (The email app does more work if it thinks you are looking at it.)

Thanks for any information you can provide!
Flags: needinfo?(bugmail) → needinfo?(d.treumann)
(In reply to Andrew Sutherland [:asuth] from comment #12)
> I very much appreciate your willingness to help track this down further! 
> There have been a number of other enhancements made to the sync process
> through version 2.2 that mean reproducing on an old version is unlikely to
> be helpful.  (Specifically, we replaced many of the underlying email
> libraries in v2.1 and timezone-related mismatches with internaldate that
> could cause churn were fixed in v2.2 with bug 886534, plus some bisection
> related fixes when there are a lot of messages.)
> 
Good to know that things are improving! I'm looking forward to try new versions.

> Having said that, knowing more about your email scenario can likely help me
> test/repdroduce locally so this is not a problem going forward.  If you
> could elaborate on:
> - How much data are we talking about getting used up?  100 Megs, 2 gigs,
> etc.?
300 Mb over two or three days which is all the data I got for one month on my contract.
> - What periodic sync setting you had enabled.  (5 minutes?)
Every 5 Minutes
> - What server(s) you were using?  Your email address you are using for
imap.googlemail.com:993
> bugzilla is gmail, was it just a gmail account?  (I'm interested in gmail
> versus dovecot versus courier, etc.  Just knowing the server type is enough,
> or if you don't know the server type, I can probe the server given just the
> mail server's domain.)
Yes, my gmail account.
> - Along those lines, what account types were you using?  By default, gmail
> uses IMAP, but many people tried to manually use ActiveSync with gmail.
Account type: IMAP+SMTP
> - What timezone were you in from the perspective of the device and of the
> mail server?  (Especially if you use gmail, they have a timezone setting in
> their web UI at least for calendaring.)
I'm in Germany so CEST but I have not set up the calendar on the ZTE Open C.
> - About how many new messages would you get each day in your inbox?
Maybe about 20.
> - Do you practice inbox zero or otherwise aggressively archive/delete
> incoming messages?
My inbox is never emptied. I mostly depend on search to find relevant emails.
> - When you'd open the email app and you had periodic sync enabled, would the
> new messages already be there, or did it seem like email was only fetching
> them now that you opened it?
I do not recall.
> - Were there other apps on the phone that you might be using while email was
> trying to background sync/etc. FxOS will kill apps when there's not enough
> memory.  If we were looking at a pathological case where email would get
> killed during periodic sync, that could explain a lot.  But if you only ever
> used the phone for email, that's also very useful to know.
I do not know which apps that might be. I installed FoxScanner, Cartes, The Next Is. I think that is about it.
> - Did you have any problems with the phone's screen turning on in your
> pocket, etc.  (The email app does more work if it thinks you are looking at
> it.)
Not as far as I know.
> 
> Thanks for any information you can provide!
Flags: needinfo?(d.treumann)
Thank you very much for the extra details!  I believe I've identified the likely explanation for your bandwidth usage problems and they are indeed fixed on trunk.

I've filed bug 1216329 for us to have automated network-usage testing/tracking against for our new converstions ("convoy") branch, which I believe is the best way for us to make sure your problem does not recur in the future while also helping us better understand and keep on top of our network use.
I've also filed bug 1216347 now about having the email app intentionally track its own data usage as part of periodic sync and potentially enforce some type of data budget with explicit user action taken if the email app exceeds its own budget, etc.
You need to log in before you can comment on or make changes to this bug.