968793 - [Cost Control] Data Usage is broken

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Cost Control, defect)

Description

[Flaburgan]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 (Beta/Release)
Build ID: 20131206145143

Steps to reproduce:

This is a regression on Keon using 1.3.0.0-prerelease image by Geeksphone, updated today.

Launch the data usage app, with data connection enable or not.


Actual results:

A message is displayed:
"Data usage
Loading interface data.
Please retry in a while"

There is only one button: ok. Press it third times and the message will disappears, but the app will stay empty (no graph).

Often after that a pop up at the bottom of the screen indicates only "crashed".


Expected results:

The app should launch normally.

Comment 1

[Jason Smith [:jsmith]]

Can someone check if this reproduces on Buri?

Comment 2

[Preeti Raghunath(:Preeti)]

Naoki,

Please help review.

Comment 3

[Peter Bylenga [:PBylenga]]

This does not reproduce on today's v1.3 Buri, user is brought immediately to graph and data page populated with expected content.

v1.3 Environmental Variables:
Device: Buri v1.3 MOZ
BuildID: 20140206004002
Gaia: 467ef8c9145d9a57d35b0619db541d23b522b958
Gecko: a1fa925c40c2
Version: 28.0
Firmware Version: v1.2-device.cfg


This sounds like bug 942060 which was verified fixed before the build id of Comment 0.  Reporter stated he updated his Keon today but shows a build ID from 12-06-2013, perhaps a data migration issue?

Comment 4

[Flaburgan]

The build ID at the beginning of my message is the one of my Firefox desktop which I use to go on bugzilla, not the one of my phone :P

My build ID is: 20140206005931

Comment 5

[Jason Smith [:jsmith]]

Not a blocker if it's keon-specific.

Comment 6

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

It takes a long time; having said that I do see the data using today's build.

Did you go from 1.4 build to 1.3 build?  or a 1.2 to 1.3?
I'm guessing the issue you are seeing is incompatible data.

If you went from 1.4 (nightly) to 1.3 build then that's not supported so you will see an issue.
If you went from 1.2 to 1.3; then that's a separate issue.

Comment 7

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

To note, I was also looking at keon and the keon build.

Comment 8

[Flaburgan]

I upgraded from 1.2 to 1.3.

Btw, when I open the notification panel, at the bottom of the screen just before the WiFi, data connection, bluetooth, flight mode and settings shortcuts, there is a message (instead of the data usage) saying "No sim card available"...

I'll try again with this night build.

Comment 9

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Could you try backing up your data, and then try resetting the device?

To back up your profile : https://wiki.mozilla.org/B2G/QA/Tips_And_Tricks#Backing_up_the_Profile_and_Restoring_it_after_flash

resetting the device : 
1. go to settings -> device information -> more information -> reset phone

And see if that works please?  If that works then I think it's a upgrade issue and I'll investigate that angle.

My concern is that the data/databases did not upgrade/migrate correctly.

Comment 10

[Jason Smith [:jsmith]]

I'm adding qawanted to see if we can reproduce this bug by doing an upgrade from 1.2 --> 1.3 using the Leo device.

Comment 11

[Flaburgan]

After a wipe data + wipe cache from the boot menu, everything is fine. So it's a migration issue, nice guess Naoki!

Comment 12

[Peter Bylenga [:PBylenga]]

I did not reproduce this bug upgrading Leo v1.2 to v1.3, here are my environmental details after the update:

v1.3 (OTA) Environmental Variables:
Device: Leo v1.3 (OTA) MOZ
BuildID: 20140207004002
Gaia: 539a25e1887b902b8b25038c547048e691bd97f6
Gecko: 1592ce49929c
Version: 26.0
Firmware Version: V10d

Comment 13

[Flaburgan]

Just for information, I upgraded my Keon from 1.0 to 1.1 to 1.2 to 1.3. The app stopped working with the 1.3, but I don't know when the migration issue occurred.

Comment 14

[Flaburgan]

Btw, when I enable and then disable the flight mode, the indicator in the notification view still indicates "No sim card available" but I guess this is a different issue.

Comment 15

[Vlado Valastiak [:wladow]]

Confirmed the behaviour described in comment 0 on Keon with the latest nightly 1.3 GP build (upgraded from 1.2)

Is there a way to wipe out cost-control related data without touching anything else?

Comment 16

[Marina Rodríguez [:mai]]

Hi, 
would you mind attach the logs? I need them to help me for diagnose the problem. 

Regards

Comment 17

[Flaburgan]

@marina, now that I wiped my phone, I do not reproduce anymore. I guess this message was for Vlado?

Comment 18

[Vlado Valastiak [:wladow]]

(In reply to marina rodríguez [:mai] from comment #16)
> Hi, 
> would you mind attach the logs? I need them to help me for diagnose the
> problem. 
> 
> Regards

sure, no problem. Is adb logcat sufficient enough? Or anything more specific? Thanks.

Comment 19

[Marina Rodríguez [:mai]]

The logcat is enough for me, thanks

Comment 20

[Vlado Valastiak [:wladow]]

Attachment: logcat.txt

logcat output. Sorry it took a bit longer. Hope this helps

Comment 21

[Albert [:albert]]

There is a problem in database upgrade from 1.2 to 1.3

Database for stats in 1.2 is called 'net_stats_v2' but in 1.3 the upgrade tries to update 'net_stats' objectstore. So the database is not created and costcontrol can't retrieve data because database does not exist:

E/GeckoConsole( 2523): [JavaScript Error: "NotFoundError: The operation failed because the requested database object could not be found. For example, an object store did not exist but was being opened." {file: "resource://gre/modules/NetworkStatsDB.jsm" line: 82}]

Comment 22

[Noemí Freire (:noemi)]

Per comment 21, any OTA process will be affected and CC app is broken under that scenario so asking for 1.3?.

Comment 23

[Albert [:albert]]

Attachment: Patch for 1.3

Added a check of objectstore name before delete the objecstore

Comment 24

[Albert [:albert]]

Attachment: Patch

I don't know if we need the patch for 1.4. Can users update from 1.2 to 1.4? If no, I think we don't need this patch.

Comment 25

[Preeti Raghunath(:Preeti)]

Data migration issue

Comment 26

[John Shih]

Comment on attachment 8375437 [details] [diff] [review]
Patch

Review of attachment 8375437 [details] [diff] [review]:
-----------------------------------------------------------------

If a user updated from v1.2 to v1.3 and countered the problem, which meant that the upgradeSchema is already be triggered. In the case I think this patch can not solve the problem, unless the database is cleared and goes through the update process again. I'm not sure if we can have such mechanism? (something like check if the store name is available, if not, reset the database)

::: dom/network/src/NetworkStatsDB.jsm
@@ +83,5 @@
> +        // In version 1.2 objectStore name was 'net_stats_v2', to avoid errors when
> +        // upgrading from 1.2 to 1.3 objectStore name should be checked.
> +        let stores = db.objectStoreNames;
> +        if(stores.contains("net_stats_v2")) {
> +          db.deleteObjectStore("net_stats_v2");

let's have
const DEPRECATED_STORE_NAME_V2 = "net_stats_v2";
or you can have a better naming ;)

Comment 27

[John Shih]

(In reply to John Shih from comment #26)
> Comment on attachment 8375437 [details] [diff] [review]
> Patch
> 
> Review of attachment 8375437 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> If a user updated from v1.2 to v1.3 and countered the problem, which meant
> that the upgradeSchema is already be triggered. In the case I think this
> patch can not solve the problem, unless the database is cleared and goes
> through the update process again. I'm not sure if we can have such
> mechanism? (something like check if the store name is available, if not,
> reset the database)
> 

We can have a new DB_VERSION, and make the check of store name in it.
if found store name is old, go through the update process again.
You can find the reference in MobileMessageDB.jsm [1].
Note that, this may cause dependency on Bug 965319.

[1] https://bugzilla.mozilla.org/page.cgi?id=splinter.html&bug=960741&attachment=8362363

> ::: dom/network/src/NetworkStatsDB.jsm
> @@ +83,5 @@
> > +        // In version 1.2 objectStore name was 'net_stats_v2', to avoid errors when
> > +        // upgrading from 1.2 to 1.3 objectStore name should be checked.
> > +        let stores = db.objectStoreNames;
> > +        if(stores.contains("net_stats_v2")) {
> > +          db.deleteObjectStore("net_stats_v2");
> 
> let's have
> const DEPRECATED_STORE_NAME_V2 = "net_stats_v2";
> or you can have a better naming ;)

Comment 28

[John Shih]

Comment on attachment 8375436 [details] [diff] [review]
Patch for 1.3

Review of attachment 8375436 [details] [diff] [review]:
-----------------------------------------------------------------

please see my comments in another patch!

Comment 29

[John Shih]

Comment on attachment 8375437 [details] [diff] [review]
Patch

Review of attachment 8375437 [details] [diff] [review]:
-----------------------------------------------------------------

After having offline discussion with Gene, we found that the way in this patch may be the only solution to reduce possible risk. Because js error would cause code stuck in the line, so it doesn't work even we add a new updateSchema.

Comment 30

[Albert [:albert]]

Comment on attachment 8375436 [details] [diff] [review]
Patch for 1.3

NOTE: This flag is now for security issues only. Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 
User impact if declined: costcontrol not working for users updating from 1.2 to 1.3
Testing completed: yes
Risk to taking this patch (and alternatives if risky): low
String or UUID changes made by this patch: none

Comment 31

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/b2g-inbound/rev/1f21bc9fc654

Comment 32

[Ryan VanderMeulen [:RyanVM]]

Backed out for xpcshell failures.
https://hg.mozilla.org/integration/b2g-inbound/rev/205763e922a1

https://tbpl.mozilla.org/php/getParsedLog.php?id=34695552&tree=B2g-Inbound

Comment 33

[Ryan VanderMeulen [:RyanVM]]

And mochitest failures.
https://tbpl.mozilla.org/php/getParsedLog.php?id=34696935&tree=B2g-Inbound

Comment 34

[bhavana bajaj [:bajaj]]

 NI on :albert to see why the tests are crying..

Comment 35

[Albert [:albert]]

Attachment: Patch

Fixed wrong objecstore name

Comment 36

[Albert [:albert]]

Try server: https://tbpl.mozilla.org/?tree=Try&rev=a74cf728b645

Comment 37

[Gene Lian (I already quit Mozilla)]

Comment on attachment 8376687 [details] [diff] [review]
Patch

Review of attachment 8376687 [details] [diff] [review]:
-----------------------------------------------------------------

Please watch out if we would have the naming issue for v1.3.

Comment 38

[Albert [:albert]]

(In reply to Gene Lian [:gene] (needinfo? encouraged) from comment #37)
> Comment on attachment 8376687 [details] [diff] [review]
> Patch
> 
> Review of attachment 8376687 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Please watch out if we would have the naming issue for v1.3.

1.3 does not have the naming issue, I first worked with 1.3 and then rebased the patch for 1.4, this is the reason of that problem in 1.4, different naming in each version.

Comment 39

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/b2g-inbound/rev/3d0a2dd0b438

Comment 40

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/3d0a2dd0b438

Comment 41

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/3d3021fdb54f

Comment 42

[Flaburgan]

Hm, I'm sorry but I now have this behavior again: black screen when I try to launch the app, 0.00 octets used in the notification panel.
I have this with a 1.3 updated today on a Keon, and which was flashed / wiped after the migration from 1.2 (see above) so it's not a migration 1.2 -> 1.3 issue.

Comment 43

[Albert [:albert]]

(In reply to Flaburgan from comment #42)
> Hm, I'm sorry but I now have this behavior again: black screen when I try to
> launch the app, 0.00 octets used in the notification panel.
> I have this with a 1.3 updated today on a Keon, and which was flashed /
> wiped after the migration from 1.2 (see above) so it's not a migration 1.2
> -> 1.3 issue.

Can't reproduce it wiht buri, can you attach your logcat output?

Comment 44

[Flaburgan]

Iḿ at work currently, Iḿ going to do that this evening.

Comment 45

[Flaburgan]

Attachment: Logcat keon 1.3

Comment 46

[Albert [:albert]]

I don't see errors related with the gecko's NetworkStats API. Just to clarify, what command are you using to flash? Are you flashing the geeksphone build from http://downloads.geeksphone.com/?

Comment 47

[Flaburgan]

Yes, I flashed it a little more than one month ago. Since that moment, I update it with the daily update. Do you want me to flash it with ./flash.sh once again?

Comment 48

[Noemí Freire (:noemi)]

Attachment: DataUsage_app_images-keon-v1.3-2014-02-27.Gecko-461476c.Gaia-b04d2a8

Hi,

We've just tested a keon device (updated with the latest v1.3 build available (http://downloads.geeksphone.com/keon/v1.3/latest_v1.3.html):
* v1.3
*Gecko:461476c
*Gaia: b04d2a8
*BuildId: 20140227010308
*Platform version: 28.0

and it works fine, data usage app can be executed as expected.

Note: option 2 selected during the update process
Do you want to keep your user data? (Some users has problems in first reboot, if you have, please reflash and select not to keep the data)
1) Yes
2) No

Comment 49

[Flaburgan]

Just after the wipe it worked fine too, I don't know exactly when it stopped working again... I have other issue when I turn on flight mode, maybe you can check with your just flashed phone if turning on then off the flight mode and then launching data usage works?

Comment 50

[Noemí Freire (:noemi)]

(In reply to Flaburgan from comment #49)
> Just after the wipe it worked fine too, I don't know exactly when it stopped
> working again... I have other issue when I turn on flight mode, maybe you
> can check with your just flashed phone if turning on then off the flight
> mode and then launching data usage works?

Hi,

Regarding Turning on/off flight mode it works as expected, you are able to access to data usage app once turning off flight mode. It was an issue already solved, see Bug 970323.

Comment 51

[Flaburgan]

So what should I do? Flash my phone again removing all the data, and then wait for the bug to reappear? This would probably not be different than the state we are now...

Comment 52

[Noemí Freire (:noemi)]

(In reply to Flaburgan from comment #51)
> So what should I do? Flash my phone again removing all the data, and then
> wait for the bug to reappear? This would probably not be different than the
> state we are now...

Hi,

I'd suggest to open new bugs and include logcat traces to be analyzed. Thanks!

Comment 53

[Flaburgan]

I posted the logcat above two days ago and it looks like it's not helpful :(
I'd like to know if I am the only one with this issue or if someone else with a Keon reproduce it.

Comment 54

[Noemí Freire (:noemi)]

In the logcat file provided in comment 45 there is no Cost Control trace error either gaia or gecko. 
On the other hand, as mentioned, we are not able to reproduce with the keon sample we have (comment 48), also working on Hamachi device. Maybe someone else with a keon could check it and give some feedback.
We'd suggest to remove the data base (see steps below *) and try again. In case the issue persists or appears again open a new bug including logcat traces.
*steps to remove the data base:
adb shell rm -r data/local/storage/persistent/chrome/idb/3249156127nsetta* && adb reboot