[Metrics] Usage time is not tracked for apps from the Marketplace

VERIFIED FIXED in 2.2 S9 (3apr)

Status

defect
VERIFIED FIXED
4 years ago
4 years ago

People

(Reporter: shinglyu, Assigned: marshall)

Tracking

unspecified
2.2 S9 (3apr)
ARM
Gonk (Firefox OS)
Dependency tree / graph

Firefox Tracking Flags

(blocking-b2g:2.2+, b2g-v2.2 verified, b2g-master verified)

Details

(Whiteboard: [dependency: marketplace][ft:media])

Attachments

(10 attachments)

Reporter

Description

4 years ago
*** Description
The usage time is always 0 for apps installed from the Marketplace

*** Steps to Reproduce
1. Install an app from the Firefox Marketplace (e.g. La Republica)
2. Open the app after install (using the Marketplace "open" button)
3. Use it for 5 sec.
4. Press the home button to go back to homescreen
5. Uninstall the app
5. Wait for the app usage ping to reach the server

*** Expected Results
The usage data for the app should be:
   "usageTime": 5,
   "installs": 1,
   "uninstalls": 1,

*** Actual Results
The usage time is 0
   "usageTime": 0, //something wrong here
   "installs": 1,
   "uninstalls": 1,

*** Other Notes


*** Reproduction Frequency
100

*** Build
  Gaia-Rev        8e98fe665f3821d10d4d982cbb14cbe5b94d0be5
  Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/62c80c92b39e
  Build-ID        20150215162503
  Version         37.0a2
  Device-Name     flame
  FW-Release      4.4.2
  FW-Incremental  65
  FW-Date         Mon Dec 15 18:51:29 CST 2014
  Bootloader      L1TC000118D0
Reporter

Updated

4 years ago
Flags: needinfo?(marshall)
Looking into this today
Flags: needinfo?(marshall)

Updated

4 years ago
Assignee: nobody → marshall
blocking-b2g: --- → 2.2?
Whiteboard: [dependency: marketplace]
Whiteboard: [dependency: marketplace] → [dependency: marketplace][ft:media]

Comment 2

4 years ago
Marshall, any update?

Thanks
Hema
Flags: needinfo?(marshall)

Comment 3

4 years ago
Per Marshall's update on 3/9:

He was unable to reproduce it last week. He says he will try reproducing again now that his laptop is back from repairs.

Shing: please retry again on 2.2 to see if you are able to reproduce this. Some of the bugs related to metrics landed in 2.2 last week. 

Thanks
Hema
Flags: needinfo?(slyu)
blocking-b2g: 2.2? → 2.2+
Reporter

Comment 4

4 years ago
I'll test again today
Flags: needinfo?(slyu)
Reporter

Comment 5

4 years ago
Posted file adb.log
Reporter

Comment 6

4 years ago
Posted file ping.txt
Reporter

Comment 7

4 years ago
I am able to reproduce this on today's 2.2 

Steps:
1. Open Marketplace, find ConnectA2 app
2. Install the app, tap "open" after the "install" button changed to "open"
3. Wait for a few seconds
4. Open the app again
5. Ping transmitted

You can see adb.log for the adb logcat output, and ping.txt for the ping payload. Notice that the usageTime=0.

Version
  Gaia-Rev        4aefc3f6f30a40ac67fdf841b7c90cd648b85369
  Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/3db27b3c9298
  Build-ID        20150312162502
  Version         37.0
  Device-Name     flame
  FW-Release      4.4.2
  FW-Incremental  eng.cltbld.20150310.200728
  FW-Date         Tue Mar 10 20:07:39 EDT 2015
  Bootloader      L1TC000118D0

Updated

4 years ago
Target Milestone: --- → 2.2 S8 (20mar)
Thanks Shing, I'll try to repro again today
Flags: needinfo?(marshall)
I think I was previously looking at only install counts (which are working), not app usage time. I've been able to repro this now with ConnectA2, and am investigating
It looks like |e.detail.installOrigin| is undefined for the appopened event, but set properly to https://marketplace.firefox.com for applicationinstall and applicationuninstall events.

What isn't clear to me ATM is why installOrigin would be left undefined for appopened events. It could be a bug, but maybe there is a privacy reason of some kind? Tagging Fabrice here to see if he has any ideas.

Assuming that the behavior is not a bug, I think I have a course of action, but want to air the idea for sanity:

Add an additional check for the app's manifestURL with a prefix of "https://marketplace.firefox.com/app/". Do any apps on Marketplace have non-Marketplace URLs? Here's ConnectA2's manifestURL for example:

https://marketplace.firefox.com/app/956189a1-c985-4824-85a9-de708d38d6f9/manifest.webapp

I'm also going to test to see if the re-querying mozApps.mgmt gives me a proper installOrigin. If it does, that would be another path forward (though it would introduce some async behavior, and be pretty risky)
Flags: needinfo?(fabrice)
Reporter

Comment 11

4 years ago
(In reply to Marshall Culpepper [:marshall_law] from comment #10)

> Do any apps on Marketplace have
> non-Marketplace URLs? 

I do have seen marketplace apps with their own domain name.
I don't know what fires the `appopened` event, but it's not gecko. So it's likely that gaia is not building the event detail correctly.
Flags: needinfo?(fabrice)
Reporter

Comment 13

4 years ago
FC is coming soon (4/6), we need to be quick.
I've followed the trail of events back to webapps-launch, and it looks Gecko is only passing along the timestamp, url, and manifestURL to Gaia. I have a hacked up solution working locally by reusing the actual app object from the 'applications' cache. It isn't perfect, but it's not as impactful as changing Gecko directly. I'll have a patch up soon.
Comment on attachment 8581763 [details] [review]
[gaia] marshall:bug1137063_trackMarketplaceApps > mozilla-b2g:master

This patch works around the current appopened event behavior by pulling from the Gaia system app 'applications' cache. An arguably more proper fix would be to pass along |app.installOrigin| from Gecko, then copy that into the appopened event in |browser_config_helper.js|, but the impact of those changes seem riskier to me.
Attachment #8581763 - Flags: review?(dflanagan)
Attachment #8581763 - Flags: feedback?(thills)
Comment on attachment 8581763 [details] [review]
[gaia] marshall:bug1137063_trackMarketplaceApps > mozilla-b2g:master

This looks okay to me.

Longer term, it would be good to stop using the variable name 'app' for these objects that we now discover are not actually apps but just objects with manifestURL properties.  Better to just take the manifestURL from the event and use that. If we'd done that in the first place then this bug would never have occurred. 

But given that this is 2.2+, this fix seems good enough as it is.
Attachment #8581763 - Flags: review?(dflanagan) → review+
Comment on attachment 8581763 [details] [review]
[gaia] marshall:bug1137063_trackMarketplaceApps > mozilla-b2g:master

Hi Marshall,

Sorry for the delay.  It looks good to me.  It seems like the better choice for a fix.  I did test out and verify as well.

Thanks,

-tamara
Attachment #8581763 - Flags: feedback?(thills) → feedback+
Comment on attachment 8585600 [details] [review]
[gaia] marshall:bug1137063_trackMarketplaceApps_v2.2 > mozilla-b2g:v2.2

Fix tracking of apps installed from the Marketplace

[Bug caused by] (feature/regressing bug #): Bug 1119009

[User impact] if declined: We will not have usage time for apps installed from marketplace

[Testing completed]: Tested w/ ConnectA2 app from the marketplace

[Risk to taking this patch] (and alternatives if risky): Adds a new step in the AUM bootup process, possibly slowing down startup of the service

[String changes made]: None
Attachment #8585600 - Flags: approval-gaia-v2.2?(bbajaj)
Attachment #8585600 - Flags: approval-gaia-v2.2?(bbajaj) → approval-gaia-v2.2+
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: 2.2 S8 (20mar) → 2.2 S9 (3apr)
Reporter

Updated

4 years ago
Keywords: verifyme
Hi Shing,
   
    Could you help to provide the ways to get "ping.txt"(usage time)?

Thank you very much.
Flags: needinfo?(slyu)
Reporter

Comment 24

4 years ago
Shally, please follow the steps here: https://wiki.mozilla.org/B2G/QA/Ping_Testing#Metrics_Testing_for_FxOS_2.2

Let me know if any step is unclear.
Flags: needinfo?(slyu)
Hi Shing,

    According to the steps of Comment 0 and Comment 24, I have verified on latest Flame v2.2.The data of "usageTime" & "installs" & "uninstalls" as shown below,could you help to confirm the below results? 

Thank you very much.

-------------------------------------------------------------------------------
Actual results:
Message 1 content: 
{"usageTime": 0,"invocations": 0,"installs": 1,"uninstalls": 0,"activities": {}}

Message 2 content: 
{"usageTime": 0,"invocations": 0,"installs": 0,"uninstalls": 1,"activities": {}
"usageTime": 68,"invocations": 1,"installs": 0,"uninstalls": 0,"activities": {}

Please refer attachment:ping_1.txt and ping_2.txt.

--------------------------------------------------------------------------------
Device: Flame 2.2 build
Build ID               20150407162504
Gaia Revision          ea735c21bfb0d78333213ff0376fce1eac89ead6
Gaia Date              2015-04-07 20:58:15
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/3f86ddb7f719
Gecko Version          37.0
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150407.195227
Firmware Date          Tue Apr  7 19:52:39 EDT 2015
Bootloader             L1TC000118D0
Flags: needinfo?(slyu)
Reporter

Comment 28

4 years ago
Shally,
This is wierd. I can see the usageTime > 0 after this patch landed. An example would be like this:
"
    "https://marketplace.firefox.com/app/bb680dfa-2ac1-45e1-b71b-3db4520c8ca8/manifest.webapp": {
        "20150408": {
            "activities": {},
            "installs": 1,
            "invocations": 1,
            "uninstalls": 0,
            "usageTime": 7
        }
    }
"
Have you opened the app after installation?

It's verified on 
  Gaia-Rev        ea735c21bfb0d78333213ff0376fce1eac89ead6
  Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/245f37f44017
  Build-ID        20150407002501
  Version         37.0
  Device-Name     hammerhead
  FW-Release      5.0
  FW-Incremental  eng.cltbld.20150407.040951
  FW-Date         Tue Apr  7 04:10:07 EDT 2015
  Bootloader      HHZ12d
Flags: needinfo?(slyu) → needinfo?(lixia)
Hi Shing,
 
    I receive several usage ping messages on the server from Flame v2.2,but the data of "installs" and "usageTime" is not in the same message and is different with Comment 28. 
    After 5 times attempt,it still has some problems about Usage time. Could you help to verify this bug?

Thank you.
-----------------------------------------------------------------------------
Please see attachments: verify_v2.2.MP4,ping_v2.2_1.txt and ping_v2.2_2.txt.

STR:
1.Install "La Republica" app from the Marketplace.
2.Open the app after install (using the Marketplace "Open" button)
3.Use it for >5 sec.
4.Press the home button to go back to homescreen.
5.Uninstall the app.
6.Wait for the app usage ping messages to reach the server.
**Actual Results:
-->Message 1:
"usageTime": 0,
"invocations": 0,
"installs": 1,
"uninstalls": 0,
"activities".

-->Message 2:
{"start": 
1428651821943,
"apps": 
{"app://verticalhome.gaiamobile.org/manifest.webapp": 
{"20150410": 
{"usageTime": 80,
"invocations": 3,
"installs": 0,
"uninstalls": 0,
"activities": {
}}},"https://marketplace.firefox.com/app/54f24b41-0bfd-4942-8651-6f397fb5614d/manifest.webapp": {"20150410": 
{"usageTime": 0,
"invocations": 0,
"installs": 0,
"uninstalls": 1,
"activities": {}}

-----------------------------------------------------------------------------------
Device: Flame 2.2 build
Build ID               20150409162502
Gaia Revision          df0e04acad7c8c993f6ffe07b0ccb0ec20ee50bb
Gaia Date              2015-04-09 22:35:05
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/091b1cc1240b
Gecko Version          37.0
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150409.200526
Firmware Date          Thu Apr  9 20:05:37 EDT 2015
Bootloader             L1TC000118D0
Flags: needinfo?(lixia) → needinfo?(slyu)
Reporter

Comment 33

4 years ago
Shally, 

I encountered that sometimes when the ping was transmitted right after the installation. Please check the pings between the first one ("installs:1") and the last one ("uninstalls:1), if any one of that has usageTime > 0 for the "https://marketplace.firefox.com/app/54f24b41-0bfd-4942-8651-6f397fb5614d/manifest.webapp" app, then the test passes.
Flags: needinfo?(slyu) → needinfo?(lixia)
I will try to verify this bug tomorrow.
This bug has been successfully verified on latest Nightly Flame v2.2.
See attachment: verified_v2.2.mp4
Reproduce rate: 0/5

Actual results:
"https://marketplace.firefox.com/app/54f24b41-0bfd-4942-8651-6f397fb5614d/manifest.webapp": {"20150414": {
"usageTime": 0,
"invocations": 0,
"installs": 1,
"uninstalls": 0,
"activities": {}
------------------------------------------------------------------
"https://marketplace.firefox.com/app/54f24b41-0bfd-4942-8651-6f397fb5614d/manifest.webapp": {"20150414": {
"usageTime": 62,
"invocations": 1,
"installs": 0,
"uninstalls": 1,
"activities": {}

------------------------------------------------------------------------
Device: Flame 2.2 user build (Pass)
Build ID               20150413002502
Gaia Revision          73645b097720f3ca594a14d288b87d3885d7fc9d
Gaia Date              2015-04-13 19:30:36
Gecko Revision         https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/5005522fd68e
Gecko Version          37.0
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150413.040508
Firmware Date          Mon Apr 13 04:05:20 EDT 2015
Bootloader             L1TC000118D0
-------------------------------------------------
I will verify Flame v3.0 later.
Flags: needinfo?(lixia)
This bug has been successfully verified on latest Nightly Flame v3.0 (The usageTime >0).
Reproduce rate: 0/5

STR:
1. Install La Republica from Marketplace.
2. Open the app(using the Marketplace "Open" button)
3. Use the app for >5 sec.
4. Long press Home button to close the app.
5. Go to homescreen,uninstall the app
6. Wait for the app usage ping to reach the server.
**Actual results:
"https://marketplace.firefox.com/app/54f24b41-0bfd-4942-8651-6f397fb5614d/manifest.webapp": {"20150414": {
"usageTime": 0,
"invocations": 0,
"installs": 1,
"uninstalls": 0,
"activities": {}
}
---------------------------------------------------
"https://marketplace.firefox.com/app/54f24b41-0bfd-4942-8651-6f397fb5614d/manifest.webapp": {"20150414": {
"usageTime": 7,
"invocations": 1,
"installs": 0,
"uninstalls": 1,
"activities": {}
}
-------------------------------------------------------------------------
Device: Flame 3.0 user build (Pass)
Build ID               20150412160203
Gaia Revision          c8cb0c0ebb8dd1f5c0c9037e38f8e4b237beb77b
Gaia Date              2015-04-13 18:03:34
Gecko Revision         https://hg.mozilla.org/mozilla-central/rev/0a46652bd992
Gecko Version          40.0a1
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150412.192808
Firmware Date          Sun Apr 12 19:28:21 EDT 2015
Bootloader             L1TC000118D0
Status: RESOLVED → VERIFIED
Keywords: verifyme
QA Whiteboard: [MGSEI-Triage+]
You need to log in before you can comment on or make changes to this bug.