1534451 - Send Mobile Activation Ping telemetry in Fennec

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Metrics, enhancement, P1)

Description

[Chris Peterson [:cpeterson]]

Fennec telemetry shows 25% more installs than the Google Play Store does (bug 1483927). Frank suggests (in bug 1501822) that we include a hash of the user's Google Ad ID in install/uninstall telemetry to count unique users and avoid double-counting reinstalls.

Google Ad ID can be reset by users. Google's policy requirement is that all apps use this ID in lieu of any others for advertising. We are not using this for advertising so that policy should not apply to us.

Fennec Mobile Activation Ping spec:
https://docs.google.com/document/d/1D-0m3TRvIAOzGtkBNjaddyVr7baE7uEhiBUFyVQr2pg/edit

Problem description:
https://docs.google.com/document/d/11PzvbJz832A9kUeBj6E3n9He25feBAYGLWDxNybhfaY/edit

Mobile data review: bug 1508839

Comment 1

[Chris Peterson [:cpeterson]]

[geckoview:fenix:m4] because Amit says we want the new Mobile Activation Pings in place before Fenix MVP.

Comment 2

[Frank Bertsch [:frank]]

Is this bug covering both the Fennec and Fenix implementations?

For the latter, if we use Glean, then that may simplify some work for the developers. We can then use the "Ping Format" section of the doc as a guideline for which metrics to implement.

Comment 3

[Chris Peterson [:cpeterson]]

(In reply to Frank Bertsch [:frank] from comment #2)

Is this bug covering both the Fennec and Fenix implementations?

Yes, unless engineering investigation shows that we need separate implementations for Fennec and GeckoView (for Fenix, Firefox Reality, and other GeckoView-powered apps).

Comment 4

[Chris Peterson [:cpeterson]]

(In reply to Chris Peterson [:cpeterson] from comment #3)

(In reply to Frank Bertsch [:frank] from comment #2)

Is this bug covering both the Fennec and Fenix implementations?

Yes, unless engineering investigation shows that we need separate implementations for Fennec and GeckoView (for Fenix, Firefox Reality, and other GeckoView-powered apps).

snorp says that we should split Fennec and Fenix into separate bugs. The Fennec fix will be in mozilla-central while the Fenix fix should use the Glean Android-Component. I don't know if any code can be shared between Fennec and Fenix.

I filed a Fenix issue to add the Mobile Activation Ping: https://github.com/mozilla-mobile/fenix/issues/1028

Comment 5

[Chris Peterson [:cpeterson]]

This is a Fennec issue. Fenix will have a separate implementation: https://github.com/mozilla-mobile/fenix/issues/1028

Comment 6

[Vlad Baicu]

Assigining myself to this in Andrei's stead as he will be in PTO next week.

Comment 7

[Devin Reams]

Dexter or Frank, to help test and confirm the Telemetry pings are returning as expected, is there an existing query or dashboard or approach you can share to help us?

Comment 8

[Alessio Placitelli [:Dexter]]

(In reply to Devin Reams (dreams) from comment #7)

Dexter or Frank, to help test and confirm the Telemetry pings are returning as expected, is there an existing query or dashboard or approach you can share to help us?

Redirecting the question to Frank, as I'm not sure how things work for Fennec.

Comment 9

[Vlad Baicu]

Attachment: Bug 1534451 - Send Mobile Activation Telemetry ping. r=JanH

Comment 10

[Vlad Baicu]

FYI After we land and test that the ping works as intended we should also file a new bug to add documentation. (e.g. https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/telemetry/pingbuilders/TelemetryCorePingBuilder.java#46)

Comment 11

[Jeff Klukas [:klukas] (UTC-4)]

:frank is out this week, so I'm filling in to help get this unblocked.

I have a PR open with the JSON schema for the new mobile/activation ping: https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/331

Once that schema is merged and deployed, the pipeline to accept these pings and we can start testing that payloads are making it through.

I've left some commentary on there to explain some differences from the structure proposed in Google Doc spec and ask some questions.

Comment 12

[Jeff Klukas [:klukas] (UTC-4)]

(In reply to Devin Reams (dreams) from comment #7)

Dexter or Frank, to help test and confirm the Telemetry pings are returning as expected, is there an existing query or dashboard or approach you can share to help us?

Once the schema is deployed and some pings are flowing in, I can help with debugging to see if these are correctly flowing through the pipeline, and I can provide some sample queries for you to do ongoing validation.

The new schema will get automatically deployed to the new GCP pipeline, allowing the data to flow into a BigQuery table. If needed, we can add a Parquet schema and deploy this to the existing AWS pipeline.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/91bc05242c6f
Send Mobile Activation Telemetry ping. r=JanH

Comment 14

[Andrei Ciure[:aciure]]

Backed out changeset 91bc05242c6f (bug 1534451) for causing AdvertisingUtil.java build bustages

push that caused the backout: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=91bc05242c6f02a5d30c71557693129303e7067e

failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&fromchange=7d59f5e3db1b3f0cbd4b095752535c2d5ef09208&selectedJob=246238117&searchStr=android%2C4.0%2Capi16%2B%2Cdebug%2Cbuild-android-api-16%2Fdebug%2C%28b%29

https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&fromchange=7d59f5e3db1b3f0cbd4b095752535c2d5ef09208&selectedJob=246238114&searchStr=android%2C4.0%2Capi16%2B%2Copt%2Candroid%2Cgradle%2Ctests%2Cbuild-android-checkstyle%2Fopt%2Ca%28checkstyle%29

backout: https://hg.mozilla.org/integration/autoland/rev/d6e21ffd8ecceee564afa0c7c8b1bcf61fbd6aff

Comment 15

[Vlad Baicu]

Fixed bustage issues and updated my diff, please try again.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6ff9d6783ae0
Send Mobile Activation Telemetry ping. r=JanH

Comment 17

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/6ff9d6783ae0

Comment 18

[Cristian Brindusan [:cbrindusan]]

Backed out changeset 6ff9d6783ae0 (Bug 1534451) for causing build bustages in AdvertisingUtil.java:6

Backout: https://hg.mozilla.org/integration/autoland/rev/db85e84612070c8cd9c75a7cc1b0a50bb5ed2d38

Push that started the failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&selectedJob=246480003&revision=f17162f343b657c6ac2f57937ff49a1840465a6b

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=246479960&repo=mozilla-central&lineNumber=3975

Comment 19

[Vlad Baicu]

(In reply to Cristian Brindusan [:cbrindusan] from comment #18)

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=246479960&repo=mozilla-central&lineNumber=3975

Apparently the AdvertisingUtil.java wasn't being excluded from the sourcesets, only from the tests which led to the bustage on the-without-gps-build from try.

Made the change, updated the diff and I've tried submitting multiple other try jobs to confirm that the issue is fixed but with no success altough I am quite certain the issue is now fixed. Let's give it another go

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/74252063fc9e
Send Mobile Activation Telemetry ping. r=JanH

Comment 21

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/74252063fc9e

Comment 22

[Georg Fritzsche [:gfritzsche]]

@liuche, just to double-check:
I know the main data review on this approach happened in bug 1508839, but does this bug need a separate light data review for the implementation?

Comment 23

[Chenxia Liu [:liuche]]

Thanks for the ping to check this data review request.

One last thing that I'd like to see is a (short) documentation file, probably mobile/android/docs/activationping.rst, similar to https://searchfox.org/mozilla-central/source/mobile/android/docs/uitelemetry.rst .
This would be a short description of what TelemetryActivationPingBuilder is sending, what the identifier is (and what values it could be, e.g., clientid in Case X or salted GAID in Case Y), and a sample ping.

Mainly, summarizing what the code in https://searchfox.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/telemetry/pingbuilders/TelemetryActivationPingBuilder.java and https://searchfox.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/telemetry/TelemetryActivationPingDelegate.java#79 are doing in a human-readable form (for documentation reasons, but also for whatever data science teams will be looking at this ping).

Vlad is that something that you could put together? Either in this bug, or a new bug linked to this one. Other than that this looks good to me, thanks for the work here!

Comment 24

[Vlad Baicu]

Sure, I'll file a new bug.

However, I think we should also have a webpage doc e.g. https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/core-ping.html