1823492 - Add timespan telemetry for Adjust OnAttributionChangedListener

Status: VERIFIED FIXED

(Firefox for Android :: Experimentation and Telemetry, task)

Description

[James Hugman [:jhugman] [@jhugman]]

This task is to add a timespan metric to the AdjustMetricsService to measure how long it takes to get attribution data from the Play Store app.

Comment 1

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1823492 Add timing metric for Adjust Attribution Listener (#1341)

Comment 2

[BMO Github Automation]

Authored by https://github.com/jhugman
https://github.com/mozilla-mobile/firefox-android/commit/f7e7fe1650f5e3142a700d4b54f0ad9320dd4a65
[main] Bug 1823492 Add timing metric for Adjust Attribution Listener

Comment 3

[Delia Pop (Android QA)]

Hello, :jhugman , is there something the QA team can check here? Thanks!

Comment 4

[Joe M [:jmahon]]

Just a quick note in case it's helpful for interpreting the results of this telemetry, I found an old issue in Adjust's Github project that has a pretty well-detailed outline of the performance expectations (at that point in time): https://github.com/adjust/android_sdk/issues/377

Comment 5

[James Hugman [:jhugman] [@jhugman]]

I found an old issue in Adjust's Github project that has a pretty well-detailed outline of the performance expectations (at that point in time): https://github.com/adjust/android_sdk/issues/377

Interesting, thank you. That post indicates a network roundtrip, which is… disappointing.

Comment 6

[Delia Pop (Android QA)]

:jhugman confirmed on Slack that this ticket does not require testing from the QA team. Marking the ticket as verified.

Comment 7

[James Hugman [:jhugman] [@jhugman]]

https://mozilla-hub.atlassian.net/browse/EXP-3323?focusedCommentId=669289

https://sql.telemetry.mozilla.org/queries/91292?p_days=30#226013 (thank you :tlong of the Glean team) shows— for the beta and nightly populations at least— that the median response time for is around 3.5–4.0s, and the p75 time is around 6 seconds.

This may change when the probe hits the release population.

This may affect product experiments for onboarding, but not messaging.

Comment 8

[Joe M [:jmahon]]

I never said thanks over here - thanks! This info is super useful. I'm going to clear my needinfo because I think the natural continuation of this ticket is over in Bug 1828972, which starts the next logical line of questioning: "can we improve that response time by migrating to using the Google Play Referrer Library directly, instead of the Adjust SDK?".

Comment 9

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1823492 - Add timespan metric for Adjust attribution (#2974)

Comment 10

[BMO Github Automation]

Authored by https://github.com/rocketsroger
https://github.com/mozilla-mobile/firefox-android/commit/385295d1e938301af6115d036641ce7e11d27874
[main] Bug 1823492 - Add timespan metric for Adjust attribution

Comment 11

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1823492 - Add timespan metric for Adjust attribution (backport #2974) (#2982)

Comment 12

[Roger Yang [:royang]]

Comment on attachment 9345501 [details] [review]
[mozilla-mobile/firefox-android] Bug 1823492 - Add timespan metric for Adjust attribution (backport #2974) (#2982)

Beta/Release Uplift Approval Request

• User impact if declined: No good time measurement for Adjust attribution for first session user
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Telemetry only, just an updated measurement for Adjust attribution
• String changes made/needed:
• Is Android affected?: Yes

Comment 13

[Roger Yang [:royang]]

Comment on attachment 9345501 [details] [review]
[mozilla-mobile/firefox-android] Bug 1823492 - Add timespan metric for Adjust attribution (backport #2974) (#2982)

Beta/Release Uplift Approval Request

• User impact if declined:
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky):
• String changes made/needed:
• Is Android affected?: Yes

Comment 14

[Delia Pop (Android QA)]

Verified fixed. The "first_session.adjust_attribution_time" metric was captured on the latest Nightly 117.0a1. Tested with Samsung Galaxy A53 5G (Android 13).
Nightly:
"first_session.adjust_attribution_time": {
"values": {
"1392470868": 1,
"1518500249": 0
},

More details on Glean: https://debug-ping-preview.firebaseapp.com/pings/test-00001/ff3c38ef-cff1-46ff-9e1b-776ef3e5cbf2

Please note that the same metric was also available in the latest Beta 116.0b8 build:
"first_session.adjust_attribution_time": {
"values": {
"2341847523": 1,
"2553802833": 0
},
"sum": 2482772191

More details on Glean: https://debug-ping-preview.firebaseapp.com/pings/test-000001/378e4e18-4298-485a-a379-5631105a4938#L481

Comment 15

[BMO Github Automation]

Authored by https://github.com/mergify[bot]
https://github.com/mozilla-mobile/firefox-android/commit/e04f66f342fcb646d063539f6f5875864647c873
[releases_v116] Bug 1823492 - Add timespan metric for Adjust attribution (#2982)

Comment 16

[Dianna Smith [:diannaS]]

Comment on attachment 9345501 [details] [review]
[mozilla-mobile/firefox-android] Bug 1823492 - Add timespan metric for Adjust attribution (backport #2974) (#2982)

Approved for Mobile 116.0 RC2

Comment 17

[Delia Pop (Android QA)]

Verified fixed on RC 116.0 build 2 with Samsung Galaxy A53 5G (Android 13).
"first_session.adjust_attribution_time": {
"values": {
"1969251187": 1,
"2147483648": 0
},

More details on Glean: https://debug-ping-preview.firebaseapp.com/pings/qa-delia/bbf8a8c8-8b50-49fc-94b9-c4d327027f29#L334