1842082 - Add a metric that counts home screen views that are not in PBM

Status: VERIFIED FIXED

(Fenix :: Experimentation and Telemetry, enhancement, P1)

Description

[mbowerman]

Description & Product Manager / Data Scientist User Story

We currently have the home_screen_view_count metric, which includes both PBM and normal home screen views. We need a metric that does not include PBM home screen views in order to count the home screen sessions where sponsored tiles should be displayed.

Hypothesis

What questions will you answer with this data?

What percent of eligible sponsored tiles slots are actually filled with tiles?

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

This will be used to calculate sponsored tile fill rate, which is a metric included as a contractual obligation with our ads partners.

Comment 1

[Joe M [:jmahon]]

Harrison, as you begin to dive into this, can you also let us know how much time you expect it to involve? There's a desire to get this into 116, which would mean it'd have to be landed on main (then tested, then uplifted to the 116 beta branch) before the last beta happens on July 20th.

What are the chances of that being possible?

Comment 2

[[:harrisono]]

I am very confident we can have this merged into main and uplifted to Beta next week.
What should we use as this new metric?
Does home_screen.normal_home_screen_view_count work?

Comment 3

[Joe M [:jmahon]]

I'm a little hesitant about the term "normal" (just because, without context, it could perhaps mean a number of different things). But I don't actually have a better idea off the top of my head - happy to defer to Michael or any of our other Android engineers (e.g. perhaps there's already a go-to term for the non-PBM homepage state).

Comment 4

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1842082 - Normal home screen view count telemetry (#2841)

Comment 5

[[:harrisono]]

I have a PR up, we can change the nomenclature of this metric as needed.

Comment 6

[mbowerman]

Yeah, I tried to give the name some thought, too, and came up with nothing better than normal. Maybe 'non_pbm', but that seems clunkier to me.

Comment 7

[[:harrisono]]

maybe "standard"?

Comment 8

[mbowerman]

I agree, that sounds better. "normal" makes me think of sessions free of errors or something, but "standard" doesn't.

Comment 9

[[:harrisono]]

ok changed to home_screen.standard_homepage_view_count because standard_home_screen_view_count was too long.

Comment 10

[BMO Github Automation]

Authored by https://github.com/HarrisonOg
https://github.com/mozilla-mobile/firefox-android/commit/04574fb825e75d0cd5b7df85565356a4d51cc573
[main] Bug 1842082 - Standard home screen view count telemetry

Comment 11

[Andrei Mocirean]

Verified as fixed on the latest Nightly build (117.0a1) from 17th of July 2023.
Devices used for testing: Google Pixel 7 Pro (Android 14).

Thanks!

Comment 12

[Joe M [:jmahon]]

Harrison, would this be safe to uplift to the beta branch? If so, could you start that process & submit the request?

Comment 13

[[:harrisono]]

yeah I'll get started on the uplift process.

Comment 14

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1842082 - Normal home screen view count telemetry (backport #2841) (#2914)

Comment 15

[[:harrisono]]

Backport PR created.

Comment 16

[[:harrisono]]

Comment on attachment 9344706 [details] [review]
[mozilla-mobile/firefox-android] Bug 1842082 - Normal home screen view count telemetry (backport #2841) (#2914)

Beta/Release Uplift Approval Request

• User impact if declined: None, impact will be on the data team-side.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is adding an additional metric to track standard home screen views. Not risky as there is no behavior change for users.
• String changes made/needed:
• Is Android affected?: Yes

Comment 17

[Dianna Smith [:diannaS] - on Leave, back on August 9th]

Comment on attachment 9344706 [details] [review]
[mozilla-mobile/firefox-android] Bug 1842082 - Normal home screen view count telemetry (backport #2841) (#2914)

Approved for Mobile 116.0b8

Comment 18

[BMO Github Automation]

Authored by https://github.com/mergify[bot]
https://github.com/mozilla-mobile/firefox-android/commit/9651a490a86a3296f12592b9b9d730a5111b693c
[releases_v116] Bug 1842082 - Standard home screen view count telemetry (#2914)

Comment 19

[Andrei Mocirean]

Verified as fixed on the latest Beta build (116.0b8).
Thanks!