1064171 - Instrumentation for new default browser prompt

Status: NEW

(Firefox :: General, defect)

Description

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

In Fx34 we introduced a notification bar that replaces the modal dialog asking for Firefox to be the default browser.

We need to closely watch the conversion rates of both approaches. I'm not sure if we have the tooling to do that, hence this bug.

Comment 1

[:Gavin Sharp [email: [e-mail]]]

We need more definition of what exactly we want to measure, and then reconcile that with what we can do with Telemetry/FHR. Who can help flesh that out?

Comment 2

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

Here's my take:

We definitely want to know:
- How many users have Fx as their default before the change
- How many users have Fx as their default after the change
I think we are already measuring this.

Apart from that it would be really interesting (in no particular order):
- How many people use the »Don't check on startup« option (both before and after the change)
- How long is the notification bar visible before users act on it? Perhaps even multiple sessions?
- Is there any correlation between the time before the user acts on the notification bar and the choices they make?
- Any differences in conversion between fresh installs and older profiles (both before and after the change)?

Comment 3

[Winston Bowden]

Hi there - 

Checking on where we are with planning measurement for the notification bar.  +1 to Philipp's suggestions. I'll add one that Kevin suggested.  

- When users set Firefox as their default, do we see a substantial increase in usage?

Comment 4

[Blake Winton (:bwinton) (:☕️)]

(Cc-ing Saptarshi, both for confirmation of my claims below, and because he's probably the person who will be running those reports. ;)

My understanding is that FHR logs whether Firefox is the default browser, and so we will be able to tell whether this is different before and after the change.  Similarly, we should be able to tell if having Firefox as the default affects usage both now and after the change.

In the really interesting section, I think FHR can tell us if there's a difference in conversion between fresh installs and older profiles, both before and after the change.

For the rest of the really interesting questions from Philipp, I believe we'll need to add (probably UITelemetry) probes, and if we wanted to compare the before and the after, we would need to uplift those into Beta.

Comment 5

["Saptarshi Guha[:joy]"]

> 
> In the really interesting section, I think FHR can tell us if there's a
> difference in conversion between fresh installs and older profiles, both
> before and after the change.

So this will be monitored Fx 34 - Beta? Is this feature present on the very first build of Fx 34 beta and all locales?

Comment 6

[(Currently slow to respond) Philipp Sackl [:phlsa] (Firefox UX) please use needinfo]

(In reply to "Saptarshi Guha[:joy]" from comment #5)
> > 
> > In the really interesting section, I think FHR can tell us if there's a
> > difference in conversion between fresh installs and older profiles, both
> > before and after the change.
> 
> So this will be monitored Fx 34 - Beta? Is this feature present on the very
> first build of Fx 34 beta and all locales?

I'm not sure about the locales, but the feature is in Aurora right now, so it should be present in 34 Beta from day one.