1824034 - explicitly count migrations started by about:welcome, rather than relying on newtab annotation

Status: NEW

(Firefox :: Messaging System, enhancement, P2)

Description

[Dan Mosedale (:dmosedale, :dmose)]

Najla and Ed were interested in telemetry that could answer the question "of the X% of users who had import checked in the about:welcome screen that started the migration, what percentage actually imported data".

A straightforward way to make this possible would be to allow it to be specified as a parameter to the SHOW_MIGRATION_WIZARD SpecialMessageAction.

This would involve adding an ABOUT_WELCOME entrypoint to MigrationUtils.MIGRATION_ENTRYPOINTS and then teaching the SHOW_MIGRATION_WIZARD SpecialMessageAction (and schema) about a data object with an entrypoint property, so that users of the action could pass in MigrationUtils.MIGRATION_ENTRYPOINTS.ABOUT_WELCOME, and finally updating about:welcome to do so

To completely support the use case described, we will need to verify that this telemetry can be analyzed together with the telemetry about data import completion.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Note that the existing FX_MIGRATION_ENTRY_POINT histogram has reached its limit of 10 possible entrypoint values. Adding ABOUT_WELCOME will add another.

However, I have good news! In bug 1822692, I landed a new histogram, FX_MIGRATION_ENTRY_POINT_CATEGORICAL, which is a categorical histogram which defaults to having a maximum of 50 values.

We're using FX_MIGRATION_ENTRY_POINT and FX_MIGRATION_ENTRY_POINT_CATEGORICAL simultaneously right now to ensure a smooth transition from one metric to the other. We might want to alert data science to this, but I suggest adding your new entrypoint, and expecting those to enter the UNKNOWN bucket on the old histogram.

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

Ed pointed out that at the moment, the SHOW_MIGRATION_WIZARD SpecialMessageAction is only used by about:welcome, and that special message action is currently the only user of the NEWTAB entrypoint, which means that using the existing migration wizard and assuming that no one else (including experiments) starts to use those things, we can get that data for the general case right now.

This is fragile. I think this wants to be dealt with in the way Mike proposes so that we can actually use the data this way in an ongoing manner. I suspect we want to do this as part of the switch to the new migration wizard. Nadja, do you have an idea when you'd like to do this sort of analysis?

Comment 3

[nbulous]

I think it would be valuable for about:welcome experiments in general, especially now that we're started to try to provide contextual flows based on whether a user is new or continuing. Is this something that we can pick up for Fx116?

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

Removing priority to consider for 116 in the next triage meeting.