Closed Bug 1275114 Opened 4 years ago Closed 4 years ago

Add telemetry probes to understand what browser users are initially importing data from

Categories

(Firefox :: Migration, defect, P2)

defect

Tracking

()

RESOLVED FIXED
Firefox 49
Tracking Status
firefox48 --- fixed
firefox49 --- fixed

People

(Reporter: Dolske, Assigned: Gijs)

References

Details

Attachments

(1 file, 1 obsolete file)

Bug 1248077 seeks to improve the experience of the first-run "import data from another browser" process by doing it automatically. We'd like to gather some additional data to help understand if our selection of what browser to import from is doing a good job, by looking more closely at the choices users are currently making. As with the original probes added in bug 731025, these will need to be opt-out because they need to record things before the browser has fully started for the first time.

1) How often does the user have to make a choice between browsers? We have FX_MIGRATION_SOURCE_BROWSER which records what choice was selected (including "nothing"), but we don't know how often the user is presented with a choice of > 1 browser. I'd suggest a simple numerical probe recording how many choices we offer. This data will help guide how important the "choose 1 of N" logic needs to be, as we don't really know how common this scenario is.

2) What is the user's current default browser? We suspect that the current default browser is a strong signal of what browser we should import from. With some telemetry analysis, we should be able to determine how often the current default browser is what they chose to import from (again, via FX_MIGRATION_SOURCE_BROWSER). [N.B.: because the installer on Win XP/7 will currently default to making Firefox the default, we'll initially have to ignore those platforms. For bug 1248077, we'll either need to remove that or have the installer record what the previous default browser was.]

3) How fresh/relevant is the data available from other browsers? In some cases, the user's default browser may not be the browser they actually use the most (and thus want to import from). In other cases, the most-recently-used browser may not have been used for anything more than downloading Firefox, so importing that minimal history and default bookmarks is not really useful. Again, with telemetry analysis, we'd like to see how often the browser users actually import from is actually what we think is the freshest.


#3 is a little tricky to define precisely, as difficulty of implementation may guide what we actually do. I'd suggest a erring on the side of simplicity to see what the initial signal looks like. One option would be to pick a particular data source (bookmarks? history? passwords? all?), and see how old the most recent entry is. A file timestamp might be enough. Another option would be to see how many entries there are for a type -- is the user importing from the browser with lots of bookmarks/history/passwords? Are browsers with < X bookmarks/history/passwords generally ignored?
Priority: -- → P2
Attached patch WIP patch (obsolete) — Splinter Review
This patch adds 3 histograms for the 3 things discussed, updates the current histograms to be kept until v53 or later, and implements the first (easiest) item.

I'm looking at #2, obtaining the default browser. At least on Windows, it seems I basically want to be reading:

HKEY_CLASSES_ROOT\http\shell\open\command

and simply determine whether it contains chrome.exe / firefox.exe / iexplore.exe / whatever-the-value-for-edge-is .

Not yet sure about the other OSes, but we probably care less about those anyway.

Matt/Dolske, can (either of) you give this a quick look and give feedback as to whether this seems right? I aim to have a reviewable thing tomorrow, but it might end up with just #1 or just #2 if I can't quickly work out the other ones.
Assignee: nobody → gijskruitbosch+bugs
Status: NEW → ASSIGNED
Attachment #8755618 - Flags: feedback?(dolske)
Attachment #8755618 - Flags: feedback?(MattN+bmo)
Comment on attachment 8755618 [details] [diff] [review]
WIP patch

Review of attachment 8755618 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to :Gijs Kruitbosch from comment #1)
> I'm looking at #2, obtaining the default browser. At least on Windows, it
> seems I basically want to be reading:
> 
> HKEY_CLASSES_ROOT\http\shell\open\command
> 
> and simply determine whether it contains chrome.exe / firefox.exe /
> iexplore.exe / whatever-the-value-for-edge-is .
> 
> Not yet sure about the other OSes, but we probably care less about those
> anyway.

I think you can use getMigratorKeyForDefaultBrowser instead of rolling the logic yourself.

::: toolkit/components/telemetry/Histograms.json
@@ +4430,5 @@
> +    "kind": "exponential",
> +    "n_buckets": 50,
> +    "high": 8760,
> +    "releaseChannelCollection": "opt-out",
> +    "description": "The 'last modified' time of the data we imported on the initial profile migration (time delta with 'now' at the time of migration, in hours)."

If you're accumulating for each entry we import then I think this is good. That way we get both the distribution of timestamps and counts IIUC instead of just a single date.

Note that we can already correlate our migration probes with PLACES_PAGES_COUNT for a simple count though we'll want to exclude users with sync setup since that is another way to bring in a bunch of pages on first use.
Attachment #8755618 - Flags: feedback?(MattN+bmo) → feedback+
(In reply to Matthew N. [:MattN] (behind on reviews) from comment #2)
> Comment on attachment 8755618 [details] [diff] [review]
> WIP patch
> 
> Review of attachment 8755618 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> (In reply to :Gijs Kruitbosch from comment #1)
> > I'm looking at #2, obtaining the default browser. At least on Windows, it
> > seems I basically want to be reading:
> > 
> > HKEY_CLASSES_ROOT\http\shell\open\command
> > 
> > and simply determine whether it contains chrome.exe / firefox.exe /
> > iexplore.exe / whatever-the-value-for-edge-is .
> > 
> > Not yet sure about the other OSes, but we probably care less about those
> > anyway.
> 
> I think you can use getMigratorKeyForDefaultBrowser instead of rolling the
> logic yourself.

Good point. This also reminds me we never updated that for Edge.
Depends on: 1199678
Attachment #8755618 - Attachment is obsolete: true
Attachment #8755618 - Flags: feedback?(dolske)
The mozreview patch implements (1) and (2) from comment #0. (3) is significantly more complicated. I'm looking into it, but I figured I'd get this out there first.
(I took a skim through the current patch, LGTM. Glad #2 turned out to be pretty easy.)
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

https://reviewboard.mozilla.org/r/54918/#review52506

Seems simple enough, so r+. Obviously needs a privacy review.

Are you going to do the #3 from comment 0 in a separate bug, or a separate patch here?

Is there a followup for handling Edge correctly?

Also, as a technicality, there's an entry for "nothing" but I don't think the code can currently hit that. Although I'm not sure how likely that is in the real world, I'd expect everyone to generally have a default, with rare expections (e.g. maybe by installing a default browser?)
Attachment #8755998 - Flags: review+
Attachment #8755998 - Flags: review?(MattN+bmo)
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

(In reply to Justin Dolske [:Dolske] from comment #7)
> Comment on attachment 8755998 [details]
> MozReview Request: Bug 1275114 - add telemetry to determine how to make
> browser choices when automatically migrating on first profile startup,
> r?MattN
> 
> https://reviewboard.mozilla.org/r/54918/#review52506
> 
> Seems simple enough, so r+. Obviously needs a privacy review.

Benjamin, can you privacy-review this, please? :-)


> Are you going to do the #3 from comment 0 in a separate bug, or a separate
> patch here?

Probably saner to do a separate bug at this stage.

> Is there a followup for handling Edge correctly?

Yes, I filed it way back when we looked at edge/win10, I added it to the dep tree for this bug.

> Also, as a technicality, there's an entry for "nothing" but I don't think
> the code can currently hit that. Although I'm not sure how likely that is in
> the real world, I'd expect everyone to generally have a default, with rare
> expections

Right, we have this because we reuse these identifiers for the things people pick in the dialog (ie what do you want to import from), and in the dialog has "nothing" as an option.

> (e.g. maybe by installing a default browser?)

do you mean uninstalling? :-)
Attachment #8755998 - Flags: review?(benjamin)
(In reply to :Gijs Kruitbosch from comment #8)

> > Also, as a technicality, there's an entry for "nothing" but I don't think
> > the code can currently hit that. Although I'm not sure how likely that is in
> > the real world, I'd expect everyone to generally have a default, with rare
> > expections
> 
> Right, we have this because we reuse these identifiers for the things people
> pick in the dialog (ie what do you want to import from), and in the dialog
> has "nothing" as an option.

Oh, my bad. I thought this was intended to signal "there is no default browser".

> > (e.g. maybe by installing a default browser?)
> 
> do you mean uninstalling? :-)

Yes.
Blocks: 1276694
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

https://reviewboard.mozilla.org/r/54918/#review53684
Attachment #8755998 - Flags: review?(benjamin) → review+
Pushed by gijskruitbosch@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/03f8bccf10ea
add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r=bsmedberg,Dolske
https://hg.mozilla.org/mozilla-central/rev/03f8bccf10ea
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 49
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

Ritu, we'd like to get this into 48, would that still be OK?

Approval Request Comment
[Feature/regressing bug #]: ongoing work on migrators/importers
[User impact if declined]: no data to make decisions about migrators/importers
[Describe test coverage new/current, TreeHerder]: telemetry only, so no tests;
[Risks and why]: pretty low, minor changes, only adding telemetry to existing code
[String/UUID change made/needed]: no.
Flags: needinfo?(rkothari)
Attachment #8755998 - Flags: approval-mozilla-beta?
Attachment #8755998 - Flags: approval-mozilla-aurora?
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

Since we are in the very early stages of Beta48 cycle, I think this can be uplifted to both 49 and 48. Hi Sylvestre, fyi in case you disagree (since you own Fx48 release).
Flags: needinfo?(rkothari) → needinfo?(sledru)
Attachment #8755998 - Flags: approval-mozilla-beta?
Attachment #8755998 - Flags: approval-mozilla-beta+
Attachment #8755998 - Flags: approval-mozilla-aurora?
Attachment #8755998 - Flags: approval-mozilla-aurora+
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

This is already on 49.
Attachment #8755998 - Flags: approval-mozilla-aurora+
Agreed, thanks for the info
Flags: needinfo?(sledru)
You need to log in before you can comment on or make changes to this bug.