Add telemetry probes to understand what browser users are initially importing data from

RESOLVED FIXED in Firefox 48

Status

()

defect
P2
normal
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: Dolske, Assigned: Gijs)

Tracking

(Blocks 1 bug)

Trunk
Firefox 49
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox48 fixed, firefox49 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

Reporter

Description

3 years ago
Bug 1248077 seeks to improve the experience of the first-run "import data from another browser" process by doing it automatically. We'd like to gather some additional data to help understand if our selection of what browser to import from is doing a good job, by looking more closely at the choices users are currently making. As with the original probes added in bug 731025, these will need to be opt-out because they need to record things before the browser has fully started for the first time.

1) How often does the user have to make a choice between browsers? We have FX_MIGRATION_SOURCE_BROWSER which records what choice was selected (including "nothing"), but we don't know how often the user is presented with a choice of > 1 browser. I'd suggest a simple numerical probe recording how many choices we offer. This data will help guide how important the "choose 1 of N" logic needs to be, as we don't really know how common this scenario is.

2) What is the user's current default browser? We suspect that the current default browser is a strong signal of what browser we should import from. With some telemetry analysis, we should be able to determine how often the current default browser is what they chose to import from (again, via FX_MIGRATION_SOURCE_BROWSER). [N.B.: because the installer on Win XP/7 will currently default to making Firefox the default, we'll initially have to ignore those platforms. For bug 1248077, we'll either need to remove that or have the installer record what the previous default browser was.]

3) How fresh/relevant is the data available from other browsers? In some cases, the user's default browser may not be the browser they actually use the most (and thus want to import from). In other cases, the most-recently-used browser may not have been used for anything more than downloading Firefox, so importing that minimal history and default bookmarks is not really useful. Again, with telemetry analysis, we'd like to see how often the browser users actually import from is actually what we think is the freshest.


#3 is a little tricky to define precisely, as difficulty of implementation may guide what we actually do. I'd suggest a erring on the side of simplicity to see what the initial signal looks like. One option would be to pick a particular data source (bookmarks? history? passwords? all?), and see how old the most recent entry is. A file timestamp might be enough. Another option would be to see how many entries there are for a type -- is the user importing from the browser with lots of bookmarks/history/passwords? Are browsers with < X bookmarks/history/passwords generally ignored?
Reporter

Updated

3 years ago
Priority: -- → P2
Assignee

Comment 1

3 years ago
Posted patch WIP patch (obsolete) — Splinter Review
This patch adds 3 histograms for the 3 things discussed, updates the current histograms to be kept until v53 or later, and implements the first (easiest) item.

I'm looking at #2, obtaining the default browser. At least on Windows, it seems I basically want to be reading:

HKEY_CLASSES_ROOT\http\shell\open\command

and simply determine whether it contains chrome.exe / firefox.exe / iexplore.exe / whatever-the-value-for-edge-is .

Not yet sure about the other OSes, but we probably care less about those anyway.

Matt/Dolske, can (either of) you give this a quick look and give feedback as to whether this seems right? I aim to have a reviewable thing tomorrow, but it might end up with just #1 or just #2 if I can't quickly work out the other ones.
Assignee: nobody → gijskruitbosch+bugs
Status: NEW → ASSIGNED
Attachment #8755618 - Flags: feedback?(dolske)
Attachment #8755618 - Flags: feedback?(MattN+bmo)
Comment on attachment 8755618 [details] [diff] [review]
WIP patch

Review of attachment 8755618 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to :Gijs Kruitbosch from comment #1)
> I'm looking at #2, obtaining the default browser. At least on Windows, it
> seems I basically want to be reading:
> 
> HKEY_CLASSES_ROOT\http\shell\open\command
> 
> and simply determine whether it contains chrome.exe / firefox.exe /
> iexplore.exe / whatever-the-value-for-edge-is .
> 
> Not yet sure about the other OSes, but we probably care less about those
> anyway.

I think you can use getMigratorKeyForDefaultBrowser instead of rolling the logic yourself.

::: toolkit/components/telemetry/Histograms.json
@@ +4430,5 @@
> +    "kind": "exponential",
> +    "n_buckets": 50,
> +    "high": 8760,
> +    "releaseChannelCollection": "opt-out",
> +    "description": "The 'last modified' time of the data we imported on the initial profile migration (time delta with 'now' at the time of migration, in hours)."

If you're accumulating for each entry we import then I think this is good. That way we get both the distribution of timestamps and counts IIUC instead of just a single date.

Note that we can already correlate our migration probes with PLACES_PAGES_COUNT for a simple count though we'll want to exclude users with sync setup since that is another way to bring in a bunch of pages on first use.
Attachment #8755618 - Flags: feedback?(MattN+bmo) → feedback+
Assignee

Comment 3

3 years ago
(In reply to Matthew N. [:MattN] (behind on reviews) from comment #2)
> Comment on attachment 8755618 [details] [diff] [review]
> WIP patch
> 
> Review of attachment 8755618 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> (In reply to :Gijs Kruitbosch from comment #1)
> > I'm looking at #2, obtaining the default browser. At least on Windows, it
> > seems I basically want to be reading:
> > 
> > HKEY_CLASSES_ROOT\http\shell\open\command
> > 
> > and simply determine whether it contains chrome.exe / firefox.exe /
> > iexplore.exe / whatever-the-value-for-edge-is .
> > 
> > Not yet sure about the other OSes, but we probably care less about those
> > anyway.
> 
> I think you can use getMigratorKeyForDefaultBrowser instead of rolling the
> logic yourself.

Good point. This also reminds me we never updated that for Edge.
Depends on: 1199678
Assignee

Updated

3 years ago
Attachment #8755618 - Attachment is obsolete: true
Attachment #8755618 - Flags: feedback?(dolske)
Assignee

Comment 5

3 years ago
The mozreview patch implements (1) and (2) from comment #0. (3) is significantly more complicated. I'm looking into it, but I figured I'd get this out there first.
Reporter

Comment 6

3 years ago
(I took a skim through the current patch, LGTM. Glad #2 turned out to be pretty easy.)
Reporter

Comment 7

3 years ago
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

https://reviewboard.mozilla.org/r/54918/#review52506

Seems simple enough, so r+. Obviously needs a privacy review.

Are you going to do the #3 from comment 0 in a separate bug, or a separate patch here?

Is there a followup for handling Edge correctly?

Also, as a technicality, there's an entry for "nothing" but I don't think the code can currently hit that. Although I'm not sure how likely that is in the real world, I'd expect everyone to generally have a default, with rare expections (e.g. maybe by installing a default browser?)
Attachment #8755998 - Flags: review+
Reporter

Updated

3 years ago
Attachment #8755998 - Flags: review?(MattN+bmo)
Assignee

Comment 8

3 years ago
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

(In reply to Justin Dolske [:Dolske] from comment #7)
> Comment on attachment 8755998 [details]
> MozReview Request: Bug 1275114 - add telemetry to determine how to make
> browser choices when automatically migrating on first profile startup,
> r?MattN
> 
> https://reviewboard.mozilla.org/r/54918/#review52506
> 
> Seems simple enough, so r+. Obviously needs a privacy review.

Benjamin, can you privacy-review this, please? :-)


> Are you going to do the #3 from comment 0 in a separate bug, or a separate
> patch here?

Probably saner to do a separate bug at this stage.

> Is there a followup for handling Edge correctly?

Yes, I filed it way back when we looked at edge/win10, I added it to the dep tree for this bug.

> Also, as a technicality, there's an entry for "nothing" but I don't think
> the code can currently hit that. Although I'm not sure how likely that is in
> the real world, I'd expect everyone to generally have a default, with rare
> expections

Right, we have this because we reuse these identifiers for the things people pick in the dialog (ie what do you want to import from), and in the dialog has "nothing" as an option.

> (e.g. maybe by installing a default browser?)

do you mean uninstalling? :-)
Attachment #8755998 - Flags: review?(benjamin)
Reporter

Comment 9

3 years ago
(In reply to :Gijs Kruitbosch from comment #8)

> > Also, as a technicality, there's an entry for "nothing" but I don't think
> > the code can currently hit that. Although I'm not sure how likely that is in
> > the real world, I'd expect everyone to generally have a default, with rare
> > expections
> 
> Right, we have this because we reuse these identifiers for the things people
> pick in the dialog (ie what do you want to import from), and in the dialog
> has "nothing" as an option.

Oh, my bad. I thought this was intended to signal "there is no default browser".

> > (e.g. maybe by installing a default browser?)
> 
> do you mean uninstalling? :-)

Yes.
Assignee

Updated

3 years ago
Blocks: 1276694
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

https://reviewboard.mozilla.org/r/54918/#review53684
Attachment #8755998 - Flags: review?(benjamin) → review+

Comment 11

3 years ago
Pushed by gijskruitbosch@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/03f8bccf10ea
add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r=bsmedberg,Dolske

Comment 12

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/03f8bccf10ea
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 49
Assignee

Comment 13

3 years ago
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

Ritu, we'd like to get this into 48, would that still be OK?

Approval Request Comment
[Feature/regressing bug #]: ongoing work on migrators/importers
[User impact if declined]: no data to make decisions about migrators/importers
[Describe test coverage new/current, TreeHerder]: telemetry only, so no tests;
[Risks and why]: pretty low, minor changes, only adding telemetry to existing code
[String/UUID change made/needed]: no.
Flags: needinfo?(rkothari)
Attachment #8755998 - Flags: approval-mozilla-beta?
Attachment #8755998 - Flags: approval-mozilla-aurora?
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

Since we are in the very early stages of Beta48 cycle, I think this can be uplifted to both 49 and 48. Hi Sylvestre, fyi in case you disagree (since you own Fx48 release).
Flags: needinfo?(rkothari) → needinfo?(sledru)
Attachment #8755998 - Flags: approval-mozilla-beta?
Attachment #8755998 - Flags: approval-mozilla-beta+
Attachment #8755998 - Flags: approval-mozilla-aurora?
Attachment #8755998 - Flags: approval-mozilla-aurora+
Comment on attachment 8755998 [details]
MozReview Request: Bug 1275114 - add telemetry to determine how to make browser choices when automatically migrating on first profile startup, r?MattN

This is already on 49.
Attachment #8755998 - Flags: approval-mozilla-aurora+
Agreed, thanks for the info
Flags: needinfo?(sledru)
You need to log in before you can comment on or make changes to this bug.