1602463 - (wdba-phase1) Windows Default Browser Agent - Phase 1

Status: RESOLVED FIXED

(Toolkit :: Default Browser Agent, enhancement, P1)

Description

[Romain Testard [:RT]]

Context:

• The new Chromium Edge is set to be released on January 15th 2020
• The last Windows 7 security update will happen on January 14th 2020
• We fear that Microsoft will aggressively promote Chromium Edge when it releases and we want to be able to detect trend shifts in Win10 and Win7 default browsers so we’re armed to react to forced changes on browser default

What is necessary:

• An “always on” agent on Windows that allows:
  --- Collecting data on default browser set
  --- Re-engaging Firefox users through OS notifications based on a set of events. Events would initially include a change in system settings from a Firefox default browser to an Edge default browser but should be extensible to allow driving engagement for other purposes.

Acceptance criteria:
P1 - Win10 support: Installing Firefox will install the agent, uninstalling Firefox will uninstall the agent, updating Firefox will install the agent.
P1 - The agent runs well in environments where anti-virus software is installed
P1 - The agent launches as Windows starts
P1 - The agent can be disabled for managed deployments
P1 - The agent sends a daily ping with information about the default browser on the system
P2 - Win7 support: Installing Firefox will install the agent, uninstalling Firefox will uninstall the agent, updating Firefox will install the agent.
P2 - The agent delivers a system notification to the Windows 10 action center if the default browser on the system changed from Firefox to Edge
P2 - The agent auto launch on startup behavior is controlled by a Firefox preference to allow A/B testing through Experimenter
P2 - Telemetry allows identifying browser sessions that start by opening Firefox through a notification
P3 - The agent delivers a system notification on Windows 7 if the default browser on the system changed from Firefox to Edge

Comment 1

[Jim Mathies [:jimm]]

Open question - Microsoft currently promotes Edge heavily through notifications. They also question user default browser changes through notifications. I'm curious why Product feels Microsoft's behavior will become more aggressive than it already is? Also I don't understand why we think users will switch to Chromium Edge by large numbers (Is there some way we can research this?), it's really just a new shell on top of Chrome.

I like the idea of countering Microsoft's default browser promoting but I fail to see why this project is critically tied to the release of Chromium Edge. I'd also like point out the timelines published for this aren't realistic so hopefully I'm right about this.

Comment 2

[Peter Dolanjski [:pdol]]

(In reply to Jim Mathies [:jimm] from comment #1)

Open question - Microsoft currently promotes Edge heavily through notifications. They also question user default browser changes through notifications. I'm curious why Product feels Microsoft's behavior will become more aggressive than it already is? Also I don't understand why we think users will switch to Chromium Edge by large numbers (Is there some way we can research this?), it's really just a new shell on top of Chrome.

We saw Microsoft switch defaults on users previously with OS updates, via Win10 updates (beyond the notifications they do). They have a large investment tied to this revamp and they will want to maximize success, so their incentives will be to push it in whatever ways they can so they can prove to their leadership and shareholders that this was the right strategy.

We've done an initial user study [1]: that concluded: "Respondents think positively of the new Edge compared to other browsers as well as the previous version of Edge. Compared to other browsers and the previous of Edge, the respondents think the new Edge has more features, is more stable, is faster, is safer, works well with more websites, and is better for privacy. "

These two reasons make it potentially a larger threat. dd

I'd also like point out the timelines published for this aren't realistic so hopefully I'm right about this.

We're happy to work with you on what is realistic. Even if we're not ready on day 1, better late than never.

[1] https://docs.google.com/document/d/1ufLIZDoUlSLyNB0q05UNvaBhyfScd9al2UXB1KeNYLU/edit?ts=5de090ec

Comment 3

[Molly Howell (she/her) (no longer active)]

Attachment: Bug 1602463 Part 1 - Add a build flag to disable the default browser agent. r=agashlin,bytesized,nalexander

Comment 4

[Molly Howell (she/her) (no longer active)]

Attachment: Bug 1602463 Part 3 - Windows default browser agent. r=agashlin,bytesized,nalexander

Depends on D61888

Comment 5

[Molly Howell (she/her) (no longer active)]

Attachment: Bug 1602463 Part 4 - Register/unregister the default browser agent scheduled task during install/uninstall. r=agashlin,bytesized,nalexander

Depends on D61889

Comment 6

[Molly Howell (she/her) (no longer active)]

Attachment: Bug 1602463 Part 2 - Add a pref to enable/disable the default browser agent and reflect it to the registry where the agent can read it. r=agashlin,bytesized,nalexander

Depends on D61891

Comment 7

[Chris H-C :chutten]

Hi! A drive-by reminder that new data collections are subject to Data Collection Review (and the data review must be made publicly accessible to all those subject to the collection).

(( If you were already planning on requesting data review, Good Work and sorry for the noise ))

Comment 8

[Molly Howell (she/her) (no longer active)]

I'm filling out the form right now. ;)

Comment 9

[Molly Howell (she/her) (no longer active)]

Comment on attachment 9124828 [details]
Bug 1602463 Part 3 - Windows default browser agent. r=agashlin,bytesized,nalexander

What questions will you answer with this data?

Comment 0 and the linked PRD [1] explain this, but the specific idea is to monitor for changes in default browser settings on Windows resulting from the Chrome-based Edge rollout.

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

We need to determine whether an operating system change has an effect on user behavior, to know if we need to take specific steps to counteract it. We still don't know the full details on how the rollout/migration for the new Edge will be implemented or what effects on the default browser setting might result.

What alternative methods did you consider to answer these questions? Why were they not sufficient?

There's not really any existing Firefox telemetry that will work, for the basic reason that it all requires running Firefox, and the scenario we're worried about is one in which the default browser has been changed and therefore Firefox no longer runs.

Can current instrumentation answer these questions?

See above.

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Category 1:

• Firefox release channel
• Firefox version
• Operating system (Windows) version
• Operating system locale setting

Category 2 (I'm not 100% confident in this category assignment):

• Current system default browser, represented as one of the strings "firefox", "chrome", "edge", "edge-chrome", "ie", "opera", or "brave".
• Previous system default browser, before it was changed to the current one; one of the same set of strings as above.

There's ping documentation included in this patch which provides more details.

How long will this data be collected?

Permanently as far as I know (with me, mhowell, as the responsible engineer), but product may have a timeframe in mind that I'm unaware of.

What populations will you measure?

All Windows users (other operating systems are excluded).

Which release channels?

All.

Which countries?

All.

Which locales?

All.

Any other filters? Please describe in detail below.

None at this time.

If this data collection is default on, what is the opt-out mechanism for users?

The normal Firefox telemetry opt-out pref is honored, and this patch series adds a new pref defaultagent.enabled which specifically disables the agent's functionality. The telemetry opt-out enterprise policy is also honored, and a new agent-specific policy is added here as well; these policies allow opting out before it is possible that any pings would have been generated.

Please provide a general description of how you will analyze this data.

I'm not going to be doing this analysis myself so I can't be certain (contact Romain Testard for details), but I believe the analysis would be conducted mostly (and perhaps entirely) via STMO queries on these pings (which are custom pings without any telemetry identifiers).

Where do you intend to share the results of your analysis?

I think the results would not be widely shared (i.e., the STMO queries would be accessible, but nothing broadly distributed otherwise). But again, ask Romain for details.

Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No.

Are you using that on the Mozilla backend? Or going directly to the third-party?

The Mozilla backend.

[1] Note that the notification functionality described in the PRD has not been implemented yet.

Comment 10

[Molly Howell (she/her) (no longer active)]

I've opened https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/495 for the schema for the newly added ping.

Comment 11

[Chris H-C :chutten]

Comment on attachment 9124828 [details]
Bug 1602463 Part 3 - Windows default browser agent. r=agashlin,bytesized,nalexander

Some notes:

1. Please file future data review requests as attachments on bugs. This fits better into Data Stewardship's workflow (and keeps the comments timeline cleaner)
2. The review request seems complete and correct, though I worry that the agent might perhaps send data before the user has had a chance to review the Privacy Notice and take action to opt out. This might fall outside the scope of data review, so I apologize if I overstep, but has Legal/Trust been involved in the consent model design?
3. Will this bug be made public when the data collection begins? By policy, Data Reviews must be available to the population affected by the collection. (Usually in these cases where you want some discussion/design to remain private a bug just for the data review is filed and is either public or is confidential until it's merged and then is made public)

Comment 12

[Molly Howell (she/her) (no longer active)]

(In reply to Chris H-C :chutten from comment #11)

1. Please file future data review requests as attachments on bugs. This fits better into Data Stewardship's workflow (and keeps the comments timeline cleaner)

Sorry about that. To clarify, does that mean that the recommended procedure is to fill out the form in a text file (or I guess a Markdown file, but there's disagreement in different docs about that), attach that to the bug, and then set the request flag on that attachment? I sure wouldn't mind if some piece of documentation were a little more explicit about that, since it sounds like a pretty unique flow for Bugzilla and I obviously assumed it worked like how other request flags work.

2. The review request seems complete and correct, though I worry that the agent might perhaps send data before the user has had a chance to review the Privacy Notice and take action to opt out. This might fall outside the scope of data review, so I apologize if I overstep, but has Legal/Trust been involved in the consent model design?

The task runs on a 24-hour schedule that starts counting from the time of installation, so nothing is sent for 24 hours after then; I was hoping that would be enough delay. I certainly don't mind consulting legal and/or trust though if you feel like we should.

3. Will this bug be made public when the data collection begins? By policy, Data Reviews must be available to the population affected by the collection. (Usually in these cases where you want some discussion/design to remain private a bug just for the data review is filed and is either public or is confidential until it's merged and then is made public)

Redirecting this question to Romain because I'm honestly not certain why this bug was filed as private in the first place.

Comment 13

[Chris H-C :chutten]

(In reply to Molly Howell (she/her) [:mhowell] from comment #12)

(In reply to Chris H-C :chutten from comment #11)

1. Please file future data review requests as attachments on bugs. This fits better into Data Stewardship's workflow (and keeps the comments timeline cleaner)

Sorry about that. To clarify, does that mean that the recommended procedure is to fill out the form in a text file (or I guess a Markdown file, but there's disagreement in different docs about that), attach that to the bug, and then set the request flag on that attachment? I sure wouldn't mind if some piece of documentation were a little more explicit about that, since it sounds like a pretty unique flow for Bugzilla and I obviously assumed it worked like how other request flags work.

No worries, the work is imperfect and we try to make it better each day. You can straight up paste the text of the review request into the "Add Attachment" BMO UI and it'll take care of making it into a file for you. The documentation covering how data reviews are requested can be found on wikimo: https://wiki.mozilla.org/Firefox/Data_Collection#Requesting_Data_Collection

Happy to help out with clarifying that page to make it better. Ideally, Data Stewardship would get some budget to get a web service set up so that the reviews could be, y'know, real forms. And a database so we could audit things more easily and so users could query things more easily and so we could interoperate with other data tools.

Alas : |

2. The review request seems complete and correct, though I worry that the agent might perhaps send data before the user has had a chance to review the Privacy Notice and take action to opt out. This might fall outside the scope of data review, so I apologize if I overstep, but has Legal/Trust been involved in the consent model design?

The task runs on a 24-hour schedule that starts counting from the time of installation, so nothing is sent for 24 hours after then; I was hoping that would be enough delay. I certainly don't mind consulting legal and/or trust though if you feel like we should.

If there were a signal that Firefox were run in the first 23h30m of being installed there'd be no worries at all with this schedule as that puts it in line with the existing data reporting policy code inside Firefox (ie, give 30min to read the notice and find the setting). Since this is decoupled from Firefox running, I don't know if the same thing applies. I say reach out to Alicia Gray about this requirement.

Comment 14

[Molly Howell (she/her) (no longer active)]

Quick update on the current status here: the patch stack for this bug is complete, and landing it is blocked on bug 1615281, which is going to be a public blog post explaining how the data collection for this thing works and what the methods for opting out of it will be. Nothing will be pushed even to Nightly until that blog post is up, so that we don't start doing any collection before there's clear information on how to turn it off.

Comment 15

[Molly Howell (she/her) (no longer active)]

(In reply to Molly Howell (she/her) [:mhowell] from comment #14)

Quick update on the current status here: the patch stack for this bug is complete [...]

Got ahead of myself: it turned out that we need some stubs in the Firefox policy engine for the default agent policy even though the engine doesn't do anything to implement it, so that it shows up properly in about:policies. So I'm adding another patch to the series to address that.

Comment 16

[Molly Howell (she/her) (no longer active)]

Attachment: Bug 1602463 Part 5 - Add policy engine support for the default browser agent policy. r=mkaply

Depends on D61891

Comment 17

[Rachel Tublitz [:rachel]]

Adjusting this issue so it's clear this work pertains to the following only:

• Win10+7 support: Installing Firefox will install the agent, uninstalling Firefox will uninstall the agent, updating Firefox will install the agent.
• The agent runs well in environments where anti-virus software is installed.
• The agent launches as Windows starts.
• The agent can be disabled for managed deployments.
• The agent sends a daily ping with information about the default browser on the system.

We'll file future follow up issues to track work on the next steps of this project.

Comment 18

[Rachel Tublitz [:rachel]]

chutten: can you confirm data review approval for this (pending us reaching out to Alicia Gray)?

agray: can you confirm that we did indeed connect with you (was done via email) and that we're moving forward with a plan to first publish a blog post here?

I'm trying to get all our approvals rounded up in one place so we can move forward with a request for uplift into 75.

Comment 19

[Alicia Gray]

HI Rachel,

Yes, confirming review was completed with Trust and approval given for this. Blog was also reviewed and commented on. No blockers on the Trust side. My review notes were as follows:

Thank you for your email - more than happy to give input for you on this question. I’ve listed out the points below that will answer your question and give some additional context for handling perceived risk.

The data collection outside of Firefox is ok if:
-it is consistent with the data categories collection (which it is)
-it honors the opt-out setting (which it looks like it can/will)
-we are transparent about it. The real risk here is in violating our “no surprises” principle. People might be surprised by this so the team needs to think about and plan for how we can be transparent about it. I’d be happy to help support the development of a comms plan, but this needs to be part of the equation.

Re: 24 hour installation v. launch issue, this would be ok with us. If the user hasn’t changed their telemetry opt-out pref by then for whatever r
reason, we’re fine accepting it as default collection.

Collection time frame: it’s my understanding the request to collect is on-going (permanent ping). We’d recommend limiting this collection to 1 year and then reassessing its usefulness. You can always extend the collection if it is useful and make it permanent.

Secondary use cases: The PRD mentions use cases that could leverage this data for notifications to users for a couple of different purposes. Please note that any use of this data beyond the measurement itself (e.g. marketing or in-product notifications etc.) would need a different review through Trust and Legal. Approval for the data collection doesn’t give approval for the other potential use cases.

Comment 20

[Chris H-C :chutten]

I think we should still perform a data review in public for the collection. This will ensure we have things like public-facing documentation for the data being collected, and a public process we've completed and can point to in order to show that this is all above board. This can be in a separate bug to maintain this as a confidential one.

Comment 21

[Rachel Tublitz [:rachel]]

(In reply to Chris H-C :chutten from comment #20)

I think we should still perform a data review in public for the collection. This will ensure we have things like public-facing documentation for the data being collected, and a public process we've completed and can point to in order to show that this is all above board. This can be in a separate bug to maintain this as a confidential one.

Thanks, Chris. Just filed this request as a public bug.

Comment 22

[Alexandru Michis [:malexandru]]

Backed out 5 changesets for causing diffoscope failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/e87acf21935d5ff91c3e9dafe554a314ad31fd2b

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=e87acf21935d5ff91c3e9dafe554a314ad31fd2b&fromchange=911385b3cbc028a6b86ab884ca7955b4aaaa9898&searchStr=diffoscope%2Copt%2Cdiff-artifact-win64-aarch64-eme-validation%2C%28dwe%29&selectedJob=293363887

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=293366507&repo=autoland&lineNumber=7026

Comment 23

[Molly Howell (she/her) (no longer active)]

Working on it.

Comment 24

[Molly Howell (she/her) (no longer active)]

Found a fix that makes the failing job pass on try, about to try landing again with that.

Comment 25

[Alexandru Michis [:malexandru]]

I'm also seeing these failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2C18.04%2Cx64%2Cshippable%2Copt%2Cmochitests%2Ctest-linux1804-64-shippable%2Fopt-mochitest-browser-chrome-e10s-4%2Cm%28bc4%29&tochange=e87acf21935d5ff91c3e9dafe554a314ad31fd2b&fromchange=9f58bd341b2f53ac6f692586991a2659240d0a0c&selectedJob=293371201

Comment 26

[Molly Howell (she/her) (no longer active)]

Thanks, didn't see that; will add in that fix also.

Comment 27

[Nick Alexander :nalexander [he/him]]

(In reply to Alexandru Michis [:malexandru] from comment #22)

Backed out 5 changesets for causing diffoscope failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/e87acf21935d5ff91c3e9dafe554a314ad31fd2b

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=e87acf21935d5ff91c3e9dafe554a314ad31fd2b&fromchange=911385b3cbc028a6b86ab884ca7955b4aaaa9898&searchStr=diffoscope%2Copt%2Cdiff-artifact-win64-aarch64-eme-validation%2C%28dwe%29&selectedJob=293363887

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=293366507&repo=autoland&lineNumber=7026

It surprises me that that diffoscope failures are causing backouts at all; that's new since I looked last. NI to me to follow-up at a time when I can think about this.

Comment 28

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/integration/autoland/rev/c717a1a030e244dfa41e60277a485dabe7483294
https://hg.mozilla.org/integration/autoland/rev/54c091d1c59c3efd0b54a5d6acf84d0c12f381ee
https://hg.mozilla.org/integration/autoland/rev/91b694e8361543ba446e84962f8d42d2868f2b5c
https://hg.mozilla.org/integration/autoland/rev/61e1afdf2642321ffb5fa73e2e6f4fa433385eb1
https://hg.mozilla.org/integration/autoland/rev/952bdf4adda8b871237f0c917e5976c22b7b9c3c
https://hg.mozilla.org/mozilla-central/rev/c717a1a030e2
https://hg.mozilla.org/mozilla-central/rev/54c091d1c59c
https://hg.mozilla.org/mozilla-central/rev/91b694e83615
https://hg.mozilla.org/mozilla-central/rev/61e1afdf2642
https://hg.mozilla.org/mozilla-central/rev/952bdf4adda8

Comment 29

[Rachel Tublitz [:rachel]]

Comment on attachment 9124827 [details]
Bug 1602463 Part 1 - Add a build flag to disable the default browser agent. r=agashlin,bytesized,nalexander

Beta/Release Uplift Approval Request

• User impact if declined: There is no direct user impact here; it’s background telemetry that we’re collecting. However, if declined, we risk losing out on collecting potentially valuable data that’s time sensitive due to the recent Edgium release.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Low: It’s a scheduled task that’s run in the background; not user facing.
  It is however worth noting that there are some PR concerns related to this release due to the additional background telemetry that we’re collecting. To mitigate these risks, we've published a blog post here: https://blog.mozilla.org/data/2020/03/16/understanding-default-browser-trends/ that we plan to point users to.
• String changes made/needed: Yes, here, in the last patch related to enterprise policies. See Molly's comment below here (comment #30).

Comment 30

[Molly Howell (she/her) (no longer active)]

There actually is a new string in the part 5 patch here. I should have recognized earlier that that would complicate uplift, I apologize for not thinking of it.

If it's a problem, my proposed workaround would be to leave part 5 out of the uplift. The only function of that part is to add documentation in about:policies for the enterprise policy that the rest of the series implements, and to prevent the new policy from appearing there as an error. In my opinion it would be okay to knowingly leave those issues in place for a single version, if it is necessary.

Comment 31

[Molly Howell (she/her) (no longer active)]

Since the uplift request here was filed, bug 1624047 and bug 1624388 have been added to the "other uplifts needed" list. My apologies.

Comment 32

[Julien Cristau [:jcristau]]

Comment on attachment 9124827 [details]
Bug 1602463 Part 1 - Add a build flag to disable the default browser agent. r=agashlin,bytesized,nalexander

approved for 75.0b9 per Selena.

Comment 33

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/releases/mozilla-beta/rev/272ce48e4c439035269d4e801a3e78ef3d0805e9
https://hg.mozilla.org/releases/mozilla-beta/rev/53fd3c5b278d0d3e8728952b9f29c8e8cba22e92
https://hg.mozilla.org/releases/mozilla-beta/rev/33c21237c0153b080d372eeb4334771131c7baa3
https://hg.mozilla.org/releases/mozilla-beta/rev/cff6f339d416472248b2f346a11c57413ad6a8f4
https://hg.mozilla.org/releases/mozilla-beta/rev/3be328e257eb8d588d8887e22e548763a91b4a63

Comment 34

[Nick Alexander :nalexander [he/him]]

It surprises me that that diffoscope failures are causing backouts at all; that's new since I looked last. NI to me to follow-up at a time when I can think about this.

OK, this is expected, and in fact looks like it was landed as part of the original Win64 EME build: https://bugzilla.mozilla.org/show_bug.cgi?id=1534522. I'm not quite sure why these changes triggered precomplete differences, but since it doesn't seem to be an ongoing problem with WDBA landings, I'm not going to pursue it.

Comment 35

[Nick Alexander :nalexander [he/him]]

This feature is no longer confidential.