1476088 - Report additional submissionURLs in search service

Status: VERIFIED FIXED

(Firefox :: Search, defect, P1)

Description

[Mike Kaply [:mkaply]]

Currently we only report search submission URLs for built in search engines.

I'm proposing that we expand this list to include any URLs that have the same domain as our built in engines.

This allows us to get more information about search hijacking without impacing user privacy.

Comment 1

[Mike Kaply [:mkaply]]

Attachment: Bug 1476088 - Additional search submission telemetry.

Comment 2

[Mike Kaply [:mkaply]]

Attachment: Data Collection Review Request

Data collection review request.

Comment 3

[François Marier [:francois]]

Comment on attachment 8996116 [details]
Data Collection Review Request

Escalating to Marshall since I believe that the data falls within the Web activity category AND is default-on.

Marshall, the configured search URLs we are interested in (hijacked by malware/crapware) might be considered Category 1 since it's part of the environment and wasn't configured by the user. However, for legitimate non-default search engines, this feels like Category 3 to me since it reveals a custom URL that a user chose to put in there.

Comment 4

[Merwin]

Mike Kaply and I spoke about this briefly...Mike, if I recall correctly, you were going use a whitelist such that the information you are getting about non-default engines is very limited.  Correct?

Comment 5

[Mike Kaply [:mkaply]]

> Mike Kaply and I spoke about this briefly...Mike, if I recall correctly, you were going use a whitelist such that the information you are getting about non-default engines is very limited.  Correct?

Yes, you can see the update patch here:

https://phabricator.services.mozilla.com/D2165

We only send submission URLs for engines we ship or for common search engines.

And they are only sent at startup as part of the default engine information.

Comment 6

[Merwin]

Given that, I think this is ok.  This is category 3 but satisfies the policy requirement that technical mechanisms mitigate the risk. Practically speaking, the whitelist makes this essentially the same as our current approach.

Comment 7

[François Marier [:francois]]

Comment on attachment 8996116 [details]
Data Collection Review Request

> 5) List all proposed measurements and indicate the category of data collection
> for each measurement, using the Firefox
> [data collection categories](https://wiki.mozilla.org/Firefox/Data_Collection) on the Mozilla wiki.   
> 
> Search submission URL (URL that would be sent to the search service with NO
> search term in it)

Could you please add a description of the filtering that you allude to in comment 5? That way the data review request can stand on its own.

Other than that, I have all of the info I need to do the data review.

Comment 8

[Mike Kaply [:mkaply]]

Attachment: Updated request with domains called out

Update with domains called out.

Comment 9

[François Marier [:francois]]

Sorry, I forgot something else. Is the schema of this data collection documented publicly anywhere? It seems like this is not a standard telemetry probe.

I would ideally like to link to that documentation in my answer to the first question on https://github.com/mozilla/data-review/blob/master/review.md.

Comment 10

[Mike Kaply [:mkaply]]

It's sent via the telemetry ping as part of the environment.

See:

https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html

In particular, the settings section:

    defaultSearchEngineData: {, // data about the current default engine
      name: <string>, // engine name, e.g. "Yahoo"; or "NONE" if no default
      loadPath: <string>, // where the engine line is located; missing if no default
      origin: <string>, // 'default', 'verified', 'unverified', or 'invalid'; based on the presence and validity of the engine's loadPath verification hash.
      submissionURL: <string> // missing if no default or for user-installed engines
    },

Comment 11

[François Marier [:francois]]

Comment on attachment 8996736 [details]
Updated request with domains called out

1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes, at https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html in the "defaultSearchEngineData" section under "settings".

2) Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, telemetry setting.

3) If the request is for permanent data collection, is there someone who will monitor the data over time?**

Yes, Michael Kaply.

4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under?  **

Category 1 given the technical mitigations to whitelist search provider URLs.

5) Is the data collection request for default-on or default-off?

Default on.

6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc.  See the appendix for more details)?

No.

7) Is the data collection covered by the existing Firefox privacy notice?

Yes.

8) Does there need to be a check-in in the future to determine whether to renew the data?

No, permanent.

Comment 12

[Mike Kaply [:mkaply]]

[Tracking Requested - why for this release]: Additional data for addressing hijacking.

Comment 13

[Phabricator Automation]

Comment on attachment 8992434 [details]
Bug 1476088 - Additional search submission telemetry.

Florian Quèze [:florian] has approved the revision.

https://phabricator.services.mozilla.com/D2165

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fb85f9a81670
Additional search submission telemetry. r=florian

Comment 15

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/fb85f9a81670

Comment 16

[Mike Kaply [:mkaply]]

Comment on attachment 8992434 [details]
Bug 1476088 - Additional search submission telemetry.

Approval Request Comment
[Feature/Bug causing the regression]: Add additional submission URLs to get a handle on hijacking.
[User impact if declined]: None
[Is this code covered by automated tests?]: Yes
[Has the fix been verified in Nightly?]: No
[Needs manual test from QE? If yes, steps to reproduce]: Go to yahoo.com and add Yahoo as a search engine and set it as the default. You do this by clicking the three dots dropdown and adding the engine, then right clicking on the engine and then selecting "set as default".
Restart the browser.
Go to about:telemetry and search on defaultSearchEngineData.submissionURL
Verify that there is a URL present (without this patch, it is empty)
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: Very low.
[Why is the change risky/not risky?]: Telemetry reporting only, not user facing
[String changes made/needed]:

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8992434 [details]
Bug 1476088 - Additional search submission telemetry.

Adds additional Telemetry around search submissions. Includes new automated test coverage. Approved for 62.0b16 and 61.0.2.

Comment 18

[Mike Kaply [:mkaply]]

Attachment: 1476088-release.patch

Rebased release patch (hg export)

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/dad476171a76

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/124fc1391603

Comment 21

[Ciprian Georgiu, Desktop QA]

I have reproduced this bug on an affected Nightly build (2018-07-16), and using the STR from comment 16.

I can confirm that the URL "https://ro.search.yahoo.com/search?p=&fr=opensearch" is present in about:telemetry, on the latest Builds tested: Nightly 63.0a1 (20180807220134), Beta 62.0b16 (20180808015758) and dot Release 61.0.2 (20180807170231). Verified on Win 10 x64, Mac OS X 10.9 and Ubuntu 16.04 x64.