1751920 - Sanitize search engine values on ingestion

Status: RESOLVED FIXED

(Firefox for Android :: General, defect, P1)

Description

[Roger Yang [:royang]]

+++ This bug was initially created as a clone of Bug #1751753 +++

We need to sanitize the keys of the search_counts histogram during ingestion based on an allowlist of known engines.

Per :standard8:

From a desktop perspective, we’re going to have this list in remote settings (existing version using prefixes), and that also gets updated a few days after remote settings into the main repositories.

From a BQ perspective, the histogram exists in main_v4 under payload.keyed_histograms.search_counts, which is a key/value struct.

From the JSON perspective, the histogram would be paylod.keyedHistograms.SEARCH_COUNTS with potential casing differences we'll need to account for.

The structure of the histogram is documented in Histograms.json:

Records search counts for search access points and in-content searches. For search access points in general, the format is: <engine-name>.<search-access-point> For the urlbar when in search mode, the format is <engine name>.urlbar-searchmode For the urlbar when an internal @engine shortcut is used, the format is: <engine-name>.alias For in-content searches, the format is <provider>.in-content:[sap|sap-follow-on|organic]:[code|none]

Comment 1

[Roger Yang [:royang]]

Attachment: code_validate_1.patch

Comment 2

[Roger Yang [:royang]]

Attachment: code_validate_2.patch

Comment 3

[Roger Yang [:royang]]

Attachment: code_validate_3.patch

Comment 4

[Roger Yang [:royang]]

Attachment: code_validate_4.patch

Comment 5

[Jonathan Almeida [:jonalmeida]]

Comment on attachment 9260720 [details] [diff] [review]
code_validate_4.patch

Looks good. (corrections were based on test failures)

Comment 6

[Roger Yang [:royang]]

Patch landed https://github.com/mozilla-mobile/android-components/pull/11607

Comment 7

[Roger Yang [:royang]]

Updated patch landed in https://github.com/mozilla-mobile/android-components/pull/11621 and https://github.com/mozilla-mobile/android-components/pull/11622