1519176 - Telemetry for language translation feature

Status: RESOLVED FIXED

(Firefox for iOS :: Telemetry, enhancement, P1)

Description

[Justin D'Arcangelo [:justindarc]]

We need to add some simple telemetry probes around the language translation feature:

• State of "Offer Translation" setting (on/off)
• Translation engine selection (Google Translate/Bing)

Also, add UI event telemetry for when an offer to translate is presented (Yes/No).

Comment 1

[Justin D'Arcangelo [:justindarc]]

Attachment: GitHub Wiki Revision

Request for data collection review form

1. What questions will you answer with this data?

• How often are users being prompted if they want to translate webpages?
• How often are users accepting offers to translate webpages?
• How often are users declining offers to translate webpages?
• How many users are enabling the feature that prompts them if they want to translate webpages?

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?:

• We are trying to gauge how much more time and attention to invest in refining this new feature.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

• There are no alternate ways to determine how popular this feature is and how it is being used.

4. Can current instrumentation answer these questions?

• No. This is a brand new feature.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description | Data Collection Category | Tracking Bug #

Prompt to translate tab to user's native language | Category 2 | Bug 1519176
Accept offer to translate tab | Category 2 | Bug 1519176
Decline offer to translate tab | Category 2 | Bug 1519176
Toggle setting to offer to translate tabs | Category 2 | Bug 1519176

6. How long will this data be collected?:

• I want to monitor this data for at least six months. (:mpopova)

7. What populations will you measure?

• The entire Firefox for iOS population (Beta and Release)

8. Please provide a general description of how you will analyze this data.

• We will determine what percentage and raw total number of our users are enabling this feature in Settings.
• We will also query percentages and raw total numbers of our users that are being prompted an offer to translate along with what percent of them are accepting/declining the offer.

9. Where do you intend to share the results of your analysis?

• Through a dashboard on sql.telemetry.mozilla.org

Comment 2

[Chenxia Liu [:liuche]]

Comment on attachment 9035736 [details]
GitHub Wiki Revision

1. Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?
   Documentation is part of "Translation" section on the firefox ios wiki: https://github.com/mozilla-mobile/firefox-ios/wiki/Mozilla's-Telemetry-Service-Pings#Translation

2. Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.
   Yes, Firefox ios data collection settings

3. If the request is for permanent data collection, is there someone who will monitor the data over time?**
   :mpopova will monitor for 6mo

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under? **
   Type 2

5. Is the data collection request for default-on or default-off?
   Default on for beta and release

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?
   No - just interactions with translations UI. Also does not actually collect the translation agent (Google, Bing)

7. Is the data collection covered by the existing Firefox privacy notice? If unsure: escalate to legal if:
   Yes, interaction data only

8. Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**
   Will review for renewal with Mobile data review

Comment 3

[Justin D'Arcangelo [:justindarc]]

Landed on master:

https://github.com/mozilla-mobile/firefox-ios/commit/751f334dc18f33c1c7ad9c56bf523be048aaf11c