1692064 - Add Histogram Telemetry on main UI localization times

Status: RESOLVED FIXED

(Core :: Internationalization: Localization, task)

Description

[Zibi Braniecki [:zbraniecki][:gandalf]]

To validate the claim that bug 1660391 makes about switching Fluent in Gecko to Rust,
we'd like to have a scalar probe that records the time document localization takes for:

• browser.xhtml (startup)
• browser.xhtml (new window)
• preferences.xhtml
• about:home
• about:newtab

Comment 1

[Zibi Braniecki [:zbraniecki][:gandalf]]

Attachment: Bug 1692064 - Add Histogram Telemetry on main UI localization times.

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

Attachment: bug1692064_telemetry_data_review.txt

Chris - does the Histogram here seem good?

Comment 3

[Chris H-C :chutten]

Comment on attachment 9202494 [details]
bug1692064_telemetry_data_review.txt

Load-balance to :sancus

Comment 4

[Andrei Hajdukewycz [:sancus]]

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #2)

Created attachment 9202494 [details]
bug1692064_telemetry_data_review.txt

This all seems fine to me, but if the intent is to revisit the probe at the end of the year, why not set an expiry of Firefox 99 or 100 instead of leaving it indefinite?

Comment 5

[Zibi Braniecki [:zbraniecki][:gandalf]]

Fair. Updated to "expires_in_version" "100".

Comment 6

[Andrei Hajdukewycz [:sancus]]

Comment on attachment 9202494 [details]
bug1692064_telemetry_data_review.txt

Data Review Form:

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in the histogram definition files and in the probe dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, using standard Telemetry preferences.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Data collection expires in Firefox 100.

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2(“Interaction data”).

5. Is the data collection request for default-on or default-off?

Default on for all channels.

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

7. Is the data collection covered by the existing Firefox privacy notice?

Yes.

8. Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No)

Yes, :gandalf or Platform i18n Team will decide whether to renew after Firefox 100.

9. Does the data collection use a third-party collection tool? If yes, escalate to legal.

No.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6ed0ffc2329d
Add Histogram Telemetry on main UI localization times. r=dminor

Comment 8

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 6ed0ffc2329d (bug 1692064) for test_telemetry.html failures.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=CQEhkW0FRhWr56LXs6dquQ.0&fromchange=6ed0ffc2329d3b659a7ff6883b4eacbd0cf7a42f&searchStr=mochitest-chrome-1proc&tochange=10109ab5a965600175792e658aa16ae6e453d1b6

Backout link: https://hg.mozilla.org/integration/autoland/rev/10109ab5a965600175792e658aa16ae6e453d1b6

Failure log: https://treeherder.mozilla.org/logviewer?job_id=329876677&repo=autoland&lineNumber=3148

[task 2021-02-13T11:47:21.615Z] 11:47:21     INFO - TEST-START | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html
[task 2021-02-13T11:47:21.645Z] 11:47:21     INFO - TEST-INFO | started process screentopng
[task 2021-02-13T11:47:21.808Z] 11:47:21     INFO - TEST-INFO | screentopng: exit 0
[task 2021-02-13T11:47:21.809Z] 11:47:21     INFO - TEST-UNEXPECTED-FAIL | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | undefined assertion name - got false, expected true
[task 2021-02-13T11:47:21.809Z] 11:47:21     INFO - SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:500:14
[task 2021-02-13T11:47:21.809Z] 11:47:21     INFO - @chrome://mochitests/content/chrome/dom/l10n/tests/mochitest/document_l10n/test_telemetry.html:25:9
[task 2021-02-13T11:47:21.809Z] 11:47:21     INFO - @chrome://mochitests/content/chrome/dom/l10n/tests/mochitest/document_l10n/test_telemetry.html:59:5
[task 2021-02-13T11:47:21.810Z] 11:47:21     INFO - GECKO(2113) | JavaScript error: chrome://mochitests/content/chrome/dom/l10n/tests/mochitest/document_l10n/test_telemetry.html, line 27: TypeError: can't access property "values", snapshot.browser_first_window is undefined
[task 2021-02-13T11:52:50.053Z] 11:52:50     INFO - Not taking screenshot here: see the one that was previously logged
[task 2021-02-13T11:52:50.054Z] 11:52:50     INFO - TEST-UNEXPECTED-FAIL | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | Test timed out. 
[task 2021-02-13T11:52:50.054Z] 11:52:50     INFO - SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:417:16
[task 2021-02-13T11:52:50.054Z] 11:52:50     INFO - reportError@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:143:22
[task 2021-02-13T11:52:50.054Z] 11:52:50     INFO - TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:165:18
[task 2021-02-13T11:52:51.058Z] 11:52:51     INFO - GECKO(2113) | MEMORY STAT | vsize 2775MB | residentFast 277MB | heapAllocated 94MB
[task 2021-02-13T11:52:51.058Z] 11:52:51     INFO - TEST-OK | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | took 329439ms

Comment 9

[Zibi Braniecki [:zbraniecki][:gandalf]]

All right, so we can't be sure if the main window loaded already by the time the test is run.

Relaxing the test to account for that.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/11c143ff4c3f
Add Histogram Telemetry on main UI localization times. r=dminor

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 11c143ff4c3f (bug 1692064) for test_telemetry.html failures.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=bD9XlalpTCG1f_btfSDbxA.0&fromchange=11c143ff4c3f961eda6001456e906809261c2ef8&searchStr=mochitest-chrome-1proc&tochange=ba6241feee313780d0fa0bb2489bec1b5458f5e0

Backout link: https://hg.mozilla.org/integration/autoland/rev/ba6241feee313780d0fa0bb2489bec1b5458f5e0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=329885159&repo=autoland&lineNumber=3147

[task 2021-02-13T12:55:31.647Z] 12:55:31     INFO - TEST-START | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html
[task 2021-02-13T12:55:31.870Z] 12:55:31     INFO - TEST-INFO | started process screentopng
[task 2021-02-13T12:55:32.193Z] 12:55:32     INFO - TEST-INFO | screentopng: exit 0
[task 2021-02-13T12:55:32.194Z] 12:55:32     INFO - Buffered messages logged at 12:55:31
[task 2021-02-13T12:55:32.195Z] 12:55:32     INFO - TEST-PASS | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | undefined assertion name 
[task 2021-02-13T12:55:32.195Z] 12:55:32     INFO - TEST-PASS | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | undefined assertion name 
[task 2021-02-13T12:55:32.196Z] 12:55:32     INFO - Buffered messages finished
[task 2021-02-13T12:55:32.197Z] 12:55:32     INFO - TEST-UNEXPECTED-FAIL | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | undefined assertion name - got +0, expected 1
[task 2021-02-13T12:55:32.197Z] 12:55:32     INFO - SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:500:14
[task 2021-02-13T12:55:32.197Z] 12:55:32     INFO - @chrome://mochitests/content/chrome/dom/l10n/tests/mochitest/document_l10n/test_telemetry.html:47:11
[task 2021-02-13T12:55:32.279Z] 12:55:32     INFO - Not taking screenshot here: see the one that was previously logged
[task 2021-02-13T12:55:32.281Z] 12:55:32     INFO - TEST-UNEXPECTED-FAIL | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | undefined assertion name - got +0, expected 1
[task 2021-02-13T12:55:32.281Z] 12:55:32     INFO - SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:500:14
[task 2021-02-13T12:55:32.282Z] 12:55:32     INFO - @chrome://mochitests/content/chrome/dom/l10n/tests/mochitest/document_l10n/test_telemetry.html:62:9
[task 2021-02-13T12:55:32.320Z] 12:55:32     INFO - GECKO(2119) | MEMORY STAT | vsize 2822MB | residentFast 338MB | heapAllocated 128MB
[task 2021-02-13T12:55:32.320Z] 12:55:32     INFO - TEST-OK | dom/l10n/tests/mochitest/document_l10n/test_telemetry.html | took 674ms

Comment 12

[Zibi Braniecki [:zbraniecki][:gandalf]]

Sorry for that.

Seems like --verify is a bit tricky to play around. Updating the test to be even more relaxed while still testing for our telemetry.

Comment 13

[Zibi Braniecki [:zbraniecki][:gandalf]]

try: https://treeherder.mozilla.org/jobs?repo=try&revision=05d66498aaaf5d95b0530faf7c1cfe1df91137e1

Comment 14

[Zibi Braniecki [:zbraniecki][:gandalf]]

Green try: https://treeherder.mozilla.org/jobs?repo=try&revision=3e19d9b9445e5e5765c4e7d58520ffa8683c7556

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/30c1f94d2f82
Add Histogram Telemetry on main UI localization times. r=dminor

Comment 16

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/30c1f94d2f82