1822395 - Telemetry for new PiP media controls

Status: VERIFIED FIXED

(Toolkit :: Picture-in-Picture, enhancement, P1)

Description

[Ania]

New media controls: PiP telemetry requirements

Comment 1

[Ania]

Attachment: PiP Data Collection Request - Probes for Fx113.pdf

Hi Chris,
We are adding some new telemetry for the new Picture-in-Picture rollout in Fx 113. Will you please take a look at the proposed probes to see if there are any concerns around them?

Thank you!
Ania

Comment 2

[Ania]

Attachment: PiP Data Collection Request - Probes for Fx113-updated.pdf

Updated the request with small changes to event descriptions to disambiguate how we are going to track them.

Comment 3

[Ania]

Attachment: PiP Data Collection Request - Probes for Fx113_updated1.pdf

Marked two older files as obsoletes to avoid the confusion.

Comment 4

[Chris H-C :chutten]

Comment on attachment 9324020 [details]
PiP Data Collection Request - Probes for Fx113_updated1.pdf

PRELIMINARY NOTES:
In future, please attach the data collection review request as plain text (it works better for Data Steward workflows)

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Ania Safko is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+

Comment 5

[Niklas Baumgardner [:niklas]]

Attachment: Bug 1822395 - Telemetry for new PiP controls. r=#pip-reviewers

Comment 6

[Ania]

Chris, we made a few smallish updates along the way (renamed a new event and corrected the name of a few objects for existing events to disambiguate them), so I'm going to request another data review.
Please let Niklas and me know if we shouldn't request reviews for housekeeping items like this one.

Comment 7

[Ania]

Attachment: PiP Data Collection Request - Probes for Fx113 - amended.txt

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7776b8a21884
Telemetry for new PiP controls. r=mconley

Comment 9

[Cristian Tuns]

Backed out for causing bc failures in browser_improved_controls.js

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_improved_controls.js | Uncaught exception in test bound testVideoScrubber - Waiting for one create pictureinpicture telemetry event. - timed out after 100 tries.

Comment 10

[Chris H-C :chutten]

Comment on attachment 9326026 [details]
PiP Data Collection Request - Probes for Fx113 - amended.txt

PRELIMINARY NOTES:

Generally speaking, the specific name and distribution of data amongst the collection is more important for wayfinding than for review. If someone wants to know why we're collecting something named X, then using the correct X in the review request is helpful. ...but Stewards don't care about it as much as the data category, the population, and that it can be turned off.

Put another way: use your judgment. And when in doubt, do what you did and file a fresh review. Or ask us: you can find us here: https://chat.mozilla.org/#/room/#data-stewards:mozilla.org

Also, on a different matter, it appears as though your Q5 and Q6 answers overlapped each other. I'm confident I've been able to tease them apart, but next time keep an eye out. (and if your collection uses Glean, be sure to use ./mach data-review)

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through Firefox's Preferences.
Yes. This collection can be controlled through the product's preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Ania Safko is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a3aaa9226d1a
Telemetry for new PiP controls. r=mconley

Comment 12

[Iulian Moraru]

Backed out for causing bc failures on browser_telemetry_enhancements.js.

• Push with failures

• Failure log

• Backout link

[task 2023-04-03T23:09:17.765Z] 23:09:17     INFO - TEST-PASS | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Video should be showing the expected state. - true == true - 
[task 2023-04-03T23:09:17.765Z] 23:09:17     INFO - Leaving test bound test_fullscreen_events
[task 2023-04-03T23:09:17.766Z] 23:09:17     INFO - Entering test bound test_seek_forward_and_backward_events
[task 2023-04-03T23:09:17.766Z] 23:09:17     INFO - Waiting for videos to be ready
[task 2023-04-03T23:09:17.766Z] 23:09:17     INFO - TEST-PASS | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Got Picture-in-Picture window. - 
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - Buffered messages finished
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Test timed out - 
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - GECKO(2780) | MEMORY STAT | vsize 8307MB | residentFast 429MB | heapAllocated 150MB
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - TEST-OK | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | took 45280ms
[task 2023-04-03T23:09:17.768Z] 23:09:17     INFO - Not taking screenshot here: see the one that was previously logged
[task 2023-04-03T23:09:17.768Z] 23:09:17     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Found a tab after previous test timed out: http://example.com/browser/toolkit/components/pictureinpicture/tests/test-page.html - 
[task 2023-04-03T23:09:17.768Z] 23:09:17     INFO - checking window state
[task 2023-04-03T23:09:17.769Z] 23:09:17     INFO - Not taking screenshot here: see the one that was previously logged
[task 2023-04-03T23:09:17.769Z] 23:09:17     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Found a Toolkit:PictureInPicture after previous test timed out - 
[task 2023-04-03T23:09:17.769Z] 23:09:17     INFO - GECKO(2780) | must wait for focus
[task 2023-04-03T23:09:17.770Z] 23:09:17     INFO - TEST-START | toolkit/components/pictureinpicture/tests/browser_text_tracks_webvtt_1.js

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/772cdfedb735
Telemetry for new PiP controls. r=mconley

Comment 14

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/772cdfedb735

Comment 15

[Daniel Bodea [:danibodea]]

This implementation has been verified in Beta v113.0b1 through to Beta v113.0b4 in Windows 10, Mac OS 12, Ubuntu 22 and Windows 7.