Closed Bug 1822395 Opened 1 year ago Closed 1 year ago

Telemetry for new PiP media controls

Categories

(Toolkit :: Picture-in-Picture, enhancement, P1)

enhancement

Tracking

()

VERIFIED FIXED
113 Branch
Tracking Status
firefox113 --- verified

People

(Reporter: asafko, Assigned: niklas)

References

Details

(Whiteboard: [fidefe-pip3])

Attachments

(3 files, 2 obsolete files)

Hi Chris,
We are adding some new telemetry for the new Picture-in-Picture rollout in Fx 113. Will you please take a look at the proposed probes to see if there are any concerns around them?

Thank you!
Ania

Flags: needinfo?(chutten)
Attachment #9323340 - Flags: data-review?(chutten)
Whiteboard: [fidefe-pip3]
Assignee: nobody → nbaumgardner
Status: NEW → ASSIGNED

Updated the request with small changes to event descriptions to disambiguate how we are going to track them.

Attachment #9323995 - Flags: data-review?(chutten)

Marked two older files as obsoletes to avoid the confusion.

Attachment #9323340 - Attachment is obsolete: true
Attachment #9323995 - Attachment is obsolete: true
Attachment #9323340 - Flags: data-review?(chutten)
Attachment #9323995 - Flags: data-review?(chutten)
Attachment #9324020 - Flags: data-review?(chutten)

Comment on attachment 9324020 [details]
PiP Data Collection Request - Probes for Fx113_updated1.pdf

PRELIMINARY NOTES:
In future, please attach the data collection review request as plain text (it works better for Data Steward workflows)

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Ania Safko is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.


Result: datareview+

Flags: needinfo?(chutten)
Attachment #9324020 - Flags: data-review?(chutten) → data-review+
Severity: -- → S3
Priority: -- → P1
Attachment #9324072 - Attachment description: WIP: Bug 1822395 - Telemetry for new PiP controls. r=#pip-reviewers,cmkm → Bug 1822395 - Telemetry for new PiP controls. r=#pip-reviewers,cmkm
Attachment #9324072 - Attachment description: Bug 1822395 - Telemetry for new PiP controls. r=#pip-reviewers,cmkm → Bug 1822395 - Telemetry for new PiP controls. r=#pip-reviewers

Chris, we made a few smallish updates along the way (renamed a new event and corrected the name of a few objects for existing events to disambiguate them), so I'm going to request another data review.
Please let Niklas and me know if we shouldn't request reviews for housekeeping items like this one.

Flags: needinfo?(chutten)
Attachment #9326026 - Flags: data-review?(chutten)
Pushed by nbaumgardner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7776b8a21884
Telemetry for new PiP controls. r=mconley

Backed out for causing bc failures in browser_improved_controls.js

  • Backout link
  • Push with failures
  • Failure Log
  • Failure line: TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_improved_controls.js | Uncaught exception in test bound testVideoScrubber - Waiting for one create pictureinpicture telemetry event. - timed out after 100 tries.
Flags: needinfo?(nbaumgardner)

Comment on attachment 9326026 [details]
PiP Data Collection Request - Probes for Fx113 - amended.txt

PRELIMINARY NOTES:

Generally speaking, the specific name and distribution of data amongst the collection is more important for wayfinding than for review. If someone wants to know why we're collecting something named X, then using the correct X in the review request is helpful. ...but Stewards don't care about it as much as the data category, the population, and that it can be turned off.

Put another way: use your judgment. And when in doubt, do what you did and file a fresh review. Or ask us: you can find us here: https://chat.mozilla.org/#/room/#data-stewards:mozilla.org

Also, on a different matter, it appears as though your Q5 and Q6 answers overlapped each other. I'm confident I've been able to tease them apart, but next time keep an eye out. (and if your collection uses Glean, be sure to use ./mach data-review)

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through Firefox's Preferences.
Yes. This collection can be controlled through the product's preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Ania Safko is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.


Result: datareview+

Flags: needinfo?(chutten)
Attachment #9326026 - Flags: data-review?(chutten) → data-review+
Pushed by nbaumgardner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a3aaa9226d1a
Telemetry for new PiP controls. r=mconley

Backed out for causing bc failures on browser_telemetry_enhancements.js.

[task 2023-04-03T23:09:17.765Z] 23:09:17     INFO - TEST-PASS | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Video should be showing the expected state. - true == true - 
[task 2023-04-03T23:09:17.765Z] 23:09:17     INFO - Leaving test bound test_fullscreen_events
[task 2023-04-03T23:09:17.766Z] 23:09:17     INFO - Entering test bound test_seek_forward_and_backward_events
[task 2023-04-03T23:09:17.766Z] 23:09:17     INFO - Waiting for videos to be ready
[task 2023-04-03T23:09:17.766Z] 23:09:17     INFO - TEST-PASS | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Got Picture-in-Picture window. - 
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - Buffered messages finished
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Test timed out - 
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - GECKO(2780) | MEMORY STAT | vsize 8307MB | residentFast 429MB | heapAllocated 150MB
[task 2023-04-03T23:09:17.767Z] 23:09:17     INFO - TEST-OK | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | took 45280ms
[task 2023-04-03T23:09:17.768Z] 23:09:17     INFO - Not taking screenshot here: see the one that was previously logged
[task 2023-04-03T23:09:17.768Z] 23:09:17     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Found a tab after previous test timed out: http://example.com/browser/toolkit/components/pictureinpicture/tests/test-page.html - 
[task 2023-04-03T23:09:17.768Z] 23:09:17     INFO - checking window state
[task 2023-04-03T23:09:17.769Z] 23:09:17     INFO - Not taking screenshot here: see the one that was previously logged
[task 2023-04-03T23:09:17.769Z] 23:09:17     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_telemetry_enhancements.js | Found a Toolkit:PictureInPicture after previous test timed out - 
[task 2023-04-03T23:09:17.769Z] 23:09:17     INFO - GECKO(2780) | must wait for focus
[task 2023-04-03T23:09:17.770Z] 23:09:17     INFO - TEST-START | toolkit/components/pictureinpicture/tests/browser_text_tracks_webvtt_1.js
Flags: needinfo?(nbaumgardner)
Flags: needinfo?(nbaumgardner)
Pushed by nbaumgardner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/772cdfedb735
Telemetry for new PiP controls. r=mconley
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 113 Branch
Flags: needinfo?(nbaumgardner)
Flags: qe-verify+

This implementation has been verified in Beta v113.0b1 through to Beta v113.0b4 in Windows 10, Mac OS 12, Ubuntu 22 and Windows 7.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
See Also: → 1832455
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: