Closed Bug 1567604 Opened 1 year ago Closed 11 months ago

Enable VIDEO_PLAY_TIME probe for release channel users as an opt-out probe after it is renewed in bug 1356046 .

Categories

(Core :: Audio/Video, task, P2)

task

Tracking

()

RESOLVED FIXED
mozilla70
Tracking Status
firefox70 --- fixed

People

(Reporter: mconley, Assigned: mconley)

References

Details

Attachments

(2 files)

Something that tdsmith, astevenson and I were talking about in Whistler was whether or not we can determine if Firefox users watch more video when the Picture-in-Picture feature is enabled.

I started digging into what it would take to add a Telemetry probe to measure video playback time, and was pleased to find out that such a probe already exists - it's just expired, and has been since Firefox 63.

This bug is about reviving that probe to help measure the impact (if any) Picture-in-Picture has on video playback time.

Blocks: 1527926

This only enables the probe on Firefox Desktop, and only for content processes. We're
reviving this probe mainly as a way of determining if Picture-in-Picture causes users
to watch more video.

Assignee: nobody → mconley

Check bug 1356046 instead.

However, tbh, I seriously doubt this probe will achieve what you describe. What would you be comparing things with? We have no comparison point and as telemetry isn't available by default in release, we have seen in the past that the numbers are so skewed towards one category of users that it makes any analysis impossible

I agree that determining that information would be great, but reviving that telemetry key alone won't do it.

See Also: → 1356046
Summary: Revive VIDEO_PLAY_TIME_MS probe → Add way to determine if PiP has an impact on total playback time

telemetry isn't available by default in release

It doesn't answer your critique, but we should also start collecting this in release if we're interested in it.

Note that probe renewal is subject to a new data-collection review.

Priority: -- → P2

However, tbh, I seriously doubt this probe will achieve what you describe. What would you be comparing things with? We have no comparison point and as telemetry isn't available by default in release, we have seen in the past that the numbers are so skewed towards one category of users that it makes any analysis impossible

Thanks, jya. One way we might do this is to try to break some part of either the Beta or Release populations into two groups, one with PiP enabled, and one without, to see whether or not that reveals a signal.

I'm glad to see that dminor is already working on renewing the probe in bug 1356046! Perhaps I'll repurpose this one for requesting a data-review to getting it enabled to release by default.

Summary: Add way to determine if PiP has an impact on total playback time → Enable VIDEO_PLAY_TIME probe for release channel users as an opt-out probe after it is renewed in bug 1356046 .
Depends on: 1356046

Yes lets wait for bug 1356046 to land.

From my point of view we want a playback length probe in Release (because otherwise the values are not reliable) - which could be either this or a new one (see below).

jya: is VIDEO_PLAY_TIME_MS good enough going forward or do want/need to update this probe (or create a new one) which records how many frames got rendered (taking into consideration the frame rate of the material)?

Flags: needinfo?(jyavenard)

how long things played is hopefully sufficient

Flags: needinfo?(jyavenard)
Attachment #9079494 - Attachment description: Bug 1567604 - Revive VIDEO_PLAY_TIME_MS Telemetry probe. data-review=chutten,r?jya → Bug 1567604 - Revive VIDEO_PLAY_TIME_MS Telemetry probe. data-review=chutten,r?dminor
Attached file Data review request
Attachment #9084814 - Flags: data-review?(chutten)
Comment on attachment 9084814 [details]
Data review request

Load balancing to :bmiroglio
Attachment #9084814 - Flags: data-review?(chutten) → data-review?(bmiroglio)
Comment on attachment 9084814 [details]
Data review request

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way? 
Data will be documented in the [probe dictionary](https://telemetry.mozilla.org/probe-dictionary/). 

2. Is there a control mechanism that allows the user to turn the data collection on and off? 
Yes, this can be turned off by disabling telemetry in the Firefox preferences. 

3. If the request is for permanent data collection, is there someone who will monitor the data over time?
Yes: mconley.

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Category 2 - Interaction Data

5. Is the data collection request for default-on or default-off?
Default on. 

6. Does the instrumentation include the addition of any new identifiers?
No new identifiers. 

7. Is the data collection covered by the existing Firefox privacy notice?
Yes.

8. Does there need to be a check-in in the future to determine whether to renew the data? 
No. 

9. Does the data collection use a third-party collection tool? 
No. 
—
Data review +
Attachment #9084814 - Flags: data-review?(bmiroglio) → data-review+
Attachment #9079494 - Attachment description: Bug 1567604 - Revive VIDEO_PLAY_TIME_MS Telemetry probe. data-review=chutten,r?dminor → Bug 1567604 - Make VIDEO_PLAY_TIME_MS Telemetry probe opt-out. data-review=bmiroglio,r?dminor
Depends on: 1570634
Pushed by mconley@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a5784d103862
Make VIDEO_PLAY_TIME_MS Telemetry probe opt-out. data-review=bmiroglio,r=dminor
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
You need to log in before you can comment on or make changes to this bug.