1499803 - Collect media Telemetry Scalar in Release version

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, enhancement)

Description

[Alastor Wu [:alwu]]

We want to collect these scalar [1] on Release channel, because the autoplay shield-study which will run on Release 63 needs one of these data and we also want to analysis other data in order to improve the user experience of blocking-autoplay.

[1] https://searchfox.org/mozilla-central/rev/eef79962ba73f7759fd74da658f6e5ceae0fc730/toolkit/components/telemetry/Scalars.yaml#913-1040

Comment 1

[Alastor Wu [:alwu]]

Attachment: Bug 1499803 - collect media Telemetry Scalar in Release version.

The autoplay shield-study which will run on Release 63 needs one of these data and we also want to analysis other data in order to improve the user experience of blocking-autoplay.

Comment 2

[Alastor Wu [:alwu]]

Attachment: Data review request

Comment 3

[Chris H-C :chutten]

Comment on attachment 9017989 [details]
Data review request

Preliminary note:
Generally I try not to do both the Telemetry Peer and Data Collection reviews for the same bug, but in this case the Telemetry Peer review was a rubberstamp for adding bug numbers and expanding collections to release.

Also, we need a list of all the measurements you're elevating to be collecting on all channels. Please take their names and descriptions and list them here in the bug.

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate? 

Yes. Standard Telemetry mechanisms apply.

    Is there a control mechanism that allows the user to turn the data collection on and off? 

Yes. Standard Telemetry mechanisms apply.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A expires in 68 (last version recorded: 67)

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

    Is the data collection request for default-on or default-off?

Default on, all channels.

    Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

    Is the data collection covered by the existing Firefox privacy notice? 

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :alwu will be responsible for renewing or removing these probes before they expire.

---
Result: datareview+, as soon as a full list of data collections is added to this bug.

Comment 4

[Alastor Wu [:alwu]]

We will collect following Telemetry Scalar in Release channel.

page_count:  The number of times a top level document is loaded.

page_had_media_count : The number of times a document hierarchy contained at least one audible HTMLMediaElement that had play() called upon it.

page_had_play_revoked_count : The number of times a document hierarchy contained at least one muted playing HTMLMediaElement that was paused due to becoming unmuted while not being allowed to autoplay.

autoplay_would_not_be_allowed_count : The number of HTMLMediaElement autoplays on audible HTMLMediaElements which would not be allowed to play if block autoplay was enabled; we'd either prompt for permission to play or block outright depending on user preferences.

allowed_autoplay_no_audio_track_count : The number of HTMLMediaElement autoplays on HTMLMediaElements without an audio track which would be allowed to play.

autoplay_would_be_allowed_count : The number of HTMLMediaElement autoplays on audible HTMLMediaElements that would have been allowed (not blocked) by the autoplay policy if block autoplay was enabled.

blocked_no_metadata : The number of HTMLMediaElements which we blocked because they hadn't loaded metadata yet.

blocked_no_metadata_endup_no_audio_track : The number of HTMLMediaElements which were blocked because they hadn't loaded metadata yet, but ended up having no audio track.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b0f3a71aca93
collect media Telemetry Scalar in Release version. r=chutten

Comment 6

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/b0f3a71aca93

Comment 7

[Alastor Wu [:alwu]]

Comment on attachment 9017976 [details]
Bug 1499803 - collect media Telemetry Scalar in Release version.

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1475099

User impact if declined: We can't collect correct data for shield study

Is this code covered by automated tests?: No

Has the fix been verified in Nightly?: No

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: No

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): this change is to just make some telemetry scalar observable in Release channel

String changes made/needed: no

Comment 8

[Pascal Chevrel:pascalc]

Comment on attachment 9017976 [details]
Bug 1499803 - collect media Telemetry Scalar in Release version.

Approved for 63 RC2 as this is a blocker to the shield study. I will add this bug as an item to our 63 postmortem as it seems to me that this is a problem that should have been detected and uplifted weeks ago and not in our RC week.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/84219fbf133cacfc6e31c9471ad20ee7162a02af

Yes, this feels like something we could & should have known about well before Wednesday of RC week.

Comment 10

[Alastor Wu [:alwu]]

(In reply to Pascal Chevrel:pascalc from comment #8)
> Comment on attachment 9017976 [details]
> Bug 1499803 - collect media Telemetry Scalar in Release version.
> 
> Approved for 63 RC2 as this is a blocker to the shield study. I will add
> this bug as an item to our 63 postmortem as it seems to me that this is a
> problem that should have been detected and uplifted weeks ago and not in our
> RC week.

Sorry for inconvenience, we didn't notice this error until QA reported to us that they found this issue on the Release 63 candidate.