1819336 - Collect Telemetry to compare success rate of regular image, video, audio loads and upgraded image, video, audio loads

Status: RESOLVED FIXED

(Core :: DOM: Security, task, P3)

Comment 1

[Christoph Kerschbaumer [:ckerschb]]

Attachment: Bug 1819336: Collect Telemetry to compare success rate of regular image loads and upgraded image loads, r=freddyb

Comment 3

[Christoph Kerschbaumer [:ckerschb]]

Attachment: data-review-1819336.txt

Comment 4

[Christoph Kerschbaumer [:ckerschb]]

Comment on attachment 9320284 [details]
Bug 1819336: Collect Telemetry to compare success rate of regular image loads and upgraded image loads, r=freddyb

Beta/Release Uplift Approval Request

• User impact if declined: No user impact. This patch is just collecting telemetry for image failure rates in regular browsing mode and auto-upgrading mode (http to https) because we want to run a study.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's just an if-else branch check for collecting telemetry.
• String changes made/needed: no
• Is Android affected?: Yes

Comment 5

[Christoph Kerschbaumer [:ckerschb]]

Attachment: data-review-1819336.txt

Comment 6

[Christoph Kerschbaumer [:ckerschb]]

Attachment: Bug 1819336: Collect Telemetry to compare success rate of regular image, video, audio loads and upgraded image, video, audio loads, r=freddyb

Comment 7

[Chris H-C :chutten]

Comment on attachment 9320764 [details]
data-review-1819336.txt

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, ckerschbaumer and freddyb are responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9a955a5bcfc2
Collect Telemetry to compare success rate of regular image, video, audio loads and upgraded image, video, audio loads, r=freddyb

Comment 9

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/9a955a5bcfc2

Comment 10

[Frederik Braun [:freddy]]

Comment on attachment 9320771 [details]
Bug 1819336: Collect Telemetry to compare success rate of regular image, video, audio loads and upgraded image, video, audio loads, r=freddyb

Beta/Release Uplift Approval Request

• User impact if declined: No user impact. This patch is just collecting telemetry for failure rates in regular browsing mode and auto-upgrading mode (http to https) because we want to run a study that requires flipping a pref and this new telemetry probe.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's just an if-else branch check for collecting telemetry.
• String changes made/needed: None
• Is Android affected?: Yes

Comment 11

[Donal Meehan [:dmeehan]]

Comment on attachment 9320771 [details]
Bug 1819336: Collect Telemetry to compare success rate of regular image, video, audio loads and upgraded image, video, audio loads, r=freddyb

Clearing the beta request flag, this patch landed in 112 which merges to beta today.

Comment 12

[Alex Finder]

(In reply to Pulsebot from comment #8)

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9a955a5bcfc2
Collect Telemetry to compare success rate of regular image, video, audio
loads and upgraded image, video, audio loads, r=freddyb

== Change summary for alert #37456 (as of Thu, 02 Mar 2023 18:41:12 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
5%	reddit fcp	linux1804-64-shippable-qr	fission warm webrender	171.62 -> 162.88	Before/After
5%	cnn SpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	3,397.38 -> 3,228.50
5%	cnn FirstVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	2,390.08 -> 2,278.08
5%	cnn SpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	2,493.58 -> 2,377.17
5%	cnn PerceptualSpeedIndex	android-hw-a51-11-0-aarch64-shippable-qr	warm webrender	2,466.00 -> 2,353.33
...	...	...	...	...	...
4%	cnn FirstVisualChange	android-hw-a51-11-0-aarch64-shippable-qr	cold webrender	2,972.18 -> 2,847.67

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=37456

Comment 13

[Frederik Braun [:freddy]]

As much as I would like that to be true, I won't believe that adding a new telemetry probe will cause a performance improvement.

Comment 14

[Alex Finder]

(In reply to Frederik Braun [:freddy] from comment #13)

As much as I would like that to be true, I won't believe that adding a new telemetry probe will cause a performance improvement.

Hi Frederik! Sorry for the confusion, it seems I pasted the alert summary on the wrong culprit. I marked the comment as obsolete. Thanks for the ni!

Comment 15

[Donal Meehan [:dmeehan]]

Comment on attachment 9320771 [details]
Bug 1819336: Collect Telemetry to compare success rate of regular image, video, audio loads and upgraded image, video, audio loads, r=freddyb

Approved for 111.0.1

Comment 16

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-release/rev/2ec69ea6ee93