1734414 - Remove media autoplay pref settings from our profiles.

Status: RESOLVED FIXED

(Testing :: Raptor, task, P2)

Description

[Greg Mierzwinski [:sparky]]

This bug is for removing/disabling the media.autoplay.* settings in our testing/profiles/common/user.js file.

This depends on an update in browsertime which overrides our settings: https://github.com/sitespeedio/browsertime/pull/1650

It's unlikely that we can disable this for all of our tests, but pageload tests should have it disabled.

Furthermore, note that the profile these are in are used in ALL Firefox tests so we can't make the modifications in the aforementioned file - instead we should disable them through our perf profiles.

Comment 2

[Kimberly Sereduck :kimberlythegeek]

copying from bug 1734431 , from a discussion in the perf team meeting:

About three years ago we enabled videos to autoplay, but it's been debated that autoplay should be disabled for performance tests as it is highly problematic for vismet tests, and affects our checks if page is fully loaded or not in our page load recordings

https://hg.mozilla.org/mozreview/gecko/rev/913e12413e3015b075989348d7ef73e241aafedc#index_headert

setting the media.autoplay.default value to 5 will also disable autoplay of videos without audio (such as ads):
https://searchfox.org/mozilla-central/source/testing/profiles/common/user.js#64

should we disable, or enable conditionally on some tests?

:jesup

videos in recordings are highly problematic for vismet tests (namely imgur)
suggestion is to disable autoplay for recordings--possibly experiment with disabling “painting” of videos

:bas

only reason it is enabled is some unit tests require this (but not relevant to performance tests) -- tests should be representative of default behavior
If we disable in performance tests we should also disable in chrome

We need to run comparisons of tests involving videos, imgur and cnn to name a couple, with autoplay disabled and autoplay enabled

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Greg Mierzwinski [:sparky] from comment #0)

Furthermore, note that the profile these are in are used in ALL Firefox tests so we can't make the modifications in the aforementioned file - instead we should disable them through our perf profiles.

Rather than having conflicting preferences, if any preferences are not common to all test harnesses it would make sense that they are specified only where they are needed. For example if these are needed for unittests, we should remove them from common and add them to a profile only used by unittests. Having "common" preferences that need to be changed in more specific scenarios can cause confusion and increases risk of deviating from default values.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

these were intended to be for unittests, and when this was designed we only had talos. I would rather make an exception for browsertime, than edit a bunch of preference files for all the other harnesses.

Comment 5

[Greg Mierzwinski [:sparky]]

I agree with that. I think to start we should explicitly disable it in our pageload tests through the manifest pref settings and expand the change from there to any benchmarks, scenario, and interactive tests that can accept this change. After this, we can expand it to our other perftest harnesses.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Joel Maher ( :jmaher ) (UTC -0800) from comment #4)

these were intended to be for unittests, and when this was designed we only had talos. I would rather make an exception for browsertime, than edit a bunch of preference files for all the other harnesses.

Wouldn't we only need to move the unittest specific preferences into their own profile? Then all other profiles would use whatever the default value for theses preferences is. Alternatively, could we remove "common" from use in performance tests? My concern is that when we want to reset a "common" preference to the default, we need to explicitly set it in another profile template. As defaults have the potential to change, any that have been hardcoded will deviate from our users experience of Firefox.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

I would make a perf-common.js for preferences. I am not opposed to adjusting common prefs, but if this is for 1 suite, I would prefer we modify in that suite or create a new common.js. Our unittests need to have a lot of preferences set in order to meet the needs of the tests and to ensure repeatability.

I think we are talking about the same things, send a patch along and ask for #releng-reviewers to get review from releng (which is the former #cia team).

Comment 8

[Alex Finder]

Attachment: Bug 1734414 - Remove media autoplay pref settings from our profiles. r=#perftest

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e8746dd07887
Remove media autoplay pref settings from our profiles. r=perftest-reviewers,kimberlythegeek,AlexandruIonescu

Comment 10

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/e8746dd07887

Comment 11

[Cristina Cozmuta (:CrissCozmuta)]

Backed out as per dev request. CLOSED TREE
Backout link : https://hg.mozilla.org/integration/autoland/rev/e4d4dd3872c727c49d93cbddf3448661615038b7

Comment 12

[Iulian Moraru]

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/e4d4dd3872c7

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e0c7e7f0f1de
Remove media autoplay pref settings from our profiles. r=perftest-reviewers,kimberlythegeek,AlexandruIonescu

Comment 14

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/e0c7e7f0f1de