1742484 - profile preferences (e.g. testing/profiles/web-platform/user.js) are not robust against "temporary" .ini manifest file prefs

Status: NEW

(Testing :: web-platform-tests, defect)

Description

[Karl Tomlinson (:karlt)]

When a pref is set in a manifest file for one test, it gets cleared for subsequent tests, instead of being restored to the value from the profile.
This produces different results for subsequent tests according to whether or not a previous test is run in the same process.

e.g. "Clearing pref media.navigator.permission.disabled" after /mediacapture-streams/GUM-deny.https.html instead of restoring.
https://treeherder.mozilla.org/logviewer?job_id=358587355&repo=try&lineNumber=24842

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:jgraham, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 2

[James Graham [:jgraham]]

Looking at the code it seems like we're only clearing the pref when getting it returned None: https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py#776-787. I think we should be resetting the original_pref_values dictionary for each new environment, but I'm not sure this should make a difference; each time we set the environment we restore/reset the values that were set in the previous environment, and store the values that are currently being set. So I think we always have the correct values in the cache.