1129713 - Preferences defined via --pref or --preferences are enforced at each application restart

Status: NEW

(Testing :: Mozbase, defect, P3)

Description

[Martin Kimmerle]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 Iceweasel/31.4.0
Build ID: 20150113100542

Steps to reproduce:

Steps to reproduce:
(1) Start `mozrunner --pref=a:val`
(2) in `about:config`, set "a" to "new-val"
(3) Press Shift-F2, type "restart", press enter.
(4) check the "a" property in `about:config`


Actual results:

After restarting, "a" is reset to "val".

Note that changed or added prefs will persist, provided that they have not been defined by --pref nor --preferences.


Expected results:

I expect that the "a" pref does not change at a restart, i.e. it should not be set back.

I'm not sure whether this is a bug, it might be intended behaviour.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Hi Martin. Thanks for your report! I had a quick look into mozprofile and I think the problematic line is here:

https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/mozprofile/profile.py#96
https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/mozprofile/profile.py#178

We are creating also a user.js file, which entries will overwrite any preference set in prefs.js after a restart of the application.

I don't see why we have to create the user.js file here. Especially for each of the preferences we specify.

I can verify this behavior with our Firefox UI tests under Marionette like:

    def test_pref_after_restart(self):
        print self.prefs.get_pref('app.update.enabled')
        self.prefs.set_pref('app.update.enabled', True)
        print self.prefs.get_pref('app.update.enabled')
        self.marionette.restart()
        print self.prefs.get_pref('app.update.enabled')

We set app.update.enabled via the default prefs in Marionette, but after a restart the custom value has been reset. This behavior may block my work on bug 1129843.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Andrew, do you know why we are setting all the preferences also in the user.js file?

Comment 3

[Andrew Halberstadt [:ahal]]

The prefs module in firefox-ui-tests has nothing to do with mozprofile, it sets the prefs via javascript API.

I'm not sure what you mean by "also" in user.js? user.js is the correct place for user defined prefs to go, otherwise they will be overwritten (prefs.js is automatically generated by the application and isn't meant to be edited manually). See:
https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/A_brief_guide_to_Mozilla_preferences

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Andrew, I think you misunderstood the former comments, and miss a somewhat critical part for user.js! So at first I would suggest you read the section about "Saving Preferences" in the article you linked above. Especially the following sentence is the problematic part for tests:

"Note the application never changes user.js, so on launch user.js overrides conflicting preferences from the previous application session."

With that behavior in place tests won't be able to change a preference, which remains it value through an application restart! Here one example... There are a couple of logging prefs, which need a restart to be enabled or disabled. So changing this value in a Firefox session and restarting the browser will cause a preference reset, because the value from the user.js file is taken again. That means as result the test will never be able to enable or disable the logging!

This will not happen if a prefs.js file is used. And its totally valid to create such one for tests. We are not talking about admin privileges here and restrictions for users. We have automated tests and would need the flexibility to change preference values which survive a restart of the application.

Comment 5

[Andrew Halberstadt [:ahal]]

Ok, it sounds like you understand the problem, I'm not quite sure what you're needinfo'ing me for? You can pass "prefs.js" into profile.set_preferences(), you'll just need to make a profile subclass that initializes it that way instead of via user.js.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrew Halberstadt [:ahal] from comment #5)
> Ok, it sounds like you understand the problem, I'm not quite sure what
> you're needinfo'ing me for? You can pass "prefs.js" into
> profile.set_preferences(), you'll just need to make a profile subclass that
> initializes it that way instead of via user.js.

The question here is more what should be the default behavior. Using user.js by default will give your tests really no chance to work with preferences across restarts. All of them will be reset if modified before by the test.

By using the prefs.js file, we will give tests the capability to modify prefs which are needed for testing specific features. As usual we would only set those prefs when the profile gets created, or an existing one gets cloned. 

I wonder what others think about it, so adding a bunch of people here.

Comment 7

[Andrew Halberstadt [:ahal]]

Ah, I definitely think user.js should be default. Setting them in prefs.js means they won't be persistent and will break most normal consumers of mozprofile. Firefox-ui-tests feel like the edge case here, not the other way around.

Making a subclass of Profile in the harness seems like a good enough way to solve this, though tbh, even that feels pretty hacky. I wonder if making the Puppeteer prefs API call this function when setting prefs would work instead:
https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIPrefService#savePrefFile()

Comment 8

[Chris Manchester (limited bugmail, email directly)]

I'm going to leave this here because it was originally filed against mozprofile and see about a solution for this for firefox-ui-tests in bug 1135874.