475098 - Modify mochitest profile to support fennec preferences

Status: RESOLVED WORKSFORME

(Testing :: Mochitest, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

In order to run mochitest chrome on Fennec, we need to adjust some user_pref settings in the generated profile.  This is not a big deal to do, but since the profile is generated at run time, we need to resolve this in the automation.py or runtests.py scripts.

I have documented what needs to happen here:
https://wiki.mozilla.org/Mobile/Fennec_Chrome#TODO

There are two thoughts on this:
1) fix this at buildtime in automation.py.in. - I don't like this because the tests fail on fennec desktop versions
2) fix this with a cli option to runtests.py and implement this like we do with addChromeToProfile() and other similar functions.

Please provide feedback as to what is the preferred method for this.  Looking down the road, we will have other things to adjust for Fennec and possibly for specific platforms (wince, maemo, symbian).  For Maemo, we have mostly documented everything, but wince and symbian could add more modifications to the list.

Comment 1

[(not currently active) Ted Mielczarek]

I have two thoughts, I guess:
1) Can we just fix the tests to set those prefs temporarily as needed? Seems like otherwise any other app that wanted to run the tests but didn't have the pref set right would fail
2) If not, maybe we can add a --with-additional-prefs=/path/to/file option to runtests.py, where you could pass a whole prefs.js file to be appended to the profile.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I like your option #2.  

The tests depend on dom.storage size and browser history size.  There are not many tests which reference this, but we adjust these values in fennec due to the lack of space.  So fixing the tests won't really help in the Fennec scenario.  If we add more tests in the future that depend on these user_pref's, then I think they will run into similar issues.

There is probably no correct method, and at some point in time in the future, we might realize we need to do things different yet again.

Comment 3

[(not currently active) Ted Mielczarek]

Can you CC the test authors for an opinion? Feels like if the test depends on a pref, then having the test make that pref setting explicit is the right way to go.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

I have had trouble getting the correct email addresses, so let me just put the associated tests and add the test case authors after posting this:

dom.storage.default_quota (bug 472513):
chrome://mochikit/content/chrome/dom/tests/mochitest/chrome/test_domstorage.xul - Neil Deakin


browser.sessionhistory.max_total_viewers (bug 472334):
chrome://mochikit/content/chrome/docshell/test/chrome/test_bug112564.xul -  	Graeme McCutcheon
chrome://mochikit/content/chrome/docshell/test/chrome/test_bug92598.xul -  	Graeme McCutcheon
chrome://mochikit/content/chrome/docshell/test/chrome/test_bug364461.xul - Sylvain Pasche

browser.sessionhistory.max_total_viewers (bug 472341):
chrome://mochikit/content/chrome/docshell/test/chrome/test_bug396519.xul - Andrew Schultz

Comment 5

[(not currently active) Ted Mielczarek]

Hey, test authors:
Could we fix these tests to explicitly set prefs whose behavior they depend on? Fennec sets some prefs differently to Firefox, and fails these tests as a result. See bugs mentioned in comments here for more details.

Comment 6

[Sylvain Pasche]

Should it be the responsibility of the test or the test runner to specify what prefs have to be set to their default value in order for a given test to run correctly?

What I mean is that lots of tests in general would break if some preferences are changed from their default value. For instance, if the browser.cache.*.enable prefs are set to false, all the tests testing caching wouldn't pass. In consequence, all these failing tests would need to specify that they depend on the  browser.cache.*.enable prefs, which does not scale well.

I guess if there's no chance of changing lots of other prefs from their default value in the future, it should be fine to change them locally in the tests. http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/mozprefs.js should help ensure they are reverted to their previous value after the test is done.

Comment 7

[(not currently active) Ted Mielczarek]

I don't know, I don't have a great answer for you here. :) It feels to me like if a test depends on a pref, it ought to set it explicitly. We can use the mozprefs stuff to make it easier. It feels fragile to have tests depend on the default values of prefs, but maybe I'm being too paranoid.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

It is a delicate balance.  Most people assume you have a couple pages of browser history available, but on Fennec we don't have that by default.  I think it is important to have tests which utilize the history.

With the defaultquota for dom.storage, maybe the test could query the max size available.  

Something else to consider is that if these tests were run in Private Browsing mode, would they behave the same?  I am not sure if there is a well documented method for the tests to adjust/query preferences.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

I have verified that we can set these on the cli with --setpref:
python runtests.py --appname=../fennec/fennec --xre-path=../fennec/xulrunner --chrome --setpref=browser.sessionhistory.max_total_viewers=-1 --setpref=dom.storage.default_quota=640

Comment 10

[Andrew Halberstadt [:ahal]]

We run mochitest-chrome on android so I'll assume this was either unnecessary or happened in another bug.