Created attachment 626417 [details] Test preferences persist across application restarts. Using controller.restartApplication currently appears to reset preferences between restarts. I have attached a failing test to demonstrate this.
It's blocking nearly any of our restart tests to work with Mozmill 2. So it should block.
Whiteboard: [mozmill-2.0?] → [mozmill-2.0+]
Tested on windows linux and mac - looks like it is already fixed! \o/
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WORKSFORME
It looks like this started working with Firefox 13, but fails for Firefox 10, 11, 12. Here's the pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=4b9608fd670c&tochange=7c0ba1c98ff7
Dave, would you be able to do a hg bisect to identify the real bug?
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
I suspect the patch from bug 294260, which deals with restarts and resetting user preferences. I won't be able to do a bisect today.
Given that we do not have an issue with it in Mozmill 1.5 we should figure out when it has been regressed in Mozmill or Mozbase.
Status: REOPENED → ASSIGNED
Keywords: regression, regressionwindow-wanted
Git bisect has shown that this has been regressed with the landing of the patch on bug 640435. Not sure why but before this changeset I get: "message": "'test' should equal 'test'" Result of git bisect: 06c437895c9ce7cc82f0b4b708801b66a96b518e is the first bad commit commit 06c437895c9ce7cc82f0b4b708801b66a96b518e Author: Jeff Hammel <email@example.com> Date: Tue Aug 2 13:44:42 2011 -0700 Bug 640435 - 'this.server is undefined' when running user restart test;r=ctalbert :040000 040000 902477abd5be8ab7929c7d2d8c91795d48268cbb d25c355c78c365e77267bc7e5659802ef582272a M jsbridge :040000 040000 4d10bda9e36875732b8a290b65e25027399b77e5 c4470daf9c12d47d12e9bbbc38abb4f9af82fbdb M mozmill :040000 040000 348a5c3e5d749d8f7fbae80f0887d728bac38f2d 69a04e2a305b34c36ccff23d015c84f494dc4e97 M mozprofile :040000 040000 1c35f4f78f17f9fdcf48f66846846c7d77ac072c 7b1f0c8fa69eb4d822bea29a2419c2391e929fb0 M mutt
Dave, please test if this has been fixed meanwhile.
This has not been fixed. The attached testcase still fails in Firefox 10, 11, and 12.
Created attachment 639295 [details] working testcase It seems to work fine when using the Services module. Could be an issue with our preferences implementation. I will have to check that.
Assignee: nobody → hskupin
Comment on attachment 626417 [details] Test preferences persist across application restarts. I was wrong. Missed to test against the esr10 branch. :( My new testcase demonstrates this failure as expected. Lets see what it is.
Attachment #626417 - Attachment is obsolete: true
I have isolated those two test methods into two modules. When I run those tests against an existing profile the same problem occurs. But when I only run test1 the pref is still present in the profile after mozmill has been finished. Running test2 alone also passes. So this is indeed a change in the restart code. Btw. this also happens with the --restart flag.
This problem doesn't happen if you create a profile manually and make use of it during the Mozmill testrun. That keeps me thinking that something is wrong with mozprofile here.
So the problem with --restart seems to be different. Given that this should run tests in isolation mode we reset the profile: https://github.com/mozautomation/mozmill/blob/master/mozmill/mozmill/__init__.py#L381 Given that the preference is gone. Not sure what's the right behavior here would be. Shall we keep the profile and just restart the runner? That would leave everything behind and would fix the problem from above. I would say we should do that. Otherwise there will be no option for us to run our tests successfully in isolation mode. In case of restartApplication() I would assume we behave similar. But that's something I have to figure out now.
The problem goes away if you call usershutdown() instead of restartApplication() first. After the restart the pref still exists and another restartApplication() will also succeed. So something is broken with restartApplication() when the test is in the first session. I will further investigate tomorrow and hopefully be able to fix it soon. It's blocking my update testrun patch.
Let's investigate the possible work around in comment 15. We might just be able to close this as WFM if that works.
Whiteboard: [mozmill-2.0+] → [mozmill-2.1?][investigate work around comment: 15]
Tested both testcases attached here (that use restartApplication()) with all current branches (nightly, aurora, beta, release and esr17) and we have no failures. Dave confirmed that we have no interest anymore for older versions, so I think it's save to close it now.
Status: NEW → RESOLVED
Last Resolved: 6 years ago → 5 years ago
Resolution: --- → WORKSFORME
Yeah, got fixed in the Firefox 13.0 timeline which we do not support anymore.
Status: RESOLVED → VERIFIED
Whiteboard: [mozmill-2.1?][investigate work around comment: 15] → [investigate work around comment: 15]
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.