1013938 - controller.restartApplication breaks dump on Windows

Status: RESOLVED WONTFIX

(Testing Graveyard :: Mozmill, defect, P2)

Description

[Andrei Eftimie]

Issuing controller.restartApplication breaks all subsequent dump statements on Windows.

This can be tested with attachment 8426153 [details]

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Very interesting. How do the following cases work for you?

1. Have a restart test in two files. Does that work?

2. Can you reprocude in manual mode, and triggering the restart yourself via the app service? Maybe you want to try it even outside of mozmill.

I wonder if that is also related to https://github.com/mozilla/mozmill-environment/pull/58#issuecomment-42651423

Comment 2

[Andrei Eftimie]

(In reply to Henrik Skupin (:whimboo) from comment #1)
> Very interesting. How do the following cases work for you?
> 
> 1. Have a restart test in two files. Does that work?
This doesn't work (this is our original problem seen in bug 1008135).

But what is interesting is controller.stopApplication() does _not_ break dump()
I'll check the difference between them (probably not the flags but the way we treat them in mozmill)

> 2. Can you reprocude in manual mode, and triggering the restart yourself via
> the app service? Maybe you want to try it even outside of mozmill.

Not sure how I should test this (eg from where to call dump() with --manual. I guess the browser console. I'll try this.

> I wonder if that is also related to
> https://github.com/mozilla/mozmill-environment/pull/58#issuecomment-42651423

I don't see that code being landed, am I missing something?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrei Eftimie from comment #2)
> > I wonder if that is also related to
> > https://github.com/mozilla/mozmill-environment/pull/58#issuecomment-42651423
> 
> I don't see that code being landed, am I missing something?

No, it hasn't been landed yet. What I wanted to say is, maybe ConEmu is more intelligent and gives us better failures in the console compared to mintty. It would be good to check the ConEmu branch with the temporary fixes you have tried here. It might also fix that other problem.

In general I think that Firefox somehow looses env variables or similar when a restart gets triggered via restartApplication(). May it also be a regression in Firefox? We might want to test e.g. a 17esr build, which should certainly not been affected.

Comment 4

[Andrei Eftimie]

I've checked a regular windows Command Prompt and the problem is also visible there.

Comment 5

[Andrei Eftimie]

Also tested ESR17 and Firefox 13 both and exhibit the same problem.

This and comment 4 hint that the problem is indeed within mozmill.

Comment 6

[Andrei Eftimie]

Interesting behaviour:

Steps (OSX):

1. open Firefox trough mozmill with --manual
2. set `browser.dom.window.dump.enabled` to `true` in about:config
3. install the Restartless Restart addon (open AMO Cmd+Shift+A, search for `restartless`. it will be the first hit)
4. open the Web Developer Tools (Cmd+Alt+I, or Right-Click > Inspect anywhere on a page)
5. go to the Console tab and write: `dump('some text')` // text is displayed in the terminal
6. restart Firefox (Cmd+Alt+R or File > Restart) // Firefox restarts and the terminal now shows your usual shell
7. you can issue commands in the terminal, your shell session appears to not be connected to Firefox anymore. Cmd+C will not end mozmill/FF
8. In Firefox reopen the Web Developer Tool, go into Console and issue a new dump() call.
9. Its text will appear into the terminal.

> (2.0.6.1)sl77-146:mozmill-tests andrei.eftimie$ mozmill --manual -b /Applications/FirefoxNightly.app/
> === DUMP TEST ===(2.0.6.1)sl77-146:mozmill-tests andrei.eftimie$ === DUMP TEST 2 ===
> (2.0.6.1)sl77-146:mozmill-tests andrei.eftimie$ 
> (2.0.6.1)sl77-146:mozmill-tests andrei.eftimie$ ls
> data    firefox   lib   metrofirefox  readme.txt  templates
> (2.0.6.1)sl77-146:mozmill-tests andrei.eftimie$ 
> (2.0.6.1)sl77-146:mozmill-tests andrei.eftimie$ === DUMP TEST 3 ===

On Windows, once I restart Firefox the console also appears to be disconnected from mozmill/FF, but It doesn't receive any more messages as it does on OSX.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrei Eftimie from comment #6)
> 2. set `browser.dom.window.dump.enabled` to `true` in about:config

We do no longer set this pref via Mozmill! That's a regression and will certainly be the cause of our problem. See also bug 982610 comment 8.

> 7. you can issue commands in the terminal, your shell session appears to not
> be connected to Firefox anymore. Cmd+C will not end mozmill/FF

Sure, and that's expected and working that way for a long time already.

> On Windows, once I restart Firefox the console also appears to be
> disconnected from mozmill/FF, but It doesn't receive any more messages as it
> does on OSX.

Check if the above set pref is still set to true.

Comment 8

[Andrei Eftimie]

(In reply to Henrik Skupin (:whimboo) from comment #7)
> (In reply to Andrei Eftimie from comment #6)
> > 2. set `browser.dom.window.dump.enabled` to `true` in about:config
> 
> We do no longer set this pref via Mozmill! That's a regression and will
> certainly be the cause of our problem. See also bug 982610 comment 8.

That is another issue. That pref only exposes dump inside the window object. You can still use dump from within a privileged context.

I've run the same tests (testcase and update tests) and nothing changes if I have this pref set to `true`.

> > 7. you can issue commands in the terminal, your shell session appears to not
> > be connected to Firefox anymore. Cmd+C will not end mozmill/FF
> 
> Sure, and that's expected and working that way for a long time already.
> 
> > On Windows, once I restart Firefox the console also appears to be
> > disconnected from mozmill/FF, but It doesn't receive any more messages as it
> > does on OSX.
> 
> Check if the above set pref is still set to true.

That was the first thing I checked. Yes, it is still true.

Comment 9

[Andrei Eftimie]

If we would start the runner after a restart, the testcase passe: 
https://github.com/mozilla/mozmill/blob/master/mozmill/mozmill/__init__.py#L320-L326

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That's not something we can do because mozrunner would spin up another process. But that clearly shows that something is wrong in Firefox itself, and that we might loose data. So it may indeed be related to the github issue for the environment as I pointed out.

Comment 11

[Andrei Eftimie]

Attachment: log_debug.txt

This is a log from running the testcase with --debug.

Comment 12

[Andrei Eftimie]

Attachment: testcase.zip

This is the testcase. I left 2 files so we can also test stopApplication (that's why it is a zip). There are 3 tests total, each of them has a dump().

test1 and test3 show their dump correctly.
test2's dump is broken due to restartApplication

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We are not going back to use Mozmill for our UI testing of Firefox and this is not a real blocker for a possible release.