816641 - Caller tests can't continue if the test fails, leaves OOP frame alive

Status: RESOLVED WORKSFORME

(Remote Protocol :: Marionette, defect)

Description

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: test case

If you run this test, it will fail on line 71, in the iframe of the call screen. When it fails, it returns to the homescreen, but that iframe is still around. It causes problems where subsequent tests will fail since the contacts app can't properly load the iframe. Also, you can no longer manually use the phone (touching the screen has no effect)

So here, we have an app (contacts/dialer) that will launch another OOP iframe. If a call is successfully made/cancelled, it will close that iframe. If we run into a problem in the test, say an element isn't found and we throw a TimeoutException or something, then that iframe will persist.

Perhaps the right solution here is to keep track of all iframes launched by an app, and ensure they're all terminated at the end of a test, but maybe there's a less cumbersome solution.

Comment 1

[Malini Das [:mdas] - Away, not checking bugmail]

Do you have another idea for this?

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

heh, jgriffin^.

Comment 3

[Jonathan Griffin (:jgriffin)]

I don't have a good suggestion.  We could just get a list of iframes in GaiaTestCase.setUp, and then on tearDown, remove any iframes that weren't in the setUp list, but if we didn't do that using window_manager API's we'd likely still be in a bad state.

This all comes back to the problem in which we don't have any reasonable way of restoring Gaia to a good known state after a test.

Comment 4

[James Lal [:lightsofapollo]]

Hey Jonathan, I had some success "rebooting" the phone by doing something like this:

marionette.go_url "app://system.gaiamobile.org/index.html" its not a perfect reset but I suspect it would solve this particular issue.

Comment 5

[Stephen Donner [:stephend] Not actively reading bugmail]

David, mind taking this?

Comment 6

[David Burns :automatedtester]

I am looking at bug 814768 to do with OOP and hopefully it will give me inspiration. I am like Jonathan, I can't think of a nice way to solve this one...

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to James Lal [:lightsofapollo] ( AFK Back Feb 21st ) from comment #4)
> Hey Jonathan, I had some success "rebooting" the phone by doing something
> like this:
> 
> marionette.go_url "app://system.gaiamobile.org/index.html" its not a perfect
> reset but I suspect it would solve this particular issue.

I've tried using this approach to improve the reset between Gaia UI tests, but I haven't found a reliable solution.

After refreshing the system app there appears to be no way to identify when the device is ready. I've tried a number of things such as waiting for the homescreen app to be running, or waiting for the lockscreen to be visible but nothing I've tried is robust enough.

On top of that, I've tried adding an arbitrary wait of 30 seconds, and although the tests progress further there are still ultimately issues (unable to launch apps, socket closed, broken pipes). Any suggestions?

Comment 8

[Jonathan Griffin (:jgriffin)]

Ideally we could detect when a test fails in this way and actually reboot the phone before continuing.  We don't have that capability right now, but it's probably the best long-term solution.

Comment 9

[Malini Das [:mdas] - Away, not checking bugmail]

This must have been fixed somewhere along the way; after updating the test (since it uses older gaia-ui-test commands), I can no longer reproduce this.