819351 - [B2G] Marionette loses reference to imported JavaScript

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We often see issues where Marionette is unable to call imported JavaScript functions. I have been able to reliably replicate this by launching/killing the Camera application.

Expected result: Camera application launches and terminates 10 times

Actual result:

$ gaiatest --address=localhost:2828 gaiatest/test_js_ref.py 
starting httpd
running webserver on http://172.16.32.1:54680/
TEST-START test_js_ref.py
test_js_ref (test_js_ref.TestJSRef) ... Iteration: 1
ERROR

======================================================================
ERROR: test_js_ref (test_js_ref.TestJSRef)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dhunt/workspace/gaia-ui-tests/gaiatest/test_js_ref.py", line 11, in test_js_ref
    self.apps.kill(running_app)
  File "/Users/dhunt/workspace/gaia-ui-tests/gaiatest/gaia_test.py", line 69, in kill
    result = self.marionette.execute_async_script("GaiaApps.kill('%s');" % app.origin)
  File "/Users/dhunt/workspace/mozilla-central/testing/marionette/client/marionette/marionette.py", line 458, in execute_async_script
    specialPowers=special_powers)
  File "/Users/dhunt/workspace/mozilla-central/testing/marionette/client/marionette/marionette.py", line 211, in _send_message
    self._handle_error(response)
  File "/Users/dhunt/workspace/mozilla-central/testing/marionette/client/marionette/marionette.py", line 246, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
TEST-UNEXPECTED-FAIL | test_js_ref.py TestJSRef.test_js_ref | JavascriptException: ReferenceError: GaiaApps is not defined
----------------------------------------------------------------------
Ran 1 test in 7.330s

FAILED (errors=1)

SUMMARY
-------
passed: 0
failed: 1
todo: 0

Minimised test case:

from gaiatest import GaiaTestCase

class TestJSRef(GaiaTestCase):

    def test_js_ref(self):
        self.lockscreen.unlock()

        for i in range(10):
            print 'Iteration: %s' % (i + 1)
            running_app = self.apps.launch('Camera')
            self.apps.kill(running_app)

Comment 1

[David Burns :automatedtester]

This is because each frame appears to create its own file to use. we should make that global, especially in the B2G World

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Do you have a suggested workaround for now? Can we simply switch back to the top frame? Although I can frequently replicate this, it doesn't necessarily occur every time.

Comment 3

[David Burns :automatedtester]

I can't seem to create this issue but I noticed that if a listener is closed then it will delete the file. A scenario for this is if a window is closed and there is an imported script

https://tbpl.mozilla.org/?tree=Try&rev=40b344d8dd1c - B2G
https://tbpl.mozilla.org/?tree=Try&rev=c4da141d9b27 - Desktop

Comment 4

[Mozilla RelEng Bot]

Try run for c4da141d9b27 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=c4da141d9b27
Results (out of 6 total builds):
    exception: 3
    failure: 3
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-c4da141d9b27

Comment 5

[David Burns :automatedtester]

noticed issues with test so cancelled build so cancelled build above

Have done new try builds 

https://tbpl.mozilla.org/?tree=Try&rev=9ceeabb920c2
https://tbpl.mozilla.org/?tree=Try&rev=dd739c9b4b19

Comment 6

[David Burns :automatedtester]

Attachment: move import script file creation/deletion to marionette actor and have it deleted on session delete

will set reviewer when try results come in

Comment 7

[Mozilla RelEng Bot]

Try run for 9ceeabb920c2 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=9ceeabb920c2
Results (out of 9 total builds):
    success: 8
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-9ceeabb920c2

Comment 8

[Jonathan Griffin (:jgriffin)]

Waiting on the B2G try results to come in.  The wait times are really long unfortunately. :(

Comment 9

[David Burns :automatedtester]

B2G Try results are green too

Comment 10

[Jonathan Griffin (:jgriffin)]

Comment on attachment 691388 [details] [diff] [review]
move import script file creation/deletion to marionette actor and have it deleted on session delete

Review of attachment 691388 [details] [diff] [review]:
-----------------------------------------------------------------

Nice, thanks David.

Comment 11

[David Burns :automatedtester]

landed in https://hg.mozilla.org/integration/mozilla-inbound/rev/bed3d6fd43b0

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/bed3d6fd43b0

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/0c3461f4494e
https://hg.mozilla.org/releases/mozilla-b2g18/rev/9c487ffcae9f

Comment 14

[Mozilla RelEng Bot]

Try run for 9ceeabb920c2 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=9ceeabb920c2
Results (out of 10 total builds):
    exception: 1
    success: 8
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-9ceeabb920c2

Comment 15

[Mozilla RelEng Bot]

Try run for c4da141d9b27 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=c4da141d9b27
Results (out of 7 total builds):
    exception: 4
    failure: 3
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-c4da141d9b27

Comment 16

[Roy Collings]

Marionette 0.5.36:
Trying to switch back to an open application using launch, I get this (sometime in the last few days, but I¡m not sure exactly when because the error was hidden in a 'try...except' in my code):

  File "/home/rc78/projects/OWD_TEST_TOOLKIT/gaia-ui-tests/gaiatest/gaia_test.py", line 74, in launch
    result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.5.36-py2.7.egg/marionette/marionette.py", line 642, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.5.36-py2.7.egg/marionette/marionette.py", line 352, in _send_message
    self._handle_error(response)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.5.36-py2.7.egg/marionette/marionette.py", line 387, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
TEST-UNEXPECTED-FAIL | test_26845.py test_main.test_run | JavascriptException: ReferenceError: GaiaApps is not defined
	stacktrace:
	execute_async_script @gaia_test.py, line 74
	inline javascript, line 0
	src: "__marionetteParams.push(marionetteScriptFinished);let __marionetteFunc = function() { GaiaApps.launchWithName('Contacts')};__marionetteFunc.apply(null, __marionetteParams); "
----------------------------------------------------------------------
Ran 1 test in 53.105s

Comment 17

[Jonathan Griffin (:jgriffin)]

Can you attach the script that reproduces this?

Comment 18

[Roy Collings]

Hi, apologies for the delay doing this (tied up a bit at the moment). The code to reproduce this is spread across my own toolkit api so I need to 'gather' it into a simple script that doesn't use my methods when I get time (hopefully tomorrow).

Comment 19

[Francisco J Cano]

Hi, with Marionette 0.7.11 I'm still having this behaviour, simply doing:

    GaiaTestCase.setUp(self)
    self.app = self.apps.launch("Messages")
    self.wait_for_element_not_present('id', 'loading-overlay')
    self.data_layer.set_setting("lockscreen.enabled", False)
    self.device.restart_b2g()
    time.sleep(2)
    self.app = self.apps.launch("Messages")

Then you get the same error:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette_test.py", line 170, in run
    testMethod()
  File "/home/fran/owd/owd-obcertification/owd_test_cases/tests/SMS/test_SMSTCRCPTC004d.py", line 88, in test_run
    self.app = self.apps.launch("Messages")
  File "/home/fran/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 48, in launch
    result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette.py", line 1166, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.11-py2.7.egg/marionette/decorators.py", line 35, in _
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette.py", line 613, in _send_message
    self._handle_error(response)
  File "/usr/local/lib/python2.7/dist-packages/marionette_client-0.7.11-py2.7.egg/marionette/marionette.py", line 661, in _handle_error
    raise errors.JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: JavascriptException: ReferenceError: GaiaApps is not defined
TEST-UNEXPECTED-FAIL | test_SMSTCRCPTC004d.py test_SMSTCRCPTC004d.test_main.test_run | JavascriptException: JavascriptException: ReferenceError: GaiaApps is not defined
	execute_async_script @gaia_test.py, line 48
	inline javascript, line 0
	src: "__marionetteParams.push(marionetteScriptFinished);let __marionetteFunc = function() { GaiaApps.launchWithName('Messages')};__marionetteFunc.apply(null, __marionetteParams); "

Comment 20

[Jonathan Griffin (:jgriffin)]

This is by design; we don't support persisting imported scripts between sessions, and restarting B2G implicitly destroys the current session.

What you're looking for, I think, is a way to re-instantiate the objects that cause those script imports to happen mid-test; currently those only happen during setUp():

https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/gaia_test.py#L692

Potentially, you could add a line like this to your test case after the restart, although other problems might crop up; gaiatest really isn't designed with mid-test restarts in mind.

Comment 21

[Francisco J Cano]

Thanks, I will probably need to call the whole setUp method, to instantiate apps, data layer, and all the stuff. It was my first impression, but I wondered if there was another way to do it.