957901 - JavascriptException: TypeError: widgetFrame is null being thrown when launching Marketplace app during gaiatest

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[Bob Silverberg [:bsilverberg]]

Attachment: logcat from test run on CI that encountered the problem

When running our marketplace-tests-gaia test suite on device we encounter frequent test failures when the above JS exception is thrown during launch.

The stack trace from the Python test is:

test_marketplace_feedback_user (test_marketplace_feedback_login.TestMarketplaceFeedback) ... ERROR

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/marketplacetests/tests/test_marketplace_feedback_login.py", line 24, in test_marketplace_feedback_user
    self.marketplace.launch()
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/marketplacetests/marketplace/app.py", line 48, in launch
    Base.launch(self, launch_timeout=120000)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/gaiatest-0.21.2-py2.7.egg/gaiatest/apps/base.py", line 26, in launch
    self.app = self.apps.launch(self.name, launch_timeout=launch_timeout)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/gaiatest-0.21.2-py2.7.egg/gaiatest/gaia_test.py", line 75, in launch
    result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 1073, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 577, in _send_message
    self._handle_error(response)
  File "/var/jenkins/workspace/b2g.hamachi.mozilla-aurora.marketplace/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 612, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: TypeError: widgetFrame is null
	stacktrace:
	@app://system.gaiamobile.org/js/cost_control.js, line 70
TEST-UNEXPECTED-FAIL | test_marketplace_feedback_login.py test_marketplace_feedback_login.TestMarketplaceFeedback.test_marketplace_feedback_user | 
----------------------------------------------------------------------
Ran 1 test in 74.436s

FAILED (errors=1)

The issue is that, while this does not appear to be a critical exception, Marionette is catching it and causing the test to fail immediately.

I am attaching the logcat from the latest test run that included two of these failures.

Comment 1

[Florin Strugariu [:Bebe]]

We still see this failing sometimes on GaiaUI-Tests

http://qa-selenium.mv.mozilla.com:8080/job/b2g.hamachi.mozilla-aurora.marketplace/25/

http://qa-selenium.mv.mozilla.com:8080/job/b2g.hamachi.mozilla-aurora.marketplace/38/

Comment 2

[[:fabrice] Fabrice Desré]

Salva, can you take a look at failing gracefully there?

Comment 3

[Salvador de la Puente González [:salva]]

Attachment: Preventing system cost control network usage monitor to fail while Marionette testing

Bob, can you repeat the Marionette test and check if this is still reproducing with my patch? Thank you.

Comment 4

[Bob Silverberg [:bsilverberg]]

salva, this was actually happening on v1.3, not master, although we only have the tests running on v1.3 so I cannot say that it wasn't also present on master. Also, we have not seen the issue again for some time, so perhaps it somehow fixed itself.

I will try first to see if I can reproduce the issue on master with the test that was causing it, and, if I can, then I'll try applying your patch and seeing if the issue goes away. As it has always been an intermittent issue I don't know how easy it will be to prove that the patch eliminates it, but I'll give it a go.

Comment 5

[Bob Silverberg [:bsilverberg]]

Well, when trying to reproduce this on master I was unable to, but I did encounter a different JavascriptException when launching Marketplace:

test_marketplace_feedback_anonymous (test_marketplace_feedback_anonymous.TestMarketplaceFeedback) ... ERROR

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bsilverberg/.virtualenvs/marketplace-tests-gaia/lib/python2.7/site-packages/marionette_client-0.7.2-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/Users/bsilverberg/gitRepos/marketplace-tests-gaia/marketplacetests/tests/test_marketplace_feedback_anonymous.py", line 17, in test_marketplace_feedback_anonymous
    self.marketplace.launch()
  File "/Users/bsilverberg/gitRepos/marketplace-tests-gaia/marketplacetests/marketplace/app.py", line 48, in launch
    Base.launch(self, launch_timeout=120000)
  File "/Users/bsilverberg/.virtualenvs/marketplace-tests-gaia/lib/python2.7/site-packages/gaiatest-0.21.3-py2.7.egg/gaiatest/apps/base.py", line 26, in launch
    self.app = self.apps.launch(self.name, launch_timeout=launch_timeout)
  File "/Users/bsilverberg/.virtualenvs/marketplace-tests-gaia/lib/python2.7/site-packages/gaiatest-0.21.3-py2.7.egg/gaiatest/gaia_test.py", line 81, in launch
    result = self.marionette.execute_async_script("GaiaApps.launchWithName('%s')" % name, script_timeout=launch_timeout)
  File "/Users/bsilverberg/.virtualenvs/marketplace-tests-gaia/lib/python2.7/site-packages/marionette_client-0.7.2-py2.7.egg/marionette/marionette.py", line 1080, in execute_async_script
    filename=os.path.basename(frame[0]))
  File "/Users/bsilverberg/.virtualenvs/marketplace-tests-gaia/lib/python2.7/site-packages/marionette_client-0.7.2-py2.7.egg/marionette/marionette.py", line 584, in _send_message
    self._handle_error(response)
  File "/Users/bsilverberg/.virtualenvs/marketplace-tests-gaia/lib/python2.7/site-packages/marionette_client-0.7.2-py2.7.egg/marionette/marionette.py", line 619, in _handle_error
    raise JavascriptException(message=message, status=status, stacktrace=stacktrace)
JavascriptException: TypeError: app is undefined
	stacktrace:
	@app://system.gaiamobile.org/js/stack_manager.js, line 109
TEST-UNEXPECTED-FAIL | test_marketplace_feedback_anonymous.py test_marketplace_feedback_anonymous.TestMarketplaceFeedback.test_marketplace_feedback_anonymous |
----------------------------------------------------------------------
Ran 1 test in 57.393s

FAILED (errors=1)

I assume this is coming from a different component, but will be a blocker for us when we try to turn on Marketplace testing for the master branch.

Would there be some kind of global way of dealing with these types of non-critical exceptions that would prevent them from killing the tests whenever they occur?

Should I open a new bug about this new JSException?

Comment 6

[Zac C (:zac)]

(In reply to Bob Silverberg [:bsilverberg] from comment #5)
> 
> I assume this is coming from a different component, but will be a blocker
> for us when we try to turn on Marketplace testing for the master branch.
> 
> Would there be some kind of global way of dealing with these types of
> non-critical exceptions that would prevent them from killing the tests
> whenever they occur?
> 
Yes there was a fair bit of discussion about this during the work week and it spilled over onto dev-gaia email list too and https://bugzilla.mozilla.org/show_bug.cgi?id=959493 .
The security team are also working on a similar thing but not in CI. If you want to know more let's chat outside this bug.

> Should I open a new bug about this new JSException?

Yes because the source is different the component and assignee will be too.

Comment 7

[Bob Silverberg [:bsilverberg]]

salva: I have been unable to reproduce the issue on v1.3 (where it was previously occurring) without your patch, so I am also unable to verify your patch on v1.3. I have never seen the issue on master, so I guess we can just close this as WFM now.

Comment 8

[Bob Silverberg [:bsilverberg]]

Also, I am now unable to reproduce the new JavascriptException that I reported above, so I won't be opening a bug for that.

Comment 9

[Salvador de la Puente González [:salva]]

Ok, I'm solving as WFM. Let's reopen if the issue is seen again.
Than you very much, Bob.

Comment 10

[Bob Silverberg [:bsilverberg]]

salva, we just saw this again in automation, on v1.3, so perhaps your fix would be good to add. Please note that we're seeing this on v1.3, so it should either be added just there, or on both master and v1.3.

Comment 11

[Salvador de la Puente González [:salva]]

The solution is compatible with both branches.

Comment 12

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8365650 [details]
Preventing system cost control network usage monitor to fail while Marionette testing

Sorry, salva, but I am once again not able to reproduce the error locally, so I cannot verify whether this fixes it. This doesn't constitute an r- though. I think it's worth requesting a review from a peer and getting it into the code base anyway.

Comment 13

[Salvador de la Puente González [:salva]]

What about this bug?

Comment 14

[Jason Smith [:jsmith]]

Salva - qawanted is really used primarily for manual QA needs, not automation. As such, I'd like to pull the qawanted flag here. I'd suggest using needinfo here instead to work with Bob on the needs you have here.

Comment 15

[Salvador de la Puente González [:salva]]

Understood, useful advice, thank you!

Comment 16

[Bob Silverberg [:bsilverberg]]

This problem has not cropped its head for some time so I think we can just close this bug. Salva, if you want to get your code in I suggest you ask someone else to review it as I am not in a position to evaluate it.

Comment 17

[Salvador de la Puente González [:salva]]

Let's close as WFM. Thank you.