Closed Bug 1022693 Opened 6 years ago Closed 6 years ago

[UI Tests] Intermittent Travis failure: TEST-UNEXPECTED-FAIL | test_persona_app.py test_persona_app.TestPersonaStandard.test_persona_standard_sign_in | KeyError: 'email'

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

x86_64
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: julienw, Assigned: AndreiH)

References

Details

Attachments

(1 file)

TEST-START test_persona_app.py

test_persona_standard_sign_in (test_persona_app.TestPersonaStandard) ... ERROR

======================================================================

ERROR: None

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/marionette_test.py", line 145, in run

self.setUp()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/persona/test_persona_app.py", line 26, in setUp

"verifier": TESTUSER_VERIFIER}

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/mocks/persona_test_user.py", line 56, in create_user

return MockUser(email=decode['email'], password=decode['pass'], name=decode['email'].split('@')[0])

TEST-UNEXPECTED-FAIL | test_persona_app.py test_persona_app.TestPersonaStandard.test_persona_standard_sign_in | KeyError: 'email'

----------------------------------------------------------------------

Ran 1 test in 71.985s

FAILED (errors=1)
This test has recently gone perma-red. This is a different failure, but putting it here because I think we need to disable this:

TEST-START test_persona_app.py

test_persona_standard_sign_in (test_persona_app.TestPersonaStandard) ... ERROR

======================================================================

ERROR: None

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/marionette_test.py", line 163, in run

testMethod()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/persona/test_persona_app.py", line 38, in test_persona_standard_sign_in

persona.login(self.user.email, self.user.password)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/persona/app.py", line 28, in login

self.tap_continue()

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/persona/app.py", line 62, in tap_continue

self.wait_for_element_not_displayed(*self._continue_button_locator)

File "/home/travis/build/mozilla-b2g/gaia/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 48, in wait_for_element_not_displayed

lambda m: not m.find_element(by, locator).is_displayed())

File "/home/travis/build/mozilla-b2g/gaia/travis_venv/local/lib/python2.7/site-packages/marionette_client-0.7.7-py2.7.egg/marionette/wait.py", line 143, in until

cause=last_exc)

TimeoutException: TimeoutException: Timed out after 10.0 seconds

TEST-UNEXPECTED-FAIL | test_persona_app.py test_persona_app.TestPersonaStandard.test_persona_standard_sign_in |

----------------------------------------------------------------------

Ran 1 test in 28.899s

FAILED (errors=1)
Zac - Could you have someone look at this? It's gone perma-red, and this test constantly fails. Is it because persona goes down? 

Disabled: https://github.com/mozilla-b2g/gaia/commit/0361adc64a17dc2e45d851a7edaf7fd3312667a0
Flags: needinfo?(zcampbell)
It seems persona will take whole life to connecting. I'm checking the root cause.
It looks like the connection to the provider was just very slow.
Flags: needinfo?(zcampbell)
Finding:

There are two issues I found till now.

a) I cannot get the login page. The page stopped with "Sign In" title and empty contain.
b) I cannot finish loading from "Checking with your email provider"

This doesn't happen on mobile testing.
I'm getting stuck on b) 100% on desktopb2g
Depends on: 1024385
I will take a look over this after issue: https://github.com/mozilla/persona/issues/4147 is resolved
Assignee: nobody → andrei.hutusoru
I started a build on Travis with the required changes, running only this test, let's see how it performs:
https://travis-ci.org/mozilla-b2g/gaia/builds/27938716
Priority: -- → P1
From the looks of it, this is a Persona issue, not something from our tests.
Running just this test on travis passed:
https://travis-ci.org/mozilla-b2g/gaia/jobs/29429917
Could we simulate a persona server in our tests?
(I know it's likely not easy but this would make our tests more resilient)
Attachment #8452957 - Flags: review?(florin.strugariu)
Attachment #8452957 - Flags: review?(florin.strugariu) → review+
(In reply to Julien Wajsberg [:julienw] from comment #12)
> Could we simulate a persona server in our tests?
> (I know it's likely not easy but this would make our tests more resilient)

I've looked into this and it's possible but we need the Persona team to take a bit more control here I think.

One thing we could do is try and get the credentials but then skip the test if we couldn't get the credentials.
For future reference, please notify test authors (or at least someone/anyone on the team responsible for the tested app) before you disable their tests.

We have seen a number of critical Persona bugs in the past week that should have been caught by this test.

While Persona is not a high priority right now, some of the bugs seem to be popping up on 2.0, where it's far too late to do anything about it.
Aw man, after looking at the persona bug referenced in comment 8, I see that you did tell someone on the team - but Jed left Mozilla shortly after the bug was filed :-(

Sorry for the misunderstanding, looks like this thread got dropped when he left
You need to log in before you can comment on or make changes to this bug.