951516 - Intermittent of email tests

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Askeing Yen[:askeing][:fyen]]

Try to fix intermittent of email tests

Comment 1

[Askeing Yen[:askeing][:fyen]]

Attachment: PR 14791

Comment 2

[[:AndreiH]]

is this the intermittent fail that we are expecting?

Traceback (most recent call last):
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_send_imap_email.py", line 45, in test_send_imap_email
    self.email.wait_for_email('test email %s' % curr_time)
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 119, in wait_for_email
    self.toolbar.tap_refresh()
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 165, in tap_refresh
    self.marionette.find_element(*self._refresh_locator).tap()
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 83, in tap
    return self.marionette._send_message('singleTap', 'ok', id=self.id, x=x, y=y)
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 577, in _send_message
    self._handle_error(response)
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 604, in _handle_error
    raise ElementNotVisibleException(message=message, status=status, stacktrace=stacktrace)
TEST-UNEXPECTED-FAIL | test_send_imap_email.py test_send_imap_email.TestSendIMAPEmail.test_send_imap_email | ElementNotVisibleException: Element is not currently visible and may not be manipulated

Comment 3

[Askeing Yen[:askeing][:fyen]]

not only this fail.
sometimes email will spent a long time to "Setting up account", and get this fail:

 File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/gaiatest/apps/email/regions/setup.py", line 36, in tap_next
self.wait_for_element_displayed(*self._check_for_new_messages_locator)
File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 69, in wait_for_element_displayed
raise TimeoutException('Element %s not present before timeout' % locator)
TimeoutException: Element .tng-account-check-interval.mail-select not present before timeout

Comment 4

[Askeing Yen[:askeing][:fyen]]

And I found that the tap_account_prefs_next() of ManualSetupEmail will wait_for_element_displayed(*self._account_prefs_next_locator), however the tap_account_prefs_next() of SetupEmail do not wait for this locator.

Comment 5

[Viorela Ioia [:viorela]]

I got this fail: 

Traceback (most recent call last):
  File "/home/viorelaioia/.virtualenvs/gaia_mozilla/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/home/viorelaioia/WebQA/gaia_reviews/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_send_active_sync_email.py", line 41, in test_send_active_sync_email
    self.email.wait_for_email('test email %s' % curr_time)
  File "/home/viorelaioia/WebQA/gaia_reviews/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 126, in wait_for_email
    raise TimeoutException('Email %s not received before timeout' % subject)
TEST-UNEXPECTED-FAIL | test_send_active_sync_email.py test_send_active_sync_email.TestSendActiveSyncEmail.test_send_active_sync_email | TimeoutException: Email test email 1387361278462919 not received before timeout

Comment 6

[Askeing Yen[:askeing][:fyen]]

Here's another fail https://pastebin.mozilla.org/3802280 , so I add a wait before tap subject_locator.

Comment 7

[Viorela Ioia [:viorela]]

I ran test_send_active_sync_email.py with your fix and it still fails. Here are the run results: https://pastebin.mozilla.org/3802303

Comment 8

[Askeing Yen[:askeing][:fyen]]

Weird, I run with "--restart --type=b2g gaiatest/tests/functional/email/manifest.ini --repeat=2" options, and got
SUMMARY
-------
passed: 18
failed: 0
todo: 0

I will check it again tomorrow.
Thank you Viorela :)

Comment 9

[Florin Strugariu [:Bebe]]

Comment on attachment 8349242 [details] [review]
PR 14791

R- because of Viorela's comments

Comment 10

[Askeing Yen[:askeing][:fyen]]

Comment on attachment 8349242 [details] [review]
PR 14791

Hi Viorela,

I updated the pull and got passed, could you please help to try again and tell me the result.
If you can pass then I will rebase the commits into one commit.

Thank you.

Comment 11

[Viorela Ioia [:viorela]]

Hey, I ran the test again and I got 11/11 passes: https://pastebin.mozilla.org/3808733

Comment 12

[[:AndreiH]]

Comment on attachment 8349242 [details] [review]
PR 14791

The tests are running a little bit slow, I don't thinks that these changes are causing that

Comment 13

[Askeing Yen[:askeing][:fyen]]

Rebased two commits into one commit.

Comment 14

[Zac C (:zac)]

Askeing, I think it may be possible that we merge some of these test cases into one for example the setup/send cases.

This will reduce intermittents a little bit or at least make it easier to target the intermittent parts of the test.