Intermittent of email tests

RESOLVED FIXED

Status

RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: askeing, Assigned: askeing)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

46 bytes, text/x-github-pull-request
viorela
: review+
AndreiH
: review+
Details | Review | Splinter Review
(Assignee)

Description

5 years ago
Try to fix intermittent of email tests
(Assignee)

Updated

5 years ago
Assignee: nobody → fyen
(Assignee)

Comment 1

5 years ago
Created attachment 8349242 [details] [review]
PR 14791
Attachment #8349242 - Flags: review?(florin.strugariu)

Comment 2

5 years ago
is this the intermittent fail that we are expecting?

Traceback (most recent call last):
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_send_imap_email.py", line 45, in test_send_imap_email
    self.email.wait_for_email('test email %s' % curr_time)
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 119, in wait_for_email
    self.toolbar.tap_refresh()
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 165, in tap_refresh
    self.marionette.find_element(*self._refresh_locator).tap()
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 83, in tap
    return self.marionette._send_message('singleTap', 'ok', id=self.id, x=x, y=y)
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 577, in _send_message
    self._handle_error(response)
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", line 604, in _handle_error
    raise ElementNotVisibleException(message=message, status=status, stacktrace=stacktrace)
TEST-UNEXPECTED-FAIL | test_send_imap_email.py test_send_imap_email.TestSendIMAPEmail.test_send_imap_email | ElementNotVisibleException: Element is not currently visible and may not be manipulated
(Assignee)

Comment 3

5 years ago
not only this fail.
sometimes email will spent a long time to "Setting up account", and get this fail:

 File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/gaiatest/apps/email/regions/setup.py", line 36, in tap_next
self.wait_for_element_displayed(*self._check_for_new_messages_locator)
File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/gaiatest/apps/base.py", line 69, in wait_for_element_displayed
raise TimeoutException('Element %s not present before timeout' % locator)
TimeoutException: Element .tng-account-check-interval.mail-select not present before timeout
(Assignee)

Comment 4

5 years ago
And I found that the tap_account_prefs_next() of ManualSetupEmail will wait_for_element_displayed(*self._account_prefs_next_locator), however the tap_account_prefs_next() of SetupEmail do not wait for this locator.
I got this fail: 

Traceback (most recent call last):
  File "/home/viorelaioia/.virtualenvs/gaia_mozilla/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/home/viorelaioia/WebQA/gaia_reviews/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_send_active_sync_email.py", line 41, in test_send_active_sync_email
    self.email.wait_for_email('test email %s' % curr_time)
  File "/home/viorelaioia/WebQA/gaia_reviews/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 126, in wait_for_email
    raise TimeoutException('Email %s not received before timeout' % subject)
TEST-UNEXPECTED-FAIL | test_send_active_sync_email.py test_send_active_sync_email.TestSendActiveSyncEmail.test_send_active_sync_email | TimeoutException: Email test email 1387361278462919 not received before timeout
(Assignee)

Comment 6

5 years ago
Here's another fail https://pastebin.mozilla.org/3802280 , so I add a wait before tap subject_locator.
I ran test_send_active_sync_email.py with your fix and it still fails. Here are the run results: https://pastebin.mozilla.org/3802303
(Assignee)

Comment 8

5 years ago
Weird, I run with "--restart --type=b2g gaiatest/tests/functional/email/manifest.ini --repeat=2" options, and got
SUMMARY
-------
passed: 18
failed: 0
todo: 0

I will check it again tomorrow.
Thank you Viorela :)
Comment on attachment 8349242 [details] [review]
PR 14791

R- because of Viorela's comments
Attachment #8349242 - Flags: review?(florin.strugariu) → review-
(Assignee)

Comment 10

5 years ago
Comment on attachment 8349242 [details] [review]
PR 14791

Hi Viorela,

I updated the pull and got passed, could you please help to try again and tell me the result.
If you can pass then I will rebase the commits into one commit.

Thank you.
Attachment #8349242 - Flags: feedback?(viorela.ioia)
Hey, I ran the test again and I got 11/11 passes: https://pastebin.mozilla.org/3808733
Attachment #8349242 - Flags: review+

Comment 12

5 years ago
Comment on attachment 8349242 [details] [review]
PR 14791

The tests are running a little bit slow, I don't thinks that these changes are causing that
Attachment #8349242 - Flags: review+
Attachment #8349242 - Flags: feedback?(viorela.ioia)
(Assignee)

Comment 13

5 years ago
Rebased two commits into one commit.
(Assignee)

Updated

5 years ago
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(Assignee)

Updated

5 years ago
Attachment #8349242 - Flags: review-

Comment 14

5 years ago
Askeing, I think it may be possible that we merge some of these test cases into one for example the setup/send cases.

This will reduce intermittents a little bit or at least make it easier to target the intermittent parts of the test.
You need to log in before you can comment on or make changes to this bug.