Investigate failure in test_setup_and_send_imap_email.py

RESOLVED FIXED

Status

RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: viorela, Assigned: viorela)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments)

(Assignee)

Description

4 years ago
Test test_setup_and_send_imap_email.py is failing intermittently in Jenkins: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.ui.functional.smoke/89/HTML_Report/
I was not able to reproduce the failure locally, by running the automated test several times, or manually.
The test is failing only on master, it works as expected on v2.1. 

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette_test.py", line 264, in run
testMethod()
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_setup_and_send_imap_email.py", line 70, in test_setup_and_send_imap_email
self.assertEqual(_body, read_email.body.splitlines()[0])
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/email/regions/read_email.py", line 17, in body
return self.marionette.find_element(*self._body_locator).text
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 1279, in find_element
response = self._send_message('findElement', 'value', **kwargs)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/decorators.py", line 35, in _
return func(*args, **kwargs)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 638, in _send_message
self._handle_error(response)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette.py", line 672, in _handle_error
raise errors.NoSuchElementException(message=message, status=status, stacktrace=stacktrace)
NoSuchElementException: NoSuchElementException: Unable to locate element: .card.center .msg-body-content

Device firmware (date) 	28 Oct 2014 04:31:24
Device firmware (incremental) 	eng.cltbld.20141028.073110
Device firmware (release) 	4.4.2
Device identifier 	flame
Gaia date 	27 Oct 2014 10:43:23
Gaia revision 	6a7fb482a03c
Gecko build 	20141028040202
Gecko revision 	a255a234946e
Gecko version 	36.0a1

We need to still try to reproduce the issue.
(Assignee)

Updated

4 years ago
Assignee: nobody → viorela.ioia
(Assignee)

Updated

4 years ago
QA Whiteboard: fxosqa-auto-from-s2, fxosqa-auto-s3
(Assignee)

Updated

4 years ago
QA Whiteboard: fxosqa-auto-from-s2, fxosqa-auto-s3 → [fxosqa-auto-from-s2] [fxosqa-auto-s3]
(Assignee)

Comment 1

4 years ago
I've been able to reproduce this failure locally by running the automated test, on latest v2.2. 
It is intermittent, repro rate would be 1/10

The test is failing because the subject of the most recent received email is not tapped, so we won't reach the read email view, to get the subject & body of the read email. 
I'm working on finding a better wait before tapping on capture button. 
I will add a PR afterwards.
(Assignee)

Comment 2

4 years ago
Test test_setup_and_send_imap_email.py is also failing at some other step: 
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/965/HTML_Report/

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.4-py2.7.egg/marionette/marionette_test.py", line 264, in run
testMethod()
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_setup_and_send_imap_email.py", line 70, in test_setup_and_send_imap_email
self.assertEqual(_body, read_email.body.splitlines()[0])
AssertionError: 'b1415176107152861' != u'Test email message 1415134095.91'

I suppose this failure is caused by the fact that the wrong subject was tapped.
It is possible to receive a new email right before tapping the subject, as the IMAP account is used by several devices.
In that case, that new email's subject will be tapped, because it is the most recent in the list. 

I did some changes in the test, so that we tap the email that has the expected subject, and not the most recent email.
I don't think these changes will fix the issue described in comment 0, but they are an improvement.
(Assignee)

Comment 3

4 years ago
Created attachment 8517379 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25765
Attachment #8517379 - Flags: review?(zcampbell)
Attachment #8517379 - Flags: review?(robert.chira)
Attachment #8517379 - Flags: review?(florin.strugariu)
Attachment #8517379 - Flags: review?(florin.strugariu) → review+

Comment 4

4 years ago
Comment on attachment 8517379 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25765

r+
Attachment #8517379 - Flags: review?(zcampbell) → review+
(Assignee)

Updated

4 years ago
Attachment #8517379 - Flags: review?(robert.chira)
(Assignee)

Updated

4 years ago
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Comment 6

4 years ago
Sorry, this fix was for the error described in comment 2. I'm not sure it also fixed the error from comment 0. I started an adhoc to check this: http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/283/console
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Updated

4 years ago
QA Whiteboard: [fxosqa-auto-from-s2] [fxosqa-auto-s3] → [fxosqa-auto-from-s2] [fxosqa-auto-s4][fxosqa-auto-from-s3]
Created attachment 8522887 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26145

Uplift for 2.1
Attachment #8522887 - Flags: review?(viorela.ioia)
(Assignee)

Comment 9

4 years ago
Created attachment 8525909 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25894

Fix for the error described in comment 0. See adhoc runs results in the PR.
Attachment #8525909 - Flags: review?(zcampbell)
Attachment #8525909 - Flags: review?(robert.chira)
Attachment #8525909 - Flags: review?(florin.strugariu)
Comment on attachment 8525909 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25894

comment in the PR
Attachment #8525909 - Flags: review?(robert.chira) → review-
(Assignee)

Updated

4 years ago
Attachment #8525909 - Flags: review- → review?(robert.chira)
Attachment #8525909 - Flags: review?(robert.chira) → review+
Attachment #8525909 - Flags: review?(florin.strugariu) → review+
https://github.com/mozilla-b2g/gaia/commit/717808b02cd904d3c6ec694faab6d4652c7b0615
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Attachment #8525909 - Flags: review?(zcampbell)
You need to log in before you can comment on or make changes to this bug.