956641 - Fix intermittent failure of test_send_xxx_email.py

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Askeing Yen[:askeing][:fyen]]

There are intermittent failure of send email tests.
It seems like there is no subject due to the keyboard init/loading issue.

test_send_active_sync_email (test_send_active_sync_email.TestSendActiveSyncEmail) ... ERROR

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.2-py2.7.egg/marionette/marionette_test.py", line 143, in run
    testMethod()
  File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_send_active_sync_email.py", line 41, in test_send_active_sync_email
    self.email.wait_for_email('test email %s' % curr_time)
  File "/var/lib/jenkins/jobs/B2G.master.pvt.hamachi.gaiatest/workspace/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 127, in wait_for_email
    raise TimeoutException('Email %s not received before timeout' % subject)
TEST-UNEXPECTED-FAIL | test_send_active_sync_email.py test_send_active_sync_email.TestSendActiveSyncEmail.test_send_active_sync_email | TimeoutException: Email test email 1388983346356097 not received before timeout
----------------------------------------------------------------------
Ran 1 test in 211.794s

Comment 1

[Askeing Yen[:askeing][:fyen]]

Attachment: PR 15029

Wait for Keyboard app init and be displayed, then do the following steps.

Run with "gaiatest/tests/functional/email/test_send_active_sync_email.py --repeat=9" options and get all passed.
SUMMARY
-------
passed: 10
failed: 0
todo: 0

Comment 2

[Zac C (:zac)]

Comment on attachment 8356003 [details] [review]
PR 15029

If you debug this closely you see that the `clear()` method causes the keyboard to show, whereas here we wait after `send_keys()`

I think it would be safer to do: clear(), wait_for_keyboard, send_keys().

Comment 3

[Askeing Yen[:askeing][:fyen]]

(In reply to Zac C (:zac) from comment #2)
> Comment on attachment 8356003 [details] [review]
> PR 15029
> 
> If you debug this closely you see that the `clear()` method causes the
> keyboard to show, whereas here we wait after `send_keys()`
> 
> I think it would be safer to do: clear(), wait_for_keyboard, send_keys().

Cool! I don't know the "clearElement" action on input field will show the keyboard before.
Thanks for your information :)

Comment 4

[Askeing Yen[:askeing][:fyen]]

A)
> el.clear()
> el.send_keys(value)
> self.wait_for_condition(lambda m: self.keyboard.is_displayed())
It works fine when we send_keys() before keyboard is displayed.
Then wait for keyboard is desplayed to make sure the following actions will work correctly.

B)
> el.clear()
> self.wait_for_condition(lambda m: self.keyboard.is_displayed())
> el.send_keys(value)
The HTMLElement.send_keys() doesn't work after wait for keyboard is displayed.

======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py"
, line 143, in run
    testMethod()
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_send_active_sync_email.py", line 34, in test_send_active_sync_emai
l
    new_email.type_to(self.testvars['email']['ActiveSync']['email'])
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/gaiatest/apps/email/regions/new_email.py", line 28, in type_to
    self.wait_for_condition(lambda m: self.keyboard.is_displayed())
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", lin
e 107, in send_keys
    return self.marionette._send_message('sendKeysToElement', 'ok', id=self.id, value=typing)
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", lin
e 567, in _send_message
    raise TimeoutException(message='socket.timeout', status=ErrorCodes.TIMEOUT, stacktrace=None)
TEST-UNEXPECTED-FAIL | test_send_active_sync_email.py test_send_active_sync_email.TestSendActiveSyncEmail.test_send_active_sync_email | TimeoutException: socket.timeou
t
======================================================================
ERROR: None
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py"
, line 173, in run
    self.tearDown()
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 1084, in tearDown
    MarionetteTestCase.tearDown(self)
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette_test.py"
, line 318, in tearDown
    self.marionette.set_context("content")
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", lin
e 803, in set_context
    return self._send_message('setContext', 'ok', value=context)
  File "/home/askeing/workspace/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.7.1-py2.7.egg/marionette/marionette.py", lin
e 553, in _send_message
    raise MarionetteException(message="Please start a session")
TEST-UNEXPECTED-FAIL | test_send_active_sync_email.py test_send_active_sync_email.TestSendActiveSyncEmail.test_send_active_sync_email | MarionetteException: Please sta
rt a session
----------------------------------------------------------------------

C)
> el.clear()
> self.wait_for_condition(lambda m: self.keyboard.is_displayed())
> self.keyboard.send(value)
If we use "keyboard.send()" to replace the "HTMLElement.send_keys()", and it works fine.
However, it will be slower than HTMLElement.send_keys() due to keyboard.send() using the Actions.press to input the string, not input string directly by "sendKeysToElement" action.
And it also trigger the auto-correction function of keyboard. (ex: Lorem will be changed to Loren.)
So we have to disable the auto-correction function before running tests.

Comment 5

[Askeing Yen[:askeing][:fyen]]

Comment on attachment 8356003 [details] [review]
PR 15029

I updated the commit base on solution C because it make tests closer to real user behavior.
Please feel free to give me your idea :)

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8356003 [details] [review]
PR 15029

I gave a review because I was seeing this locally and also used a similar solution. See pull request for comments.

Comment 7

[Askeing Yen[:askeing][:fyen]]

Comment on attachment 8356003 [details] [review]
PR 15029

Thank you Dave, I updated the commit.

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8356003 [details] [review]
PR 15029

Looks great, and passes for me on device and desktop. Landed in:
https://github.com/mozilla-b2g/gaia/commit/4736ca3c3e38a5dfa4ad3aaaeac4f3154c0ef056

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/2f26978eb84c