Closed
Bug 999447
Opened 10 years ago
Closed 10 years ago
Active sync inbox seems empty in the tests on Jenkins
Categories
(Firefox OS Graveyard :: Gaia::UI Tests, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: Bebe, Unassigned)
References
Details
Attachments
(5 files)
Description: test_setup_and_send_active_sync_email.TestSetupAndSendActiveSyncEmail Test can't find any email in the active sync inbox We are seeing this issue on multiple devices and different build's Environment: Gaia f0463704888881b8ed1619e8d4b0d851b0e0311b Gecko https://hg.mozilla.org/mozilla-central/rev/1d0496e30feb BuildID 20140422040203 Version 31.0a1 ro.build.version.incremental=324 ro.build.date=Thu Dec 19 14:04:55 CST 2013 Traceback (most recent call last): Traceback (most recent call last): File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/marionette_test.py", line 163, in run testMethod() File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui/tests/python/gaia-ui-tests/gaiatest/tests/functional/email/test_setup_and_send_active_sync_email.py", line 56, in test_setup_and_send_active_sync_email self.email.wait_for_email(_subject) File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 124, in wait_for_email self.email_exists(self, subject)) File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui/.env/local/lib/python2.7/site-packages/marionette_client-0.7.6-py2.7.egg/marionette/wait.py", line 122, in until rv = condition(self.marionette) File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui/tests/python/gaia-ui-tests/gaiatest/apps/email/app.py", line 138, in __call__ self.app.mails[0].scroll_to_message() IndexError: list index out of range Reproducible manually: NO
Reporter | ||
Comment 1•10 years ago
|
||
Can someone from MV verify this tests in MV and also please check that the credentials are OK on the nodes. I checked the test_vars in the credentials repo and all seems OK
Reporter | ||
Comment 2•10 years ago
|
||
retornam can you take a look over this issue?
Flags: needinfo?(mozbugs.retornam)
Comment 3•10 years ago
|
||
(In reply to Florin Strugariu [:Bebe] from comment #1) > Can someone from MV verify this tests in MV and also please check that the > credentials are OK on the nodes. > > I checked the test_vars in the credentials repo and all seems OK The credentials are fine on all the machines. Is the an email in the active sync mailbox you are using? Which branch should I test on? Can you provide more information here.
Flags: needinfo?(mozbugs.retornam)
Reporter | ||
Comment 4•10 years ago
|
||
We are waiting for email's to be in the Inbox. As we don't delete the emails from the account there are for sure emails in the Inbox We run these tests on master Last reproducible build was: Gaia f0463704888881b8ed1619e8d4b0d851b0e0311b Gecko https://hg.mozilla.org/mozilla-central/rev/1d0496e30feb BuildID 20140422040203 Version 31.0a1 ro.build.version.incremental=324 ro.build.date=Thu Dec 19 14:04:55 CST 2013 Keep in mind I could not reproduce this locally
Reporter | ||
Comment 5•10 years ago
|
||
See: http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.ui/702/HTML_Report/?
Comment 6•10 years ago
|
||
It looks like perhaps the http request is timing out and it's assuming there's no mail in the folder. Retornam can you run the test locally at your desk and observe whether it's completing the steps correctly? We just can't tell from one screenshot (running remotely) whether it is working properly or not.
Flags: needinfo?(mozbugs.retornam)
Comment 7•10 years ago
|
||
This issue is still reproduced intermittently on the latest v1.4 build (Hamachi) with automated tests. Build info: Gaia b1242f33981024de59b8b4c26bacff8b876211b1 Gecko https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/fe4080728c60 BuildID 20140506000202 Version 30.0 ro.build.version.incremental=eng.tclxa.20131223.163538 ro.build.date=Mon Dec 23 16:36:04 CST 2013
Comment 8•10 years ago
|
||
Comment 9•10 years ago
|
||
Johan, would you be able to a) find the device(s) from our QA lab this has been failing on and b) get the device and run the test, locally (wait for any jobs to finish their run, first, of course) Thanks!
Flags: needinfo?(jlorenzo)
Comment 11•10 years ago
|
||
Created an adhoc job and run it on 3 different devices: http://selenium.qa.mtv2.mozilla.com:8080/view/B2G%20Hamachi/job/b2g.hamachi.mozilla-central.tinderbox.ui.active-sync-adhoc/ The 3 devices failed 100% times. Reproduces locally, same thing. After talking to asuth, it is related to 825538. ActiveSync servers maintain a limited set of known devices for synchronization purposes. Hitting refresh won't do anything until there's a couple of new messages in the account. So a workaround is to have a couple of new messages in the inbox and then tap refresh button on the device to reset the synchronization with ActiveSync. I tried it locally and I could retrieve some message and go further in the test case (without passing it for the same reason). TL;DR from asuth: It's absolutely essential that the automated tests never run concurrently with the same credentials until 825538 is fixed.
Depends on: 825538
Flags: needinfo?(jlorenzo)
Reporter | ||
Comment 12•10 years ago
|
||
asuth, jlorenzo the email account has tons of emails in it... I saw the same result with the new emails and refreshing. @asuth @stephend would it work if we would use a different email account for every phone
Flags: needinfo?(stephen.donner)
Flags: needinfo?(bugmail)
Comment 13•10 years ago
|
||
Jlorenzo, thanks for the v thorough research. Let's just disable this test until bug 825538 is resolved, IMO maintaining sets of credentials for all devices will be painful. We still have imap tests run on-device as a smoke check.
Comment 14•10 years ago
|
||
I agree not running the test may be the simplest thing. Note that even once we address bug 825538 it's still possible that there will be problems for the tests if you have a large number of devices running concurrently. ActiveSync servers have some unknown internal limitation on the amount of devices they will track for an account; they could still invalidate credentials out from under devices. Also, note that most IMAP servers enforce some types of connection limit. Having said that, if there's some way you can just run the test ~once a day from a single device running a trunk or most-recent-stable version, I think that would be useful. It seems like there are 2 things currently under test here: 1) Has the ActiveSync server changed its implementation and broken the email back-end? 2) Have we regressed the e-mail back-end or front-end as it relates to ActiveSync. The second case is already largely covered by back-end tests and the JS Marionette tests will shortly be running against our ActiveSync fake-server too. We don't have anything happening for the first case right now, although there is a medium term plan.
Flags: needinfo?(bugmail)
Comment 15•10 years ago
|
||
Thanks asuth, thorough as usual! We'll disable the test for now.
Flags: needinfo?(stephen.donner)
Flags: needinfo?(mozbugs.retornam)
Reporter | ||
Comment 16•10 years ago
|
||
Attachment #8419368 -
Flags: review?(zcampbell)
Attachment #8419368 -
Flags: review?(andrei.hutusoru)
Comment 17•10 years ago
|
||
I don't think we should be waiting for bug 825538 to get resolved here, as I don't think there's an ETA on resolution of that bug. What we should be doing instead is working on modifying the test infrastructure to ensure that only one email test can run. That's going to require some overhead work, so the temporary disabling of that test is okay, but we can't let this stay disabled, since this is a smoketest of the phone.
Comment 18•10 years ago
|
||
Comment on attachment 8419368 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/19064 r-, make it disabled =
Attachment #8419368 -
Flags: review?(zcampbell) → review-
Reporter | ||
Updated•10 years ago
|
Attachment #8419368 -
Flags: review- → review?(zcampbell)
Comment 19•10 years ago
|
||
Comment on attachment 8419368 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/19064 r+ test is disabled: https://github.com/mozilla-b2g/gaia/commit/4f352142a54f7f7ae2c460aad9049eda4b0edb14
Attachment #8419368 -
Flags: review?(zcampbell)
Attachment #8419368 -
Flags: review?(andrei.hutusoru)
Attachment #8419368 -
Flags: review+
Comment 21•10 years ago
|
||
and on v1.4: https://github.com/mozilla-b2g/gaia/commit/ac35d864d5b79520e190549037dcff644c4c665c
Reporter | ||
Comment 22•10 years ago
|
||
Attachment #8420076 -
Flags: review?(zcampbell)
Comment 23•10 years ago
|
||
Comment on attachment 8420076 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/19113 r+
Attachment #8420076 -
Flags: review?(zcampbell) → review+
Comment 24•10 years ago
|
||
Merged: https://github.com/mozilla-b2g/gaia/commit/0b63974c96dd6b766be337dd4e982919696c61c9 v1.4: https://github.com/mozilla-b2g/gaia/commit/8f26ea9682d8d09c8f3de762cff97670ea11937f
Updated•10 years ago
|
Whiteboard: [xfail]
Comment 25•10 years ago
|
||
Resovled by using different accounts for all devices: https://bugzilla.mozilla.org/show_bug.cgi?id=1012616
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Whiteboard: [xfail]
You need to log in
before you can comment on or make changes to this bug.
Description
•