1150920 - test_dialer_miss_call_from_known_contact_notification.py: " IndexError: list index out of range "

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Oliver Nelson [:oliverthor]]

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/135/HTML_Report/

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/767/HTML_Report/

Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_miss_call_from_known_contact_notification.py", line 63, in test_dialer_miss_call_from_known_contact_notification
    self.assertEqual(lock_screen.notifications[0].title, 'Missed call')
AssertionError: u'' != 'Missed call'

Test is failing with a 'blank' screenshot, a black screen. Phone may be unable to discern that it received a call at all? The traceback hints that the lockscreen didn't receive a notification, or following up with the screenshot, is unable to read it's lockscreen if it had a notification to parse regardless.

Attempting to reproduce manually, I did receive the missed call notification but the screen went black quickly afterwards. The test could be too slow to check the screen, or that may be the screenshot being slow to react.

Comment 1

[Oliver Nelson [:oliverthor]]

While this test was failing consistently at the above Traceback yesterday, today the error seems to have changed slightly to assert on the line immediately before (62 versus 63):

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/145/HTML_Report/

Traceback (most recent call last):
  File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_miss_call_from_known_contact_notification.py", line 62, in test_dialer_miss_call_from_known_contact_notification
    self.assertTrue(lock_screen.notifications[0].is_visible)
AssertionError: False is not true

5/5 failure from Jenkins adhoc
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk.ui.adhoc/779/HTML_Report/


Since the error appears so similar, I'm going to update this bug. I can begin opening separate bugs for similar issues if that would be preferable.

Comment 2

[Oliver Nelson [:oliverthor]]

The resulting error has morphed again for this issue, though the rest of the traceback stays the same. I reproed this error 5/5 on Jenkins adhoc, as well as 5/5 through my own local automation:

Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.11-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_miss_call_from_known_contact_notification.py", line 62, in test_dialer_miss_call_from_known_contact_notification
self.assertTrue(lock_screen.notifications[0].is_visible)
IndexError: list index out of range

This issue does not reproduce manually, missed calls are properly assigned on the lockscreen.

Comment 3

[Martijn Wargers (dead)]

I can reproduce it too.

Comment 4

[Martijn Wargers (dead)]

This is a regression from bug 1147731.

Comment 5

[Martijn Wargers (dead)]

Attachment: test_dialer_miss_call_from_known_contact_notification.py.diff

This would fix it. This might be the solution, but perhaps you know of a better way to solve this, Paul?

Comment 6

[Paul Yang [:pyang] (away)]

Yes I think it's the most suitable solution so far.
Lockscreen is now a part of system app, and when calling switch_to_displayed_app(), system app can't be considered displayed so we see this issue. ni Greg for more lock screen insight.

Comment 7

[GitHub Autolander Bot]

Attachment: [gaia] mwargers:1150920 > mozilla-b2g:master

Comment 8

[Martijn Wargers (dead)]

Comment on attachment 8597334 [details] [review]
[gaia] mwargers:1150920 > mozilla-b2g:master

Ok, let's make this change then.

Comment 9

[No-Jun Park [:njpark]]

Comment on attachment 8597334 [details] [review]
[gaia] mwargers:1150920 > mozilla-b2g:master

Ah, this is same change I made for bug 1157326.

Comment 10

[Greg Weng [:snowmantw][:gweng][:λ]]

LockScreen from the beginning is unfortunately a part of System app, so the solution is the way we solve previous issues like this.

Comment 11

[Martijn Wargers (dead)]

Comment on attachment 8597334 [details] [review]
[gaia] mwargers:1150920 > mozilla-b2g:master

Sorry, I forgot about this one.

Comment 12

[Martijn Wargers (dead)]

https://github.com/mozilla-b2g/gaia/commit/ed3071e5cb7f1b9ef90b2e34e661d79adce86b30

Comment 13

[Oliver Nelson [:oliverthor]]

This issue is still reproducing on Jenkins today with different errors that have been discussed here (plus an additional timeout exception):

* http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2.bitbar/64/HTML_Report/
* http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc.bitbar/59/HTML_Report/


#1 AssertionError: False is not true (1/6 repro today):
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.13-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_miss_call_from_known_contact_notification.py", line 65, in test_dialer_miss_call_from_known_contact_notification
self.assertTrue(self.contact.givenName in lock_screen.notifications[0].content)
AssertionError: False is not true

#2 AssertionError: u'' != 'Missed call' (3/6 repro today)
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.13-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_miss_call_from_known_contact_notification.py", line 64, in test_dialer_miss_call_from_known_contact_notification
self.assertEqual(lock_screen.notifications[0].title, 'Missed call')
AssertionError: u'' != 'Missed call'

#3 TimeoutException: Timed out after 10.1 seconds (2/6 repro today)
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.13-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_miss_call_from_known_contact_notification.py", line 73, in test_dialer_miss_call_from_known_contact_notification
system.wait_for_status_bar_displayed()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 36, in wait_for_status_bar_displayed
Wait(self.marionette).until(expected.element_displayed(*self._status_bar_locator))
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_driver-0.7-py2.7.egg/marionette_driver/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 10.1 seconds


Environment Variables:
Device firmware (base) 	L1TC100118D0
Device firmware (date) 	28 May 2015 08:40:44
Device firmware (incremental) 	eng.cltbld.20150528.044033
Device firmware (release) 	4.4.2
Device identifier 	flame
Device memory 	219772 kB
Device serial 	1d9e570a
Device uptime 	0 days 0 hours 2 minutes 38 seconds
Gaia date 	27 May 2015 06:27:27
Gaia revision 	05380df3158f
Gecko build 	20150528010203
Gecko revision 	baa9c64fea6f
Gecko version 	41.0a1

Comment 14

[Oliver Nelson [:oliverthor]]

Returning to resolved fixed and opening a new bug establishing the above comment and behavior. 
Please refer to bug 1169332 for further developments of this test.