Investigate test_toggle_airplane_mode intermittent failure

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: RobertC, Assigned: Martijn Wargers (zombie))

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments)

(Reporter)

Description

3 years ago
test_toggle_airplane_mode is failing intermittently with the following stacktrace:
Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette_test.py", line 283, in run
testMethod()
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_airplane_mode.py", line 35, in test_toggle_airplane_mode
system_app.wait_for_airplane_mode_icon_displayed()
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 91, in wait_for_airplane_mode_icon_displayed
Wait(self.marionette).until(expected.element_displayed(*self._airplane_mode_statusbar_locator))
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.0 seconds

Build info:
Device firmware (base) 	L1TC100118D0
Device firmware (date) 	09 Feb 2015 01:57:06
Device firmware (incremental) 	eng.cltbld.20150209.045655
Device firmware (release) 	4.4.2
Device identifier 	flame
Gaia date 	08 Feb 2015 12:44:20
Gaia revision 	0d7b35f23402
Gecko build 	20150209010211
Gecko revision 	3436787a82d0
Gecko version 	38.0a1

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/15/HTML_Report/

In the screenshot we can see the aiplane icon in the statusbat.
The issue is not reproduced manually.

Locally, with automation I got a reproduction rate of 5 out of 11.
(Reporter)

Updated

3 years ago
QA Whiteboard: [fxosqa-auto-backlog+]
(Assignee)

Comment 1

3 years ago
I can't reproduce with:
Gaia-Rev        0d7b35f23402c4cb29bca6b98280fec48a196dec
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/3436787a82d0
Build-ID        20150209010211
Version         38.0a1
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20150204.192122
FW-Date         Wed Feb  4 19:21:34 EST 2015
Bootloader      L1TC100118D0

Although I've turned off the connect_to_cell_data() function, since I currently don't have a working sim card here. But I don't think that should matter for this failure to be seen.
(Assignee)

Comment 2

3 years ago
Oh wait, I have been able to reproduce it actually, once out of 11 times.
(Assignee)

Comment 3

3 years ago
And out of another 11 times, it didn't reproduce. It's difficult to fix for me, since it's so hard to reproduce for me.
Created attachment 8572752 [details]
mar04_airplane.html

Repro rate for this seemed to get a lot worse: the failure on Jenkins seemed to indicate a connecting to data failure, however in local automation I encountered the same failure as in comment # 0, where the test is unable to locate the 'airplane mode' icon in the status bar. Does not reproduce manually, as the the icon is indeed updated and within the bar.

gaiatest --testvars=gaiatest/testvars_6253.json --address=localhost:2828 --restart --repeat=9 --timeout=30000 --html-output=results/mar04_airplane.html gaiatest/tests/functional/settings/test_settings_airplane_mode.py

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/64/testReport/junit/%28root%29/test_settings_airplane_mode_py%20TestAirplaneMode_test_toggle_airplane_mode/test_settings_airplane_mode_py_TestAirplaneMode_test_toggle_airplane_mode/

Jenkins:
Traceback (most recent call last):
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette_test.py", line 283, in run
    testMethod()
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_airplane_mode.py", line 35, in test_toggle_airplane_mode
    system_app.wait_for_airplane_mode_icon_displayed()
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 91, in wait_for_airplane_mode_icon_displayed
    Wait(self.marionette).until(expected.element_displayed(*self._airplane_mode_statusbar_locator))
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/wait.py", line 143, in until
    cause=last_exc)

TimeoutException: TimeoutException: Timed out after 30.1 seconds


Local (plus attached html):
Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_inbox_to_settings.py", line 21, in test_settings_from_inbox
    activities = messages.tap_options()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/apps/messages/app.py", line 34, in tap_options
    self.marionette.find_element(*self._options_icon_locator).tap()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 80, in tap
    return self.marionette._send_message('singleTap', 'ok', id=self.id, x=x, y=y)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 670, in _send_message
    self._handle_error(response)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 715, in _handle_error
    raise errors.ElementNotVisibleException(message=message, status=status, stacktrace=stacktrace)
ElementNotVisibleException: ElementNotVisibleException: Element is not currently visible and may not be manipulated
Flags: needinfo?(pbylenga)
Flags: needinfo?(pbylenga)
Created attachment 8573947 [details] [review]
[gaia] JohanLorenzo:bug-1131070 > mozilla-b2g:master
Blocks: 1140190
Attachment #8573947 - Flags: review?(martijn.martijn)
(Assignee)

Updated

3 years ago
Attachment #8573947 - Flags: review?(martijn.martijn) → review+
(Assignee)

Updated

3 years ago
Keywords: checkin-needed, leave-open

Updated

3 years ago
Keywords: checkin-needed

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
It seems like autolander doesn't support "leave-open".
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
(Assignee)

Updated

3 years ago
Assignee: nobody → martijn.martijn
Created attachment 8682136 [details] [review]
[gaia] mwargers:1131070 > mozilla-b2g:master
(Assignee)

Comment 9

3 years ago
Comment on attachment 8682136 [details] [review]
[gaia] mwargers:1131070 > mozilla-b2g:master

I repeated this test 10 times and they all passed. I think we should re-enable this test and see how it goes.
Attachment #8682136 - Flags: review?(npark)
Attachment #8682136 - Flags: review?(jlorenzo)

Updated

3 years ago
Attachment #8682136 - Flags: review?(npark) → review+
Comment on attachment 8682136 [details] [review]
[gaia] mwargers:1131070 > mozilla-b2g:master

Yeah, let's see how it goes under 512MB.
Attachment #8682136 - Flags: review?(jlorenzo) → review+
(Assignee)

Comment 11

3 years ago
Merged: https://github.com/mozilla-b2g/gaia/commit/4f65f9c741ef068b48f73d8f2da4b7a8a50368cf
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Depends on: 1221630
You need to log in before you can comment on or make changes to this bug.