1179891 - test_ftu_skip_tour.py: "TimeoutException: Timed out after 10.3 seconds with message: No languages listed on screen"

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::First Time Experience, defect)

Description

[Oliver Nelson [:oliverthor]]

Description:
test_ftu_skip_tour is failing consistently now due to a format change for the languages in the FTU. THe languages themselves also have a long delay before appearing, which may be the underlying cause behind the failure.
This could be blocking bug 1179267 now.

Repro Steps:
1) Update phone to 20150702010204
2) Observe languages in beginning of FTU

Actual:
Language display is delayed with different formatting, test fails

Expected: 
Test selects language and continues
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.16-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/ftu/test_ftu_skip_tour.py", line 40, in test_ftu_skip_tour
self.wait_for_condition(lambda m: self.ftu.languages_list > 0, message="No languages listed on screen")
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 1122, in wait_for_condition
Wait(self.marionette, timeout).until(method, message=message)
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_driver-0.9-py2.7.egg/marionette_driver/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 10.3 seconds with message: No languages listed on screen


Environmental Variables:
Device firmware (base) 	L1TC100118D0
Device firmware (date) 	02 Jul 2015 08:43:16
Device firmware (incremental) 	eng.cltbld.20150702.044305
Device firmware (release) 	4.4.2
Device identifier 	flame
Device memory 	219772 kB
Device serial 	1b41c231
Device uptime 	0 days 0 hours 2 minutes 28 seconds
Gaia date 	01 Jul 2015 20:03:18
Gaia revision 	b901c8b7be21
Gecko build 	20150702010204
Gecko revision 	f5e3bacfb60e
Gecko version 	42.0a1


Reproducible manually: NO

Repro frequency: 0/5

Comment 1

[Oliver Nelson [:oliverthor]]

Repro Frequency should be 5/5; this is always failing currently.

* http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke.bitbar/260/
* http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc.bitbar/115/HTML_Report/

Comment 2

[Martijn Wargers (dead)]

Yes, I'm looking at that and trying to fix that right now as part of bug 1179267.

Comment 3

[Martijn Wargers (dead)]

Attachment: https://github.com/mozilla-b2g/gaia/pull/30810

Actually, it looks like there is a real bug in bug 1179267.
There are 3 parts in this pull request:
- Adjustments of the locators so the failure in this bug is fixed
- Using execute_script calls to access the .checked property of gaia-checkbox and gaia-switch (needed because of bug 1113742). Apparently, this code was changed to use gaia-checkbox and gaia-switch now. Gaia-checkbox was introduced for "shared data" in bug 1176905.
- The geolocation part gets the same treatment as the data_share one was done in bug 1134257 (this is not really necessary, but it is nice to have, a better check than we currently use).

The issue in bug 1179267 is that data_share gaia-checkbox checking/unchecking doesn't change the debug.performance_data.shared setting. This might be a regression from bug 1176905.

Comment 4

[Martijn Wargers (dead)]

Perhaps this: http://mxr.mozilla.org/gaia/source/apps/ftu/js/ui.js#262
262     this.sharePerformance.addEventListener('click', this);
Needs to be using 'change' event now?

Comment 5

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8629144 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/30810

I'm not sure acceptance tests should check the back end state anymore. But apart from that, everything looks good!

Comment 6

[Martijn Wargers (dead)]

But integration tests are not run on the device at all, are they?
Bug 1179267 was caught with this Gaia UI test, not with any integration test (how can they, they are not run on device), even though that part of the test would be considered an integration test.

Comment 7

[Martijn Wargers (dead)]

We decided that we should keep the back end state check in, because currently, there are no integration tests running on device.

Merged: https://github.com/mozilla-b2g/gaia/commit/d2706da8d541e8d6efd28bac2052402e46035daf