Description: test_rocketbar_offline_behavior appears to not be spawning the keyboard when attempting an offline search. Repro Steps: 1) Update phone to 20150423010203 2) Enable Wi-Fi & Data 3) Enable Airplane mode (lose network connection) 4) Tap into Rocketbar Search bar: search for 'test' 5) Observe UI Manual: Rocketbar displays 'No Internet Connection' label; keyboard prompts and lets user type Automated: No keyboard displayed, http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/805/HTML_Report/ http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/176/HTML_Report/ Traceback (most recent call last): File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_client-0.11-py2.7.egg/marionette/marionette_test.py", line 296, in run testMethod() File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/tests/functional/rocketbar/test_rocketbar_offline_behavior.py", line 28, in test_rocketbar_offline_behavior search_panel.type_into_search_box(test_string) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py", line 43, in type_into_search_box self.keyboard.send(search_term) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 232, in send self.switch_to_keyboard() File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/apps/keyboard/app.py", line 167, in switch_to_keyboard % (input_window.is_displayed(), input_window.get_attribute('class'))) File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_driver-0.4-py2.7.egg/marionette_driver/wait.py", line 143, in until cause=last_exc) TimeoutException: TimeoutException: Timed out after 29.9 seconds with message: Keyboard inputWindow not interpreted as displayed. Debug is_displayed(): False, class: inputWindow. Environmental Variables: Device firmware (base) L1TC100118D0 Device firmware (date) 23 Apr 2015 08:29:45 Device firmware (inc-remental) eng.cltbld.20150423.042933 Device firmware (release) 4.4.2 Device identifier flame Device memory 219772 kB Device serial e472d8d7 Device uptime 0 days 0 hours 2 minutes 13 seconds Gaia date 22 Apr 2015 17:32:36 Gaia revision 9d4f756aa35c Gecko build 20150423010203 Gecko revision 0b202671c9e2 Gecko version 40.0a1 Reproducing in automation: 2/6 passing Reproducible manually: No Repro frequency: 5/5 passing
QA Whiteboard: [QAnalyst-Triage?][fxosqa-auto-backlog?]
We also should make sure that the keyboard is fully visible here.
QA Whiteboard: [QAnalyst-Triage?][fxosqa-auto-backlog?] → [QAnalyst-Triage+][fxosqa-auto-backlog?]
Comment on attachment 8597335 [details] [review] [gaia] mwargers:1157772 > mozilla-b2g:master Oliver, if you could try this out and see if it fixes the intermittent failure, that would be great. I think this will fix it. This code is horrible, but it is still necessary as bug 1020974 is still around.
Comment on attachment 8597335 [details] [review] [gaia] mwargers:1157772 > mozilla-b2g:master So I adhoced this behavior via Jenkins and saw a 10/10 passing rate, so it's hard to see where this fix may ensure an intermittent failure doesn't occur. http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/810/HTML_Report/ I also performed the test locally on a machine with and without your fix and observed 10/10 passing on both. Not sure if something else already fixed this, or the repro rate lowered from some other cause. If there's anything else I should test around, don't hesitate to ask.
Comment on attachment 8597335 [details] [review] [gaia] mwargers:1157772 > mozilla-b2g:master I don't minus the patch, because I don't know if this fix is worth it. I'd like to know what Geo and John think about it. IMO, I'd prefer to have this test intermittent, rather than adding 4 more seconds to each execution of the test. The value is too magic and I think waiting 5 seconds can hide some performance issues.
(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #6) > Comment on attachment 8597335 [details] [review] > [gaia] mwargers:1157772 > mozilla-b2g:master > > I don't minus the patch, because I don't know if this fix is worth it. I'd > like to know what Geo and John think about it. > > IMO, I'd prefer to have this test intermittent, rather than adding 4 more > seconds to each execution of the test. The value is too magic and I think > waiting 5 seconds can hide some performance issues. OK, thoughts: * I personally don't care about hiding performance and don't think that should be considered. These are functional tests, and should by and large adapt to the responsiveness of the system under test. For proper isolation, use performance timing tests to ensure performance. * Waiting 5 seconds during one test isn't a big deal. If we were doing this in some function being used by global test startup code or something I'd care a lot more. * That said, it sure is a magic number. At the very least I'd like some documentation in the way of a comment as to why this is being done. It should read something like "A small amount of time is required for .... to initialize" or something--don't follow the "workaround for bug foo" example, that's meaningless. has to be relevant to the operation at hand. * Being a magic sleep, this won't be resistant to slow builds like debug builds and emulator. It would possibly be better if there was a Wait done here instead, even if it's a wait whose polling function repeatedly taps until the keyboard actually appears (keyboard appearing is the wait success condition). Of course, at that point the test no longer tests the Rocketbar appearing after an immediate tap. But that's true of the sleep as well. Best bet is to actually figure out -why- the keyboard is nonresponsive and put a wait on that if appropriate. * Man, not Martijn's fault, but tap_search_bar() is no longer a good name for that function. It's display_search_panel() now. A tap function should look for the element, tap, then exit. Once this kind of logic is in there, it's a "ensure something happens" function, not a "do an input" function and we should have named it after the thing that you're returning/ensuring. TL;DR/Summary: You should dig further and see if you can do a wait on it, though if you told me this was our only route I'd probably r+ it. And Oliver can better look to see if this improves things by going to something more like 50 tries. 10 tries isn't always enough. That said, if we're talking once in 50 tries or something for the original, I agree with Johan: maybe just leave it be.
I'll wait for a reply as to whether this should be dropped or investigated further before r+'ing.
Comment on attachment 8597335 [details] [review] [gaia] mwargers:1157772 > mozilla-b2g:master Ok, I'm now using Wait calls to fix the intermittent failure. This similar thing was used here: http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py#53 I wasn't able to remove the last remaining time.sleep call, but I think this is an improvement of what we have.
Looks great! I started  to weigh up the effects of your patch.  http://jenkins1.qa.scl3.mozilla.com/job/flame-kk.ui.adhoc/817/
It seems like we have a Marionette error. This might be caused by the upgrade that happened a couple of days ago. Could you see if a rebase of your PR helps to fix the error we see in Jenkins,Martijn?  https://github.com/mozilla-b2g/gaia/commit/277ff361005fc37b764e01e2d32fec072a0189af
Comment on attachment 8600894 [details] [review] [gaia] mwargers:1157772_v2 > mozilla-b2g:master Rebasing always causing trouble here, so I just created a new branch, applied the patch and pushed that.
Attachment #8600894 - Flags: review?(jlorenzo)
Comment on attachment 8600894 [details] [review] [gaia] mwargers:1157772_v2 > mozilla-b2g:master Not a single failure in 100 tries. This patch looks like a fix! Worst case scenario, we remove a time.sleep(), which is awesome. r+!
Attachment #8600894 - Flags: review?(jlorenzo) → review+
Comment on attachment 8600894 [details] [review] [gaia] mwargers:1157772_v2 > mozilla-b2g:master Fix looks good to me.
Attachment #8600894 - Flags: review?(npark) → review+
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.