1208522 - test_cost_control_data_alert_mobile.py : TimeoutException: TimeoutException: Timed out after 180.1 seconds

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Max Ivanov[:MaxIvanov]]

Description: Test case fails with Timeout" error or ElementNotVisibleException

Steps:
1. Update a flame device to 20150923030227
2. Run this test case http://lxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_data_alert_mobile.py

Actual result:
Timeout" error or ElementNotVisibleException

Expected result:
Test completes successfully.

Repro Rate: Reproducible via Jenkins (1/5 passing, unable to reproduce manually.
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk.ui.adhoc.bitbar/251/HTML_Report/

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/461/testReport/junit/%28root%29/test_cost_control_data_alert_mobile_py%20TestCostControlDataAlertMobile_test_cost_control_data_alert_mobile/test_cost_control_data_alert_mobile_py_TestCostControlDataAlertMobile_test_cost_control_data_alert_mobile/

Test case fails with two different errors:
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_data_alert_mobile.py", line 48, in test_cost_control_data_alert_mobile
system.wait_for_notification_toaster_displayed(timeout=180)
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 42, in wait_for_notification_toaster_displayed
Wait(self.marionette, timeout).until(expected.element_displayed(el), message=message)
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 180.1 seconds

-----------------------------------------------------------------------------

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_data_alert_mobile.py", line 42, in test_cost_control_data_alert_mobile
search.go_to_url('http://mozqa.com/qa-testcase-data/Images/sample_png_02.png')
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/search/app.py", line 32, in go_to_url
return search_panel.go_to_url(url)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py", line 57, in go_to_url
self.marionette.find_element(*self._rocketbar_input_locator).tap()
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 81, in tap
self.marionette._send_message("singleTap", body)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/decorators.py", line 36, in _
return func(*args, **kwargs)
File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 711, in _send_message
    self._handle_error(resp)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 752, in _handle_error
    raise errors.lookup(error)(message, stacktrace=stacktrace)
ElementNotVisibleException: ElementNotVisibleException: Element is not currently visible and may not be manipulated

Comment 1

[Martijn Wargers (dead)]

The 2nd failure I suspect happens, because the browser gets mlk-ed, there is bug 1204837 out there that might fix this.

As for the first failure, I was able to reproduce this the first time. The thing is, the data alert system notification didn't show up, while it should, because the png image is 264kb big.
After that, I couldn't reproduce anymore, I tried 4 times afterwards.
I'm going to retry using 296MB on my phone and see if I can then reproduce this.

Comment 2

[Martijn Wargers (dead)]

I give up, I now tried 2 times with 296MB and this test now keeps on passing locally for me.

Comment 3

[Martijn Wargers (dead)]

I'm now getting this failure with this test, because the data alert notification already shows up while the browser is starting up:
TEST-START | test_cost_control_data_alert_mobile.py TestCostControlDataAlertMobile.test_cost_control_data_alert_mobile
TEST-UNEXPECTED-ERROR | test_cost_control_data_alert_mobile.py TestCostControlDataAlertMobile.test_cost_control_data_alert_mobile | ElementNotVisibleException: ElementNotVisibleException: Element is not currently visible and may not be manipulated


Traceback (most recent call last):
  File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/tests/functional/cost_control/test_cost_control_data_alert_mobile.py", line 42, in test_cost_control_data_alert_mobile
    search.go_to_url('http://mozqa.com/qa-testcase-data/Images/sample_png_02.png')
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/apps/search/app.py", line 32, in go_to_url
    return search_panel.go_to_url(url)
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/apps/homescreen/regions/search_panel.py", line 57, in go_to_url
    self.marionette.find_element(*self._rocketbar_input_locator).tap()
  File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 81, in tap
    self.marionette._send_message("singleTap", body)
  File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 711, in _send_message
    self._handle_error(resp)
  File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_driver-0.13-py2.7.egg/marionette_driver/marionette.py", line 752, in _handle_error
    raise errors.lookup(error)(message, stacktrace=stacktrace)
TEST-INFO took 133935ms

Comment 4

[Martijn Wargers (dead)]

It looks like when I set the "privacy.trackingprotection.enabled" preference to false again, I don't get this failure.

Could it be that the tracking protection code is consuming a couple of MB's lately, when the browser is starting up, perhaps?
We have a limit of 0.1MB in the test, so we're hitting this quickly here.
When a user is using cell data, they don't want to consume much data, because it can be costly.

Comment 5

[GitHub Autolander Bot]

Attachment: [gaia] mwargers:1208522 > mozilla-b2g:master

Comment 6

[Martijn Wargers (dead)]

Comment on attachment 8666408 [details] [review]
[gaia] mwargers:1208522 > mozilla-b2g:master

This makes the test pass every time with 5 repeats again.
I guess we could use this as a workaround for now.
Unless perhaps it is expected that the privacy protection feature can consume a couple of MB's even when using cell data. Then this would be a permanent solution, I guess.

Comment 7

[François Marier [:francois]]

Tracking protection will cause two lists to be downloaded:

  mozstd-track-digest256 (52KB)
  mozstd-trackwhite-digest256 (300KB)

I would expect that it pays for itself after loading a page or two from a typical news site. Monica found that it decreases data usage by 39% (http://ieee-security.org/TC/SPW2015/W2SP/papers/W2SP_2015_submission_32.pdf).

So turning it off will actually increase data costs for our users. We should probably change the test instead.

Comment 8

[Martijn Wargers (dead)]

Ok, thanks, then the pull request is the right solution.

Comment 9

[No-Jun Park [:njpark]]

Comment on attachment 8666408 [details] [review]
[gaia] mwargers:1208522 > mozilla-b2g:master

Seems straightforward to me. r+

Comment 10

[Johan Lorenzo [:jlorenzo]]

Comment on attachment 8666408 [details] [review]
[gaia] mwargers:1208522 > mozilla-b2g:master

Looks good to me.

Comment 11

[Martijn Wargers (dead)]

Merged: https://github.com/mozilla-b2g/gaia/commit/e11dbcaa01dffcbabc4a0b1225cbfb2923285165