1040063 - [v2.1][v2.0] Intermittent test_settings_do_not_track.py failure

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P1)

Description

[[:AndreiH]]

http://jenkins1.qa.scl3.mozilla.com/job/flame.mozilla-aurora.ui.functional.non-smoke/5/HTML_Report/


Traceback (most recent call last):
  File "/home/andreihutusoru/.virtualenvs/b2g-gaia/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 170, in run
    testMethod()
  File "/home/andreihutusoru/webQA/b2g-gaia/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_do_not_track.py", line 31, in test_enable_do_not_track_via_settings_app
    self.assertEqual(self.data_layer.get_bool_pref('privacy.donottrackheader.enabled'), True)
TEST-UNEXPECTED-FAIL | test_settings_do_not_track.py test_settings_do_not_track.TestSettingsDoNotTrack.test_enable_do_not_track_via_settings_app | AssertionError: False != True

Comment 1

[[:AndreiH]]

I remember seeing this on v2.1 too

Comment 2

[[:AndreiH]]

started adhock:

v2.0: http://jenkins1.qa.scl3.mozilla.com/view/Flame/job/flame.mozilla-aurora.ui.adhoc/1/console

master: http://jenkins1.qa.scl3.mozilla.com/view/Flame/job/flame.mozilla-central.ui.adhoc/3/console

Comment 3

[[:AndreiH]]

Attachment: Master PR

I don't know if you agree withe this changes, but it makes more sense to me.

Comment 4

[Zac C (:zac)]

AndreiH, it failed in the Aurora adhoc, are you sure this change is doing anything?

Comment 5

[Zac C (:zac)]

Comment on attachment 8458494 [details] [review]
Master PR

r- because the adhoc failed.
I suspect it's not actually tapping on the radio button properly, Intermittents are often the step before the one that's failing.

Comment 6

[[:AndreiH]]

Comment on attachment 8458494 [details] [review]
Master PR

Zac the adhoc is before the patch :) sorry I didn't specify that.

Comment 7

[[:AndreiH]]

Adhoc with the patch:
http://jenkins1.qa.scl3.mozilla.com/view/Flame/job/flame.mozilla-central.ui.adhoc/4/console

Comment 8

[[:AndreiH]]

The test failed again on v2.0:
http://jenkins1.qa.scl3.mozilla.com/view/Flame/job/flame.mozilla-aurora.ui.functional.non-smoke/7/HTML_Report/

Comment 9

[Florin Strugariu [:Bebe]]

Comment on attachment 8458494 [details] [review]
Master PR

see comment in the pull

I think adding those waits in the tests reduces the readability and we should wait for the changes in the app object

Comment 10

[Zac C (:zac)]

Comment on attachment 8458494 [details] [review]
Master PR

r- because the test failed again on adhoc. Moving the waits must not fix the problem, it must be something else, possibly the tap before the way not actually executing.

Comment 11

[Viorela Ioia [:viorela]]

Attachment: Github PR: https://github.com/mozilla-b2g/gaia/pull/22571

Comment 12

[Robert Chira [:RobertC]]

Comment on attachment 8468456 [details] [review]
Github PR: https://github.com/mozilla-b2g/gaia/pull/22571

You should make the pull for master and then uplift

Comment 13

[Viorela Ioia [:viorela]]

Attachment: Github PR: https://github.com/mozilla-b2g/gaia/pull/22629

I opened a PR on master and also started an adhoc run: http://jenkins1.qa.scl3.mozilla.com/job/flame.mozilla-central.ui.adhoc/91/console

Comment 14

[Robert Chira [:RobertC]]

https://github.com/mozilla-b2g/gaia/commit/dea4b90ca5e72625f7f7f27157709faba060bcd2

Comment 15

[Viorela Ioia [:viorela]]

Attachment: PR for v2.0: https://github.com/mozilla-b2g/gaia/pull/22717

Comment 16

[Robert Chira [:RobertC]]

v2.0 commit https://github.com/mozilla-b2g/gaia/commit/edaefce424b78505a82a79cc7a2b5eef9b5e53d3