Closed Bug 1102931 Opened 5 years ago Closed 2 years ago

Intermittent test_ftu_with_tour.py TestFtu.test_ftu_with_tour | TimeoutException: TimeoutException: Timed out after 10.1 seconds

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

x86
Gonk (Firefox OS)
defect
Not set

Tracking

(b2g-v2.2 fixed, b2g-master fixed)

RESOLVED WONTFIX
2.2 S4 (23jan)
Tracking Status
b2g-v2.2 --- fixed
b2g-master --- fixed

People

(Reporter: cbook, Assigned: viorela)

References

()

Details

(Keywords: intermittent-failure)

Attachments

(5 files)

b2g_macosx64 mozilla-inbound opt test gaia-ui-test-functional-1

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=4105453&repo=mozilla-inbound

03:09:32 ERROR - TEST-UNEXPECTED-ERROR | test_ftu_with_tour.py TestFtu.test_ftu_with_tour | TimeoutException: TimeoutException: Timed out after 10.1 seconds
03:09:32 INFO - Traceback (most recent call last):
03:09:32 INFO - File "/builds/slave/talos-slave/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 267, in run
03:09:32 INFO - testMethod()
03:09:32 INFO - File "/builds/slave/talos-slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/ftu/test_ftu_with_tour.py", line 29, in test_ftu_with_tour
03:09:32 INFO - self.assertEqual(self.ftu.step1_header_text, "Swipe up and down to browse your apps and bookmarks. Tap and hold an icon to delete, move, or edit it.")
03:09:32 INFO - File "/builds/slave/talos-slave/test/gaia/tests/python/gaia-ui-tests/gaiatest/apps/ftu/app.py", line 382, in step1_header_text
03:09:32 INFO - Wait(self.marionette).until(expected.element_displayed(header))
03:09:32 INFO - File "/builds/slave/talos-slave/test/build/venv/lib/python2.7/site-packages/marionette/wait.py", line 143, in until
03:09:32 INFO - cause=last_exc)
03:09:32 INFO - TEST-INFO took 27407ms
Assignee: nobody → viorela.ioia
QA Whiteboard: [fxosqa-auto-s5]
Attachment #8529808 - Flags: review?(robert.chira)
Attachment #8529808 - Flags: review?(florin.strugariu)
Attachment #8529808 - Flags: review?(florin.strugariu) → review+
I will leave this bug open to see if the issue was fixed.
Please reopen if this is failing again
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
failed again - https://treeherder.mozilla.org/logviewer.html#?job_id=741139&repo=mozilla-central
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
QA Whiteboard: [fxosqa-auto-s5] → [fxosqa-auto-from-s5][fxosqa-auto-s6]
Attachment #8535462 - Flags: review?(robert.chira) → review+
Attachment #8535462 - Flags: review?(florin.strugariu) → review+
QA Whiteboard: [fxosqa-auto-from-s5][fxosqa-auto-s6] → [fxosqa-auto-from-s5][fxosqa-auto-s7]
I've been working on finding a suitable fix for this failure, but it was really difficult, as I couldn't reproduce the failure locally, though I ran the test ~500 times on mac. 

I opened a PR with a potential fix[1], and I retriggered the job that contains this test several times[2]
I added a time.sleep(1) in [1], in my attempt to fix this intermittency issue. Though the results of the test runs are green, I can't assure that this test won't fail again.
So my question here is: should we try this fix, though it has a sleep, or disable this test on mac?

[1] https://github.com/mozilla-b2g/gaia/pull/26888
[2] https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=246437b6c7ad
Flags: needinfo?(gmealer)
Flags: needinfo?(florin.strugariu)
(In reply to Viorela Ioia [:viorela] from comment #29)

> So my question here is: should we try this fix, though it has a sleep, or
> disable this test on mac?

I'm not worried about the sleep, more that we're not sure if/why that would fix it.

This has a much higher than 1:500 repro rate on Treeherder--looks pretty close to 1:1. Were we able to try repro'ing or your fix against the similar environment?
Flags: needinfo?(gmealer)
Restarted the build a few times on mac to check the stability 

The sleep looks OK to me fi there is no other option
Flags: needinfo?(florin.strugariu)
I retriggered the job several times on gaia-try, results are looking good: https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=4e908e0622f1
Attachment #8547561 - Flags: review?(robert.chira)
Attachment #8547561 - Flags: review?(florin.strugariu)
Attachment #8547561 - Flags: review?(robert.chira) → review+
Attachment #8547561 - Flags: review?(florin.strugariu) → review+
https://github.com/mozilla-b2g/gaia/commit/b59b59792f927956d23f672b35a4e3b35f6a651f
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
I had to reopen this bug, as the previous patch didn't fix the issue, though the results in gaia-try were looking good. Bug 1120756 was open, because the issue re-appeared.
As I can't either reproduce the failure locally(using a similar env) or find a suitable fix, I will open a PR to disable this test on mac for now.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
PR to disable the test on mac
Attachment #8548858 - Flags: review?(florin.strugariu)
Comment on attachment 8548858 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/27382

As we are not running this test on mac can you also revert the changes introduced in the previous pull?

https://github.com/mozilla-b2g/gaia/commit/b59b59792f927956d23f672b35a4e3b35f6a651f#diff-5ec76afef7d24a7fd7a3f5db907687b6R29
Attachment #8548858 - Flags: review?(florin.strugariu) → review-
Attachment #8548858 - Flags: review- → review?(florin.strugariu)
Attachment #8548863 - Flags: review?(florin.strugariu) → review+
Comment on attachment 8548858 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/27382

all looks ok to me... Geo can you give us the final r? for disabling this test on mac
Attachment #8548858 - Flags: review?(gmealer)
Attachment #8548858 - Flags: review?(florin.strugariu)
Attachment #8548858 - Flags: review+
Duplicate of this bug: 1120756
Comment on attachment 8548858 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/27382

Merging this in as this test is still generating intermitents
Attachment #8548858 - Flags: review?(gmealer)
The test was disabled on mac.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
Back now that we've switched to TC.
Status: RESOLVED → REOPENED
Flags: needinfo?(kgrandon)
Resolution: FIXED → ---
Mass clear of needinfo as lots of TC changes have recently been deployed. I am still tracking and working on getting these issues fixed, but will work with the most egregious offenders first. Flag me again if needed.
Flags: needinfo?(kgrandon)
The robot message is totally unhelpful, this is the real error:
01:20:45    ERROR -  TEST-UNEXPECTED-ERROR | test_ftu_with_tour.py TestFtu.test_ftu_with_tour | ScriptTimeoutException: ScriptTimeoutException: timed out
01:20:45     INFO -  Traceback (most recent call last):
01:20:45     INFO -    File "/home/worker/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 277, in run
01:20:45     INFO -      self.setUp()
01:20:45     INFO -    File "/home/worker/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/ftu/test_ftu_with_tour.py", line 13, in setUp
01:20:45     INFO -      GaiaTestCase.setUp(self)
01:20:45     INFO -    File "/home/worker/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 805, in setUp
01:20:45     INFO -      self.cleanup_gaia(full_reset=False)
01:20:45     INFO -    File "/home/worker/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 888, in cleanup_gaia
01:20:45     INFO -      self.data_layer.set_volume(0)
01:20:45     INFO -    File "/home/worker/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 257, in set_volume
01:20:45     INFO -      self.set_setting('audio.volume.%s' % channel, value)
01:20:45     INFO -    File "/home/worker/gaia/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 217, in set_setting
01:20:45     INFO -      result = self.marionette.execute_async_script('return GaiaDataLayer.setSetting("%s", %s)' % (name, value), special_powers=True)
01:20:45     INFO -    File "/home/worker/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1489, in execute_async_script
01:20:45     INFO -      debug_script=debug_script)
01:20:45     INFO -    File "/home/worker/build/venv/local/lib/python2.7/site-packages/marionette_driver/decorators.py", line 36, in _
01:20:45     INFO -      return func(*args, **kwargs)
01:20:45     INFO -    File "/home/worker/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line 716, in _send_message
01:20:45     INFO -      self._handle_error(response)
01:20:45     INFO -    File "/home/worker/build/venv/local/lib/python2.7/site-packages/marionette_driver/marionette.py", line

That is an error that is completely different than what this bug is about, initially.

I assume TC = Taskcluster? It seems like an error in setup. Is this an issue with taskcluster, perhaps?
Firefox OS is not being worked on
Status: REOPENED → RESOLVED
Closed: 5 years ago2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.