Closed Bug 996662 Opened 6 years ago Closed 6 years ago

Rocketbar tests are intermittently failing

Categories

(Firefox OS Graveyard :: Gaia::Search, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
1.4 S6 (25apr)

People

(Reporter: kgrandon, Assigned: kgrandon)

References

Details

(Whiteboard: [systemsfe])

Attachments

(2 files)

Attached image Screenshot
These tests have been failing way too much on travis unfortunately. We need to disable them for now while we investigate.


1) Places tests Search for a string that doesnt match visited url:

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:683:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:651:60)

at Object.Rocketbar.focus (/home/travis/build/mozilla-b2g/gaia/apps/system/test/marionette/lib/rocketbar.js:52:17)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/search/test/marionette/places_search_test.js:49:15)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:211:32)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:372:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:448:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:297:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:307:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:245:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:213:5)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:257:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:197:9

at Object.executeHook (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)

at process._tickCallback (node.js:415:13)
And we have another test failing. It appears that all tests are failing on the same line, 52.


Rocketbar Navigate to URL:

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:683:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:651:60)

at Object.Rocketbar.focus (/home/travis/build/mozilla-b2g/gaia/apps/system/test/marionette/lib/rocketbar.js:52:17)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/system/test/marionette/rocketbar_test.js:60:15)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:211:32)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:372:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:448:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:297:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:307:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:245:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:213:5)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:257:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:197:9

at Object.executeHook (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)

at process._tickCallback (node.js:415:13)
Summary: Intermittent failing test, Places tests Search for a string that doesnt match visited url → Rocketbar tests are intermittently failing
Yup these have been failing intermittently on my patch and I thought it was due to my bad, back them out and I will get on reenabling them
Yeah, just going to skip them on travis. I have seen them fail before, but they've recently gone way up, probably fail ~25% of the time. We probably just need to tweak a few wait conditions in rocketbar.js.

Disabled tests here: Disabled tests here: https://github.com/mozilla-b2g/gaia/commit/12633a17052d514d1827fa5c5496585fa5f08748
Turns out these are failing on TBPL quite a lot as well. Disabled for now until we can investigate: https://github.com/mozilla-b2g/gaia/commit/fd745244dad5f213fbc44bf12c8a78c71e638b5f
See Also: → 996969
Additionally two tests which we must absolutely add that have regressed recently:

1 - Bookmarking a result after clicking it from places.
2 - Web results render properly in fullscreen.
Attached file Github pull request
Hey Guys - could anyone help me review this patch real quick? Thanks!

I am not entirely sure but I think the race condition may have been due to the fact that we were tapping on rocketbar before adding the event listeners. I have not been able to reproduce the failures since this patch, and before it was quite easy to do. (Ran 20+ times on travis).
Assignee: nobody → kgrandon
Status: NEW → ASSIGNED
Attachment #8409159 - Flags: review?(gaye)
Attachment #8409159 - Flags: review?(evanxd)
Attachment #8409159 - Flags: review?(dale)
Comment on attachment 8409159 [details] [review]
Github pull request

Apologies for not being ultra quick, needs rebased, but +1
Attachment #8409159 - Flags: review?(dale) → review+
Comment on attachment 8409159 [details] [review]
Github pull request

Let's go with Dale's review for now. If you guys have any follow-up comments I will address them. Thanks!
Attachment #8409159 - Flags: review?(gaye)
Attachment #8409159 - Flags: review?(evanxd)
Landed: https://github.com/mozilla-b2g/gaia/commit/c4c18d9ca546d50b732eea6ff252d35e4e9d38db
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Whiteboard: [systemsfe]
Target Milestone: --- → 1.4 S6 (25apr)
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.