996662 - Rocketbar tests are intermittently failing

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Search, defect)

Description

[Kevin Grandon :kgrandon]

Attachment: Screenshot

These tests have been failing way too much on travis unfortunately. We need to disable them for now while we investigate.


1) Places tests Search for a string that doesnt match visited url:

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:683:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:651:60)

at Object.Rocketbar.focus (/home/travis/build/mozilla-b2g/gaia/apps/system/test/marionette/lib/rocketbar.js:52:17)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/search/test/marionette/places_search_test.js:49:15)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:211:32)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:372:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:448:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:297:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:307:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:245:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:213:5)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:257:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:197:9

at Object.executeHook (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)

at process._tickCallback (node.js:415:13)

Comment 1

[Kevin Grandon :kgrandon]

Disabled on travis: https://github.com/mozilla-b2g/gaia/commit/83526b1084442a1476c9465e880332bed5353ca2

Comment 2

[Kevin Grandon :kgrandon]

And we have another test failing. It appears that all tests are failing on the same line, 52.


Rocketbar Navigate to URL:

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:683:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:651:60)

at Object.Rocketbar.focus (/home/travis/build/mozilla-b2g/gaia/apps/system/test/marionette/lib/rocketbar.js:52:17)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/system/test/marionette/rocketbar_test.js:60:15)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:211:32)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:372:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:448:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:297:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:307:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:245:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:213:5)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:257:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:197:9

at Object.executeHook (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)

at process._tickCallback (node.js:415:13)

Comment 3

[Dale Harvey (:daleharvey) PTO until Jan 2020]

Yup these have been failing intermittently on my patch and I thought it was due to my bad, back them out and I will get on reenabling them

Comment 4

[Dale Harvey (:daleharvey) PTO until Jan 2020]

s/backout/skip

Comment 5

[Kevin Grandon :kgrandon]

Yeah, just going to skip them on travis. I have seen them fail before, but they've recently gone way up, probably fail ~25% of the time. We probably just need to tweak a few wait conditions in rocketbar.js.

Disabled tests here: Disabled tests here: https://github.com/mozilla-b2g/gaia/commit/12633a17052d514d1827fa5c5496585fa5f08748

Comment 6

[Kevin Grandon :kgrandon]

Turns out these are failing on TBPL quite a lot as well. Disabled for now until we can investigate: https://github.com/mozilla-b2g/gaia/commit/fd745244dad5f213fbc44bf12c8a78c71e638b5f

Comment 7

[Kevin Grandon :kgrandon]

Additionally two tests which we must absolutely add that have regressed recently:

1 - Bookmarking a result after clicking it from places.
2 - Web results render properly in fullscreen.

Comment 8

[Kevin Grandon :kgrandon]

Attachment: Github pull request

Hey Guys - could anyone help me review this patch real quick? Thanks!

I am not entirely sure but I think the race condition may have been due to the fact that we were tapping on rocketbar before adding the event listeners. I have not been able to reproduce the failures since this patch, and before it was quite easy to do. (Ran 20+ times on travis).

Comment 9

[Dale Harvey (:daleharvey) PTO until Jan 2020]

Comment on attachment 8409159 [details] [review]
Github pull request

Apologies for not being ultra quick, needs rebased, but +1

Comment 10

[Kevin Grandon :kgrandon]

Comment on attachment 8409159 [details] [review]
Github pull request

Let's go with Dale's review for now. If you guys have any follow-up comments I will address them. Thanks!

Comment 11

[Kevin Grandon :kgrandon]

Landed: https://github.com/mozilla-b2g/gaia/commit/c4c18d9ca546d50b732eea6ff252d35e4e9d38db

Comment 12

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.