Closed Bug 1127939 Opened 5 years ago Closed 5 years ago

Intermittent failing test, TEST-UNEXPECTED-FAIL | apps/search/test/marionette/suggestions_test.js | Search - Suggestions Test Test suggestions

Categories

(Firefox OS Graveyard :: Gaia::Search, defect)

x86
macOS
defect
Not set

Tracking

(b2g-v2.2 fixed)

RESOLVED FIXED
Tracking Status
b2g-v2.2 --- fixed

People

(Reporter: kgrandon, Assigned: daleharvey)

References

Details

(Keywords: intermittent-failure, Whiteboard: [systemsfe])

Attachments

(3 files)

Error: timeout exceeded!
02:50:10     INFO -      at Object.Client.waitForSync (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:693:16)
02:50:10     INFO -      at Object.Client.waitFor (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:661:60)
02:50:10     INFO -      at Context.<anonymous> (/builds/slave/test/gaia/apps/search/test/marionette/suggestions_test.js:51:12)
02:50:10     INFO -      at callFn (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:223:21)
02:50:10     INFO -      at Test.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:216:7)
02:50:10     INFO -      at Runner.runTest (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:373:10)
02:50:10     INFO -      at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:451:12
02:50:10     INFO -      at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:298:14)
02:50:10     INFO -      at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:308:7
02:50:10     INFO -      at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:246:23)
02:50:10     INFO -      at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:270:7
02:50:10     INFO -      at done (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:185:5)
02:50:10     INFO -      at callFn (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:228:7)
02:50:10     INFO -      at Hook.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:216:7)
02:50:10     INFO -      at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:258:10)
02:50:10     INFO -      at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:270:7
02:50:10     INFO -      at done (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:185:5)
02:50:10     INFO -      at /builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:199:9
02:50:10     INFO -      at Object.executeHook (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:371:18)
02:50:10     INFO -      at process._tickCallback (node.js:415:13)
This is failing fairly frequently, so disabling for now: https://github.com/mozilla-b2g/gaia/commit/b33a1afb538e8c7e12f6fb792c454da34f9f5b76

Dale - can you take a look if you have a minute?
Flags: needinfo?(dale)
Keywords: leave-open
Doh sorry, taking a look
Assignee: nobody → dale
Flags: needinfo?(dale)
Blocks: 1117970
The screenshot and logs suggested that the search app couldnt load some of its own resources, havent been able to reproduce in a while and have 10 passing runs @ https://treeherder.mozilla.org/#/jobs?repo=gaia-try&revision=ba2019828880
Attachment #8559823 - Flags: review?(kgrandon)
Attachment #8559823 - Flags: review?(bfrancis)
Comment on attachment 8559823 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/27835

Hmm, I really hope we're not masking over this with the new automatic retries, but sounds good. Let's try landing it for now.
Attachment #8559823 - Flags: review?(kgrandon)
Attachment #8559823 - Flags: review?(bfrancis)
Attachment #8559823 - Flags: review+
In master: https://github.com/mozilla-b2g/gaia/commit/e3a04fc63f25e8fb57a344c96f434c5cf173608a
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Appears this is still happening. You can see in the log it retries the same test several times. This means likely there is some obscure platform issue here, or we might have a bug with our build system/profile/mozsettings =/

Log: http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/b2g-inbound-linux64_gecko/1423191863/b2g-inbound_ubuntu64_vm-b2gdt_test-gaia-js-integration-3-bm54-tests1-linux64-build122.txt.gz

Backout: https://github.com/mozilla-b2g/gaia/commit/f6b79487eb188e7a818d578a23c54b10b684b6d3
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Ok figured this one out by sprinking random long timeouts, trying to come up with a fix
Kevin, cant quite figure out how to test this reliably, any idea?

    home.waitForLaunch();
    home.focusRocketBar();

    rocketbar.enterText('sometext');
    search.goToResults();
    // Ensure we get 2 results (hardcoded in the provider results)
    client.waitFor(function() {
      return client.findElements(search.Selectors.suggestions).length === 2;
    });

The issue is the SearchProvider constructor is async, it reads xhr / settings before being set, if the search is done before its set it returns empty results, I thought I could put the rocketbar.enterText('sometext'); inside the waitFor loop, but it exits on the first loop
Flags: needinfo?(kgrandon)
I think I would prefer to refactor the SearchProvider to ensure that we search after fetching the async settings - otherwise it may be possible to miss a search from a user? This is unlikely though..

Another solution would be to add some hook to the UI so we know we're ready.

I would prefer to refactor the provider a bit though so we store the searched terms, and search them after the constructor async init is done.
Flags: needinfo?(kgrandon)
Comment on attachment 8560529 [details] [review]
[PullReq] daleharvey:1127939 to mozilla-b2g:master

Well that was obvious, sorry should have figured that, will run a bunch on try and ensure there arent too many restarts before landing
Attachment #8560529 - Flags: review?(kgrandon)
Comment on attachment 8560529 [details] [review]
[PullReq] daleharvey:1127939 to mozilla-b2g:master

Looks good, I think we have an extraneous `resolve`. Fix that and I think we can land. Thanks!
Attachment #8560529 - Flags: review?(kgrandon) → review+
Removed the extra resolve, landed - https://github.com/mozilla-b2g/gaia/commit/d93a7f38449c7849e7724fbcc6f2557813b11248, thanks
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
Wanna uplift to make sure 2.2 is green too
blocking-b2g: --- → 2.2?
I cant set feature-b2b-2.2, can you, this isnt a blocking bug so much as part of feature dev
Flags: needinfo?(anygregor)
This is not blocking. Just request uplift approval.
blocking-b2g: 2.2? → ---
Flags: needinfo?(anygregor)
Whiteboard: [systemsfe]
Comment on attachment 8560529 [details] [review]
[PullReq] daleharvey:1127939 to mozilla-b2g:master

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #): Regressed by https://bugzilla.mozilla.org/show_bug.cgi?id=1098494 feature dev
[User impact] if declined: Can impact ability to test (and not regress) the rest of feature dev on the branch
[Testing completed]: This patch increases the reliability of the tests 
[Risk to taking this patch] (and alternatives if risky): little risk
[String changes made]: None
Attachment #8560529 - Flags: approval-gaia-v2.2?
Attachment #8560529 - Flags: approval-gaia-v2.2? → approval-gaia-v2.2+
Due to nature of STR we are unable to verify on our end.
QA Whiteboard: [QAnalyst-Triage?][QAnalyst-verify-]
Flags: needinfo?(ktucker)
QA Whiteboard: [QAnalyst-Triage?][QAnalyst-verify-] → [QAnalyst-Triage+][QAnalyst-verify-]
Flags: needinfo?(ktucker)
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.