Closed Bug 1135071 Opened 11 years ago Closed 10 years ago

Intermittent app_installed_test.js | Search - Installed Apps Test app result after install/uninstall

Categories

(Firefox OS Graveyard :: Gaia, defect)

Product:
Firefox OS Graveyard
Component:
Gaia
Platform:
x86
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: cbook, Assigned: mikehenrty)

References

(
URL
)

Details

(Keywords: crash, intermittent-failure, Whiteboard: [systemsfe])

Attachments

(3 files, 1 obsolete file)

screenshot.png
74.88 KB, image/png
Details
[gaia] albertopq:1135071-fix-tests-v2.1 > mozilla-b2g:v2.1
46 bytes, text/x-github-pull-request
kgrandon
: review+
Details | Review
[gaia] mikehenrty:bug-1135071-app-installed-test-fix > mozilla-b2g:master
46 bytes, text/x-github-pull-request
Details | Review
b2g_ubuntu64_vm b2g-inbound opt test gaia-js-integration-3 https://treeherder.mozilla.org/logviewer.html#?job_id=1379403&repo=b2g-inbound 21:44:48 INFO - TEST-UNEXPECTED-FAIL | apps/search/test/marionette/app_installed_test.js | Search - Installed Apps Test app result after install/uninstall 21:44:48 INFO - Error: Polling socket recv() timeout! 21:44:48 INFO - at TcpSync._readResponse (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/drivers/tcp-sync.js:146:30) 21:44:48 INFO - at TcpSync.send (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/drivers/tcp-sync.js:159:24) 21:44:48 INFO - at Object.send (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:459:36) 21:44:48 INFO - at Object.Client._sendCommand (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:509:21) 21:44:48 INFO - at Object._executeScript (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:1482:19) 21:44:48 INFO - at Object.executeAsyncScript (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:1260:19) 21:44:48 INFO - at Object.Client.waitForSync (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:687:14) 21:44:48 INFO - at Object.Client.waitFor (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:661:60) 21:44:48 INFO - at Object.MarionetteHelper.waitForElement (/builds/slave/test/gaia/node_modules/marionette-helper/index.js:142:12) 21:44:48 INFO - at Object.Search.goToResults (/builds/slave/test/gaia/apps/search/test/marionette/lib/search.js:46:24) 21:44:48 INFO - at Object.Search.triggerFirstRun (/builds/slave/test/gaia/apps/search/test/marionette/lib/search.js:107:10) 21:44:48 INFO - at Context.<anonymous> (/builds/slave/test/gaia/apps/search/test/marionette/app_installed_test.js:33:12) 21:44:48 INFO - at callFn (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:223:21) 21:44:48 INFO - at Test.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:216:7) 21:44:48 INFO - at Runner.runTest (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:373:10) 21:44:48 INFO - at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:451:12 21:44:48 INFO - at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:298:14) 21:44:48 INFO - at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:308:7 21:44:48 INFO - at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:246:23) 21:44:48 INFO - at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:270:7 21:44:48 INFO - at done (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:185:5) 21:44:48 INFO - at /builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:199:9 21:44:48 INFO - at /builds/slave/test/gaia/apps/search/test/marionette/app_installed_test.js:26:7 21:44:48 INFO - at ChildProcess.<anonymous> (/builds/slave/test/gaia/apps/verticalhome/test/marionette/server/parent.js:183:5) 21:44:48 INFO - at ChildProcess.EventEmitter.emit (events.js:98:17) 21:44:48 INFO - at handleMessage (child_process.js:318:10) 21:44:48 INFO - at Pipe.channel.onread (child_process.js:345:11) 21:44:48 INFO - TEST-END | apps/search/test/marionette/app_installed_test.js | Search - Installed Apps Test app result after install/uninstall took 28794 ms 21:44:48 INFO - TEST-UNEXPECTED-FAIL | apps/search/test/marionette/app_installed_test.js | Search - Installed Apps Test "after each" hook
Assignee: nobody → apastor
Attachment #8576524 - Attachment is obsolete: true
Ok, so this test is only failing in v2.1, and the reason is that we are showing the search results in a different way there, so [1] doesn't make sense in 2.1. If that passes sometimes, is just a timing coincidence. As we can see in [2]'s screenshot, we have more than 1 results. In 2.2, the internet results are shown bellow the provider selector. Kevin, what should we do? Should we adapt the test for searching specifically 'Calendar' and 'Calendar2' in the results, or should we disable the test? [1] https://github.com/mozilla-b2g/gaia/blob/v2.1/apps/search/test/marionette/app_installed_test.js#L40 [2] http://ftp.mozilla.org/pub/mozilla.org/b2g/try-builds/gaiabld-2ab40780caf4/gaia-try-linux64_gecko/gaia-try_ubuntu64_vm-b2gdt_test-gaia-js-integration-3-bm122-tests1-linux64-build151.txt.gz
Flags: needinfo?(kgrandon)
Is it just a matter of changing the expected result count? If so, I'm fine with making that change only for v2.1. I'm also fine disabling the test if it's more complex than that.
Flags: needinfo?(kgrandon)
Is not about the count (we don't know exactly how many results are there because they are coming from e.me). We should change the test for searching the name of the app in the results, so I think is a little bit overkilling for the value (I don't think that will change in 2.1). I'm going to disable that test in 2.1. Thanks!
Sure, though we should have a mock server running the tests and not actually be hitting the real server - so the results should always be the same.
Attached image screenshot.png —
Not sure where those results come from, but is definitely not 1.
Comment on attachment 8576795 [details] [review] [gaia] albertopq:1135071-fix-tests-v2.1 > mozilla-b2g:v2.1 I didn't like disabling the test, so I ended up fixing it :)
Attachment #8576795 - Flags: review?(kgrandon)
Comment on attachment 8576795 [details] [review] [gaia] albertopq:1135071-fix-tests-v2.1 > mozilla-b2g:v2.1 That's great, thanks!
Attachment #8576795 - Flags: review?(kgrandon) → review+
v2.1: https://github.com/mozilla-b2g/gaia/commit/90f4ca618048b608c8e663d3e1d84781cfa719a8
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
(In reply to Kevin Grandon :kgrandon from comment #6) > Sure, though we should have a mock server running the tests and not actually > be hitting the real server - so the results should always be the same. Is the test pointing to an external server? I thought that wasn't allowed for tbpl tests?
(In reply to Martijn Wargers [:mwargers] (QA) from comment #12) > Is the test pointing to an external server? I thought that wasn't allowed > for tbpl tests? Nope, we're definitely running a mock everything.me server: https://github.com/mozilla-b2g/gaia/tree/master/shared/test/integration/eme_server
Whiteboard: [systemsfe]
Component: Marionette → Gaia
Product: Testing → Firefox OS
We saw this again the other day. I think it's because we are still relying on the old homescreen for some of these tests. I'll take a look. https://treeherder.mozilla.org/logviewer.html#?job_id=2883919&repo=gaia
Assignee: apastor → mhenretty
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Yup, I think that was it, seeing a lot of green Gij12 [1]. I'm merging, r=m a=test-only 1.) https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=dd685bc285ae9f5d059f29cc13eff15d66c59f46
master: https://github.com/mozilla-b2g/gaia/commit/f0538cc1b1dea3dbe61e7c3b8b0550e12aad8c11
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Blocks: 1236308
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: