1071943 - Intermittent failing test, TEST-UNEXPECTED-FAIL | /builds/slave/test/gaia/apps/communications/contacts/test/marionette/ice_test.js | Contacts > ICE contacts ICE contacts Select ICE contact from search, go back to ICE settings

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Contacts, defect)

Description

[Kevin Grandon :kgrandon]

Attachment: Screenshot at point of failure

1) Contacts > ICE contacts ICE contacts Select ICE contact from search, go back to ICE settings:
10:51:34     INFO -    Error: timeout exceeded!
10:51:34     INFO -        at Object.Client.waitForSync (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:682:16)
10:51:34     INFO -        at Object.Client.waitFor (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:650:60)
10:51:34     INFO -        at Object.MarionetteHelper.waitForElement (/builds/slave/test/gaia/node_modules/marionette-helper/index.js:142:12)
10:51:34     INFO -        at Context.<anonymous> (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/ice_test.js:90:35)
10:51:34     INFO -        at callFn (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:223:21)
10:51:34     INFO -        at Test.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:216:7)
10:51:34     INFO -        at Runner.runTest (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:373:10)
10:51:34     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:451:12
10:51:34     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:298:14)
10:51:34     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:308:7
10:51:34     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:246:23)
10:51:34     INFO -        at Object._onImmediate (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:275:5)
10:51:34     INFO -        at processImmediate [as _immediateCallback] (timers.js:330:15)
10:51:34     INFO -  make: *** [test-integration-test] Error 1

https://tbpl.mozilla.org/php/getParsedLog.php?id=48694983&tree=B2g-Inbound

Comment 1

[Kevin Grandon :kgrandon]

This test is failing way too much on TBPL and has been disabled: https://github.com/mozilla-b2g/gaia/commit/b3bf31d8158ba9ba0a5dbb4f0c7afb1643b0de5f

Comment 2

[Kevin Grandon :kgrandon]

Adding ni? on Francisco and Jorge for investigation. I'll also try to look into this one as well.

Comment 3

[Kevin Grandon :kgrandon]

Seems another call stack, and maybe a more recent one, is this: 

3) Contacts > ICE contacts ICE contacts Select ICE contact from search, go back to ICE settings:
13:45:53     INFO -    Error: timeout exceeded!
13:45:53     INFO -        at Object.Client.waitForSync (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:682:16)
13:45:53     INFO -        at Object.Client.waitFor (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:650:60)
13:45:53     INFO -        at Object.MarionetteHelper.waitForElement (/builds/slave/test/gaia/node_modules/marionette-helper/index.js:142:12)
13:45:53     INFO -        at Context.<anonymous> (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/ice_test.js:108:35)
13:45:53     INFO -        at callFn (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:223:21)
13:45:53     INFO -        at Test.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:216:7)
13:45:53     INFO -        at Runner.runTest (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:373:10)
13:45:53     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:451:12
13:45:53     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:298:14)
13:45:53     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:308:7
13:45:53     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:246:23)
13:45:53     INFO -        at Object._onImmediate (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:275:5)
13:45:53     INFO -        at processImmediate [as _immediateCallback] (timers.js:330:15)

Comment 4

[Francisco Jordano [:arcturus] [:francisco]]

Thanks Kevin,

I've been talking with Jorge and he told me to take a look, also ni Adrian, since he created that test to see if this is familiar to him.

Thanks folks

Comment 5

[Adrián de la Rosa]

I launched the test repeatedly and I haven't been able to reproduce locally. Looking at the trace, I may have some ideas about why it could fail, but I won't be able to be sure I fixed anything because for me it's already working.

We can investigate the problem more deeply in the future or if you want, I can take it and propose some improvements.

Sorry for causing this mess.

Comment 6

[Kevin Grandon :kgrandon]

(In reply to Adrián de la Rosa from comment #5)
> Sorry for causing this mess.

It's not really a mess, writing non-intermittent marionette tests is actually surprisingly difficult! 

I still don't have any proof yet, but one theory I have is that this is due to cards doing transitions when we are trying to click on stuff. If you look at the test in action it seems we don't wait for any cards to finish transitioning before trying some actions. This has bitten us in the past with other apps.

Comment 7

[Francisco Jordano [:arcturus] [:francisco]]

(In reply to Adrián de la Rosa from comment #5)
> I launched the test repeatedly and I haven't been able to reproduce locally.
> Looking at the trace, I may have some ideas about why it could fail, but I
> won't be able to be sure I fixed anything because for me it's already
> working.
> 
> We can investigate the problem more deeply in the future or if you want, I
> can take it and propose some improvements.
> 
> Sorry for causing this mess.

Not a mess! We are suffering a lot from that, the image can give you an idea where in the execution we got stop, probably trying to find an element that wasn't being displayed yet!

Comment 8

[Adrián de la Rosa]

Well, I'll take it, then. I would love to see integration tests in contacts working before christmas :P

Thank you both for your advice! I am sure it will be really useful.

Comment 9

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Sorry to jump in accidentally,

I just noticed there is same failure on gaia/tbpl 2.1, so should we skip this test on 2.1 too ?

Comment 10

[Kevin Grandon :kgrandon]

(In reply to EJ Chen [:eragonj][:小龍哥] from comment #9)
> Sorry to jump in accidentally,
> 
> I just noticed there is same failure on gaia/tbpl 2.1, so should we skip
> this test on 2.1 too ?

Yes, feel free to uplift with a=nbotb if you think this should be disabled on 2.1 as well.

Comment 11

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Thanks Kevin, just uplifted the patch to v2.1 

https://github.com/mozilla-b2g/gaia/commit/1cb6775fe67d7f71098d9c8b2fefb08bfe44ec87

Comment 12

[Adrián de la Rosa]

Attachment: Pull request #25365

I made a pull request with a copy of ice_test.js with a different name so gaia-try executed it (the original file is on the blacklist). I ran it twice and both times the tests were busted (no failures) and the logs blamed other tests.

Right now in the pull request is only the original file that I improved and refactored. I fixed some errors I thought could be the cause of the intermittent failures, but we can't be sure since we don't know what caused the errors on the first place. I also refactored the whole file to avoid lots of repeated code and to make it more reusable and extensible. I am also removing this test from the blacklist.

Comment 13

[Jose Manuel Cantera]

Comment on attachment 8510420 [details] [review]
Pull request #25365

please see the comments on GH

Comment 14

[Adrián de la Rosa]

Comment on attachment 8510420 [details] [review]
Pull request #25365

Added method to check if a view is visible after the new fade in animation. Every check for slide up has been adapted to this new method.

Comment 15

[Jose Manuel Cantera]

Comment on attachment 8510420 [details] [review]
Pull request #25365

We are on the right direction but we need another round.

Thanks Adrian!

Comment 16

[Jose Manuel Cantera]

(In reply to Jose Manuel Cantera from comment #15)
> Comment on attachment 8510420 [details] [review]
> Pull request #25365
> 
> We are on the right direction but we need another round.
> 
> Thanks Adrian!

https://github.com/mozilla-b2g/gaia/pull/25365#issuecomment-61079498

Comment 17

[Adrián de la Rosa]

Comment on attachment 8510420 [details] [review]
Pull request #25365

Another refactor to avoid some duplicate code. If you are ok with it, I will ask for another review to check if it's ok to get these tests out of the blacklist.

Comment 18

[Jose Manuel Cantera]

Comment on attachment 8510420 [details] [review]
Pull request #25365

thanks Adrian. 

Kevin, please could you have a look as well. We intend to re-enable some disabled tests. 

thanks

Comment 19

[Kevin Grandon :kgrandon]

Comment on attachment 8510420 [details] [review]
Pull request #25365

Thank you for helping with the test! Looks good to me.

Comment 20

[Jose Manuel Cantera]

https://github.com/mozilla-b2g/gaia/commit/caebca410102c8fc8b0cce08af584e1ef391eac9

Comment 21

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.