Closed Bug 982260 Opened 7 years ago Closed 7 years ago

Gaia-integration tests on TBPL fail with: TEST-UNEXPECTED-FAIL | Contacts > Details Merging 15 contacts

Categories

(Firefox OS Graveyard :: Gaia::Contacts, defect)

defect
Not set
normal

Tracking

(b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v2.0 fixed)

RESOLVED FIXED
Tracking Status
b2g-v1.3 --- fixed
b2g-v1.3T --- fixed
b2g-v2.0 --- fixed

People

(Reporter: jgriffin, Assigned: hola)

References

Details

(Keywords: intermittent-failure, Whiteboard: [leave-open], [NO_UPLIFT])

Attachments

(2 files, 5 obsolete files)

This seems to be fallout from https://github.com/mozilla-b2g/gaia/commit/0b7491cc15bb1a8b6de425d2cc23e7d5b474593d.

Once this is fixed, I think we can unhide gaia-integration on TBPL, it's quite green otherwise!
Forgot the actual error:

12:15:10     INFO -    1) Contacts > Details Merging 15 contacts:
12:15:10     INFO -    Error: timeout exceeded!
12:15:10     INFO -        at Object.Client.waitForSync (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:679:16)
12:15:10     INFO -        at Object.Client.waitFor (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:648:60)
12:15:10     INFO -        at Object.MarionetteHelper.waitForElement (/builds/slave/test/gaia/node_modules/marionette-helper/index.js:131:12)
12:15:10     INFO -        at Object.Contacts.waitForFormShown (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:139:35)
12:15:10     INFO -        at Object.Contacts.enterContactDetails (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:168:10)
12:15:10     INFO -        at Object.Contacts.addContact (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:189:10)
12:15:10     INFO -        at Context.<anonymous> (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/details_test.js:42:15)
12:15:10     INFO -        at Test.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:211:32)
12:15:10     INFO -        at Runner.runTest (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:372:10)
12:15:10     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:448:12
12:15:10     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:297:14)
12:15:10     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:307:7
12:15:10     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:245:23)
12:15:10     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:269:7
12:15:10     INFO -        at Hook.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:213:5)
12:15:10     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:257:10)
12:15:10     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:269:7
12:15:10     INFO -        at done (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:185:5)
12:15:10     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:197:9
12:15:10     INFO -        at Object.executeHook (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)
12:15:10     INFO -        at process._tickCallback (node.js:415:13)

https://tbpl.mozilla.org/php/getParsedLog.php?id=35951867&tree=B2g-Inbound&full=1#error0
Blocks: 982262
Attached file travis build log
I had the exact same failure on Travis an hour ago (for a patch that is totally unrelated), looks like an intermittent bug.
I agree; unfortunately it's so frequent on TBPL that it will block unhiding the tests there until it's resolved.
German, this seems to come from your patch in bug 972600, can you have a look?

Also, if this gives a frequent intermittent, I would actually backout bug 972600 until it's resolved. What do you think German?
Flags: needinfo?(gtorodelvalle)
Blocks: 960072
No longer blocks: 982262
Blocks: 972600
Assignee: nobody → gtorodelvalle
Can we fix this asap please?
Let's disable this test in TBPL if it can't be fixed ASAP, so we can unhide the Gi tests there.
Attachment #8390855 - Flags: review?(felash)
Attachment #8390855 - Flags: review?(anygregor)
Attachment #8390855 - Flags: review?(anygregor) → review+
Attachment #8390855 - Flags: review?(felash)
This test was also intermittent on travis, so I've disabled it there as well:

1) Contacts > Details Merging 15 contacts:

Error: timeout exceeded!

at Object.Client.waitForSync (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:679:16)

at Object.Client.waitFor (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:648:60)

at Object.MarionetteHelper.waitForElement (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-helper/index.js:131:12)

at Object.Contacts.waitForFormShown (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:130:35)

at Object.Contacts.enterContactDetails (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:159:10)

at Object.Contacts.addContact (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:180:10)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/details_test.js:43:15)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:211:32)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:372:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:448:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:297:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:307:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:245:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:213:5)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:257:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:197:9

at Object.executeHook (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)

at process._tickCallback (node.js:415:13)

https://github.com/mozilla-b2g/gaia/commit/38716fd79a507db6bcc655958383ca21f6c09025
https://hg.mozilla.org/mozilla-central/rev/39fc3c747bba
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → 1.4 S4 (28mar)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: [leave-open]
Hi guys! Sorry but I was out of office for the last 2 days...

I think I found the issue. It seems there is some kind of bug with the size() function of a Marionette element. I will file a bug about it.

Meanwhile, I included a function to get the window size "from the server" as a way to detect when the sliding animations end. It seems to be working fine. I created a pull request to my own forked Gaia repository (https://github.com/gtorodelvalle/gaia/pull/8) and the details_test.js integration tests passed 50 times out of 50 as you can see at https://travis-ci.org/gtorodelvalle/gaia/builds/21093069 The not passing runs have nothing to do with this integration test :-)

I prepared a pull request to master including the new patch. I'll add it as an attachment in a sec.
Flags: needinfo?(gtorodelvalle)
Attached file 17372.html (obsolete) —
BTW, should I also undo the inclusion of details_test.js in the blacklist (see shared/test/integration/tbpl-manifest.json) and include it in this PR? Thanks!
Attachment #8394057 - Flags: review?(kgrandon)
I was referring to this PR regarding the black list: attachment 8390855 [details] [review]
(In reply to gtorodelvalle from comment #12)
> Created attachment 8394057 [details]
> 17372.html
> 
> BTW, should I also undo the inclusion of details_test.js in the blacklist
> (see shared/test/integration/tbpl-manifest.json) and include it in this PR?
> Thanks!

Yes, please!
Thanks Jonathan! I have included in the PR (attachment 8394057 [details]) the deletion of details_test.js from the TBPL black list (shared/test/integration/tbpl-manifest.json).
Comment on attachment 8394057 [details]
17372.html

Ugh, I really wish size() worked here - any idea why it doesn't? I wonder if it's due to some keyboard or state thing? I know we use it in other areas and it seems to just work fine.

Please do file a bug about the .size() and it's usage if there is one, and block this bug against it so we can clean this up after the fact.

Since it's green, let's merge and hope that's the end of it :)
Attachment #8394057 - Flags: review?(kgrandon) → review+
Blocks: 986325
Attached file Pull request to v1.3 (obsolete) —
Attached file Pull request to v1.3t (obsolete) —
Uplifting to 1.3, a=TEST_ONLY.

v1.3 - https://github.com/mozilla-b2g/gaia/commit/60b1633edd33a42b05a76a13384bb5a537435a1b
v1.3t - https://github.com/mozilla-b2g/gaia/commit/266c3eab5124431570cf03c4ad0d7e99e211958d
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Too many failures - I think we need to back this out and investigate what's gone wrong here. I will also be disabling this on 1.3/1.3t due to the intermittent failures.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I am really confused - this test should've been backed out and disabled by now, why is it still running/getting reported? Wes - any pointers here? Did some commit not get mirrored properly?
Flags: needinfo?(kwierso)
(In reply to Kevin Grandon :kgrandon from comment #85)
> I am really confused - this test should've been backed out and disabled by
> now, why is it still running/getting reported? Wes - any pointers here? Did
> some commit not get mirrored properly?

The disabling landed in b2g-inbound in https://tbpl.mozilla.org/?tree=B2g-Inbound&onlyunstarred=1&rev=0a1a99529228 and so far has not been merged into m-c or the other integration branches.

The failures did go away on b-i once it landed there, so once b-i gets merged around again, they should go away for good.
Flags: needinfo?(kwierso)
BTW, do you have any info or log about the cause of the failures? Or maybe guidelines so I could get them... Thanks!
German, here is the log of the error:

08:11:18     INFO -    1 failing
08:11:18     INFO -    1) Contacts > Details Merging 15 contacts:
08:11:18     INFO -    InvalidElementState: (12) Element is not Enabled
08:11:18     INFO -    Remote Stack:
08:11:18     INFO -    <none>
08:11:18     INFO -        at Error.MarionetteError (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/error.js:67:13)
08:11:18     INFO -        at Object.Client._handleCallback (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:474:19)
08:11:18     INFO -        at /builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:508:21
08:11:18     INFO -        at TcpSync.send (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/drivers/tcp-sync.js:94:10)
08:11:18     INFO -        at Object.send (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:455:36)
08:11:18     INFO -        at Object.Client._sendCommand (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:501:19)
08:11:18     INFO -        at Object.Element._sendCommand (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/element.js:49:21)
08:11:18     INFO -        at Object.click (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/element.js:172:19)
08:11:18     INFO -        at Object.Contacts.enterContactDetails (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:168:49)
08:11:18     INFO -        at Object.Contacts.addContact (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:179:10)
08:11:18     INFO -        at Context.<anonymous> (/builds/slave/test/gaia/apps/communications/contacts/test/marionette/details_test.js:44:15)
08:11:18     INFO -        at Test.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:211:32)
08:11:18     INFO -        at Runner.runTest (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:372:10)
08:11:18     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:448:12
08:11:18     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:297:14)
08:11:18     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:307:7
08:11:18     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:245:23)
08:11:18     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:269:7
08:11:18     INFO -        at Hook.Runnable.run (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:213:5)
08:11:18     INFO -        at next (/builds/slave/test/gaia/node_modules/mocha/lib/runner.js:257:10)
08:11:18     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runner.js:269:7
08:11:18     INFO -        at done (/builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:185:5)
08:11:18     INFO -        at /builds/slave/test/gaia/node_modules/mocha/lib/runnable.js:197:9
08:11:18     INFO -        at Object.executeHook (/builds/slave/test/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)
08:11:18     INFO -        at process._tickCallback (node.js:415:13)
08:11:18     INFO -  make: *** [test-integration-test] Error 1

Not sure this is really useful ;)

You can have it by clicking on the "parsed log" link in any previous comment.
Attached file 17568.html (obsolete) —
Hi guys! First off, it helped Julien. Thank you very much! ;-)

I prepared a new PR to master https://github.com/mozilla-b2g/gaia/pull/17568 All I can say is that running 50 times the integration tests in a row, the affected integration test passes every time: https://travis-ci.org/gtorodelvalle/gaia/builds/21419988

Kevin, is there any kind of possibility of merging this PR in master and keep it there some days in "quarantine" before uplifting it to v1.3. In fact, I would prepare other PRs to v1.3 and v1.4 making Travis run the integration tests 50 times to check that in those branches the result is the same as in master. What do you think?
Attachment #8390855 - Attachment is obsolete: true
Attachment #8394057 - Attachment is obsolete: true
Attachment #8394911 - Attachment is obsolete: true
Attachment #8394916 - Attachment is obsolete: true
Attachment #8396290 - Flags: review?(kgrandon)
This is the PR to v1.3: https://github.com/mozilla-b2g/gaia/pull/17585 after successfully passing 50 times the affected integration test as it can be seen at https://travis-ci.org/gtorodelvalle/gaia/builds/21501924
This is the PR I sent to my own forked Gaia repository: https://github.com/gtorodelvalle/gaia/pull/11
Germán - good news. I think we can potentially land and just leave this bug open to avoid any uplifting. Can you tell me what specific changes were done in this PR? Thanks!
Flags: needinfo?(gtorodelvalle)
You can also use [NO_UPLIFT] in the whiteboard.
(In reply to Kevin Grandon :kgrandon from comment #109)
> Germán - good news. I think we can potentially land and just leave this bug
> open to avoid any uplifting. Can you tell me what specific changes were done
> in this PR? Thanks!

Kind of minimal, this line was the only update to the previous patch we had: https://github.com/mozilla-b2g/gaia/pull/17568/files#diff-93f450be0058b99f97a02c6657b29880L165

The point is that with that previous patch, we also had the 50 integration test runs passing or we may have not merged it :-(

I agree regarding merging the patch in master but not setting the bug as RESOLVED, although I also set the [NO_UPLIFT] in the whiteboard as Julien suggests ;-)
Flags: needinfo?(gtorodelvalle)
Whiteboard: [leave-open] → [leave-open], [NO_UPLIFT]
I want to see another large run before re-enabling this test. I can do so this weekend when travis traffic is lighter.
I am kind of lost here :-O Which code version, branch, or whatever is causing the failure? :s The last proposed PR (attachment 8396290 [details]) is not merged in master yet (or in any other branch) as far as my knowledge :-) Thanks!
It looks like that may have failed on try server? I don't think anything has landed yet, but I will look over the weekend when travis is slower and we can do a large run. Thanks for being patient.
Comment on attachment 8396290 [details]
17568.html

I have unfortunately seen this fail after running it several times on travis. I think it actually may be something wrong with our lib/contacts.js file that's causing all contacts tests to fail.

I'm wondering if we are actually missing some event that is not causing the save button to be enabled. Perhaps we need to inject either a wait or some script execution before: this.client.findElement(selectors.formSave).click();


1) Contacts > Details Merging 15 contacts:

InvalidElementState: (12) Element is not Enabled

Remote Stack:

<none>

at Error.MarionetteError (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/error.js:67:13)

at Object.Client._handleCallback (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:474:19)

at /home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:508:21

at TcpSync.send (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/drivers/tcp-sync.js:94:10)

at Object.send (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:455:36)

at Object.Client._sendCommand (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:501:19)

at Object.Element._sendCommand (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/element.js:49:21)

at Object.click (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/element.js:172:19)

at Object.Contacts.enterContactDetails (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:170:59)

at Object.Contacts.addContact (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/lib/contacts.js:181:10)

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/communications/contacts/test/marionette/details_test.js:45:15)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:211:32)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:372:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:448:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:297:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:307:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:245:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:213:5)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:257:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:269:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:197:9

at Object.executeHook (/home/travis/build/mozilla-b2g/gaia/node_modules/marionette-client/lib/marionette/client.js:370:18)

at process._tickCallback (node.js:415:13)



https://travis-ci.org/mozilla-b2g/gaia/jobs/21807803
Attachment #8396290 - Flags: review?(kgrandon)
Thanks Kevin, sounds good to me ;-) In fact, taking into consideration the implementation of the waitForElement() function (see https://github.com/mozilla-b2g/marionette-helper/blob/master/index.js#L130), I will include a waitFor clause on the enabled status of the save button (see http://lightsofapollo.github.io/marionette_js_client/api-docs/files/lib_marionette_element.js.html#l247).

Definitely it smells to having a race condition somewhere :S I'll check this part of the code too.

I'll run the 50 integration test runs once again and I'll come back to you guys with the results.

Thanks to Kevin for your comments once again! ;)
Geez! This is being hard :-) I spent quite some time investigating the issue and the thing is that, despite what I mentioned in comment 117, I am not that sure it is a race condition what we are dealing with here :-( In fact, I verified that the action (no matter if click, tap, simple action or mouse event dispatching) was properly sent to the server (the callback was called back :-)) but for some reason it was not properly "managed". In fact, in some executions of the integration tests, I got a black screen in my B2G desktop which "seems" to be related to the issue.
I saw some integration tests including time delays to solve issues which could be related to this one but I did not want to go that path... yet :p So I managed to get the integration test passing executing it 50 times as you can see at https://travis-ci.org/gtorodelvalle/gaia/builds/22517673 You will also see that there are executions failing intermittently but they are not related to the details_test.js

I will run the integration tests 50 times in a row some more times and at distinct times (including the weekend) to check how it evolves. I'll keep you posted.
Target Milestone: 1.4 S4 (28mar) → ---
Blocks: 1035939
No longer blocks: 960072
Assignee: gtorodelvalle → hola
Attached file Pull request
Working for me in local with this patch.
Attachment #8396290 - Attachment is obsolete: true
Attachment #8526120 - Flags: review?(jmcf)
Comment on attachment 8526120 [details] [review]
Pull request

cancelling review as we are going to need to use the new contacts_data.js module
Attachment #8526120 - Flags: review?(jmcf)
Comment on attachment 8526120 [details] [review]
Pull request

New test suite checking lots of things about mozContacts and facebook contacts detail using contact_data module to make tests more independent.
Attachment #8526120 - Flags: review?(jmcf)
Comment on attachment 8526120 [details] [review]
Pull request

We need another round according to the GH comments
Attachment #8526120 - Flags: review?(jmcf)
I improved the tests according to your comments but unfortunately gaia-try is complaining about one test. It's working in local and I haven't even get intermittent failures, but it seems to fail in gaia-try. Maybe I have to think of another way to do that test so it works on gaia-try before we can merge.
Attachment #8526120 - Flags: review?(jmcf)
Comment on attachment 8526120 [details] [review]
Pull request

please check the comments on GH

thanks
Attachment #8526120 - Flags: review?(jmcf) → review+
https://github.com/mozilla-b2g/gaia/commit/456c8bc17e881bcdca98671a2e4a59e38709dd27
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.