1196830 - test_import_edit_export_contact.py: "AssertionError: 9 != 8"

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Contacts, defect, P2)

Description

[Oliver Nelson [:oliverthor]]

Description:
test_import_edit_export_contact is failing consistently after import contacts. The test is finding a different number of contacts near the end of it's test. After the test imports all it's contacts from SIM, it edits a single contact and then exports all the contacts back. Observing a local run observed '15' imported, an editted contact then '15' exported, but returning to Contacts view saw the test fail with an asserition error of '16' != '15'. 
Looking through the successive adhocs, the SIM is inflating with additional SIMs. If the user (on a fresh device, no contacts) imports '6' contacts, edits one of them, then exports them back to the SIM: the SIM now contains '7' contacts.

* http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2.bitbar/234/HTML_Report/
* http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc.bitbar/177/HTML_Report/

Repro Steps:
1) Update device to 20150820030207
2) Open the Contacts app
3) Tap the 'gear' for Settings
4) Import Contacts from SIM (observe '6' imported as an example)
5) Edit a contact, change the phone number by a single digit
6) Export all contacts back to SIM (observe '6' being pushed back to SIM)
** step not taken by automation, but proves the failure **
7) Import contacts again (observe '7' imported from SIM)

Actual:
If a user edits a contact that was imported and then exports the contacts back to SIM, they will have both the 'edited' and 'unedited' contact on SIM, giving them an additional contact.

Expected: 
If a user edits a contact from their SIM after an import then exports contacts back to SIM, they will have only one copy of each contact that was imported originally (edited contact runs over previous entry)


Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.18-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/contacts/test_import_edit_export_contact.py", line 60, in test_import_edit_export_contact
self.assertEqual(len(sim_contacts), sim_contacts_number_before_import)
AssertionError: 9 != 8


Environmental Variables:
Device firmware (base) 	L1TC000118D0
Device firmware (date) 	20 Aug 2015 10:30:36
Device firmware (incremental) 	eng.cltbld.20150820.063025
Device firmware (release) 	4.4.2
Device identifier 	flame
Device memory 	219772 kB
Device serial 	94f7ce17
Device uptime 	0 days 0 hours 2 minutes 12 seconds
Gaia date 	19 Aug 2015 18:28:14
Gaia revision 	89e0096a3de0
Gecko build 	20150820030207
Gecko revision 	29b2df16e961fbe9a379362ecba6f888d1754bc3
Gecko version 	43.0a1

Reproducible manually: Yes

Repro frequency: 5/5

Comment 1

[Oliver Nelson [:oliverthor]]

This issue appears to extend into a design question, so triaging to Peter to determine the proper component owner.

Comment 2

[Martijn Wargers (dead)]

I remember vaguely this test to fail for me locally, because of a similar issue like this, I think.

Comment 3

[Martijn Wargers (dead)]

Oliver, as I understand it, this is a change in behavior, right? Then we should find out what caused this change.

Regardless, this test was always failing for me locally, because it makes some bad assumption. I filed bug 1197144 for improving this test.

Comment 4

[Martijn Wargers (dead)]

I think there is a real bug here, because going further with the actual result that we have here:
"
Actual:
If a user edits a contact that was imported and then exports the contacts back to SIM, they will have both the 'edited' and 'unedited' contact on SIM, giving them an additional contact.
"

If I after that, delete all my contacts on the phone (not on the SIM card), then import all contacts from my sim. I get only the edited contact from my SIM card, not the unedited contact.

Comment 5

[Martijn Wargers (dead)]

It's passing with:
Build ID               20150819150203
Gaia Revision          8f77edf3ac39d36f6df0f5517223d3ed35ed89e0
Gaia Date              2015-08-19 11:28:39
Gecko Revision         https://hg.mozilla.org/mozilla-central/rev/d590b9601ba8138b621b2553daefd8823ab4295a
Gecko Version          43.0a1
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150727.063909
Firmware Date          Mon Jul 27 06:39:20 EDT 2015
Bootloader             L1TC000118D0

It is failing with:
Build ID               20150820030207
Gaia Revision          89e0096a3de0378e3eda77e6a2a0bb5ca03eb8bb
Gaia Date              2015-08-19 18:28:14
Gecko Revision         https://hg.mozilla.org/mozilla-central/rev/29b2df16e961fbe9a379362ecba6f888d1754bc3
Gecko Version          43.0a1
Device Name            flame
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.cltbld.20150727.063909
Firmware Date          Mon Jul 27 06:39:20 EDT 2015
Bootloader             L1TC000118D0

Comment 6

[Martijn Wargers (dead)]

Regression range for mozilla-central:
https://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2015-08-19+12%3A02%3A05&enddate=2015-08-20+00%3A05%3A05

With Gaia, I have no idea how to get a regression range link.
However, I see bug 1183727 somewhere in that date range, could it be a regression from that bug, somehow, Borja?

Comment 7

[Borja Salguero [:borjasalguero]]

Hi! I don't know if the STR is the right behaviour or not (after testing the same with SD Card I'm having the same effect)...
The patch you mentioned is not handling any merging, however we should ask Francisco for the expected results after follow your STR. Francisco, could you take a look? Thanks!

Comment 8

[Peter Bylenga [:PBylenga]]

Johan can you help out here?

Comment 9

[Martijn Wargers (dead)]

I was running this test now with some print statements in there (and the bug 1197144 fix included). 

print(self.data_layer.sim_contacts) before the export to sim:
[{u'honorificPrefix': None, u'tel': [{u'value': u'55575197265'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183741', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Name7218'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'55597549146'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183742', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Name3323 New'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'[phone]'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183741', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Klanten service'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'[phone]'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183742', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'VoiceMail'], u'url': None, u'published': None, u'givenName': None}]

print(self.data_layer.sim_contacts) after the export to sim:
[{u'honorificPrefix': None, u'tel': [{u'value': u'55575197265'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183741', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Name7218'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'55597549146'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183742', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Name3323 New'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'55575197265'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183743', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Name7218 New'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'[phone]'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183741', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'Klanten service'], u'url': None, u'published': None, u'givenName': None}, {u'honorificPrefix': None, u'tel': [{u'value': u'[phone]'}], u'photo': None, u'sex': None, u'additionalName': None, u'toJSON': None, u'setMetadata': None, u'honorificSuffix': None, u'category': None, u'phoneticFamilyName': None, u'adr': None, u'anniversary': None, u'id': u'89310504909010183742', u'note': None, u'init': None, u'email': None, u'updated': None, u'bday': None, u'jobTitle': None, u'familyName': None, u'genderIdentity': None, u'phoneticGivenName': None, u'key': None, u'org': None, u'nickname': None, u'impp': None, u'name': [u'VoiceMail'], u'url': None, u'published': None, u'givenName': None}]
TEST-UNEXPECTED-FAIL | test_import_edit_export_contact.py TestImportEditExportContact.test_import_edit_export_contact | AssertionError: 5 != 4

Comment 10

[Johan Lorenzo [:jlorenzo]]

Based on comment 5, I put the last working gaia on top of first broken gecko => The bug is no more present, then the bug happened in Gaia.

I finished the regression window thanks to git bised. Like Martijn inferred, the regression happened in bug 1183727.

Borja, I confirm the STR reflect the right behavior. Could you look into the changes introduced in bug 1183727?

[Blocking Requested - why for this release]: Functional regression, that can lead a user to fill up his entire SIM card without knowing why.

Comment 11

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #10)
> I finished the regression window thanks to git bised. 
Typo: git bisect

Comment 12

[Johan Lorenzo [:jlorenzo]]

Comms triage: Functional regression caught by automation.

Comment 13

[GitHub Autolander Bot]

Attachment: [gaia] mwargers:1196830_disable > mozilla-b2g:master

Comment 14

[Martijn Wargers (dead)]

Merged: https://github.com/mozilla-b2g/gaia/commit/9c298f98efc916556935b84d2d7e852a5c8c76c9
test_import_edit_export_contact.py is disabled now until this is fixed.

Comment 15

[Mahendra Potharaju [:mahe]]

Marking as a P2

Comment 16

[Borja Salguero [:borjasalguero]]

This depends on the final decision on NGA, so I'll forward this to Francisco.

Comment 17

[Francisco Jordano [:arcturus] [:francisco]]

We will have this week a final decision on keeping the different documents for 2.5 or not.

Anyway, having different documents will happen either on 2.5 or next version so we will need to figure out what error we were introducing here.

But as Borja commented, if we backout bug 1183727 this won't be a blocker.

Comment 18

[Maria Angeles Oteo (:oteo)]

After reverting the commits that gave us separated views in Contacts, the issue reported in this bug is not happening anymore in master (as it raised when bug 1183727 landed and it has already been backed out)

For that reason, removing the 2.5+ flag and adding [only for NGA Contacts branch] whiteboard to ensure that this bug need to be fixed in NGA branch (but not in current master branch).

Comment 20

[BMO Automation]

Firefox OS is not being worked on