894751 - [Buri][Contacts][Performance]Import SIM contacts many times, then access Contacts, find a series of problems

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Contacts, defect, P2)

Description

[sync-1]

SW171
 AU_LINUX_GECKO_ICS_STRAWBERRY.01.01.00.019.152
 Firefox os  v1.1
 Mozilla build ID:20130702230206
 
 DEFECT DESCRIPTION:
 Import SIM contacts many times,then access Contacts,find a series of problems  
 
  REPRODUCING PROCEDURES:
 Precondition:Inserd Unicon ,and have 500 contacts in USIM card
 
 1.Launch Contacts application
 2.Settings->Import from SIM(repeatedly import contacts many times,about memory from 150MB to 81MB)
 3.Then import SIM contacts,handset occur reboot---->KO1
 4.Access Contacts,contacts list can't show conatcts---->KO2
 5.Access Contacts for a moment,then tap Create/Settings/Home/Power key,have no response,you can only pull out the battery---->KO3
 
 Remark:Software version by FOTA from 167 to 168
 
  EXPECTED BEHAVIOUR:
 Handset should not occur abnormally
 
  ASSOCIATE SPECIFICATION:
 
  TEST PLAN REFERENCE:
 
  TOOLS AND PLATFORMS USED:
 
  USER IMPACT:
 
  REPRODUCING RATE:
 1/5
 
  For FT PR, Please list reference mobile's behavior:

Comment 2

[Ben Kelly [:bkelly, not reviewing]]

Can you say roughly how many times the 500 contacts were imported?  Knowing the total number of contacts loaded would be helpful for reproducing and validating any fixes.

I haven't investigated this yet, but it sounds like maybe we're triggering the OOM killer with a huge contacts DB.

If that is the case, we may need to place a hard limit on the number of contacts that can be saved in the database.  This would likely still be necessary even if we improved memory efficiency in the app because we have to load the entire contacts list into memory in a number of places; both gecko and gaia.

Obviously it would be better not to require the full load into memory, but that would be a large architectural change with many challenges to be solved.

Comment 3

[buri.blff]

(In reply to Ben Kelly [:bkelly] from comment #2)
> Can you say roughly how many times the 500 contacts were imported?  Knowing
> the total number of contacts loaded would be helpful for reproducing and
> validating any fixes.
> 
> I haven't investigated this yet, but it sounds like maybe we're triggering
> the OOM killer with a huge contacts DB.
> 
> If that is the case, we may need to place a hard limit on the number of
> contacts that can be saved in the database.  This would likely still be
> necessary even if we improved memory efficiency in the app because we have
> to load the entire contacts list into memory in a number of places; both
> gecko and gaia.
> 
> Obviously it would be better not to require the full load into memory, but
> that would be a large architectural change with many challenges to be solved.

I don't know the import times when it will happen. There is no consistence. So i can't place a hard limit on the number of contacts.

Comment 4

[Jason Smith [:jsmith]]

QA Wanted for what purpose?

Comment 5

[Jason Smith [:jsmith]]

(In reply to Jason Smith [:jsmith] from comment #4)
> QA Wanted for what purpose?

Wayne - Can you clarify?

Comment 6

[Alex Keybl [:akeybl]]

Please provide steps to reproduce, include the contacts used for import. If this isn't provided, we will not be able to investigate.

Comment 7

[buri.blff]

the reproduce steps is as introduce in the pr description;
When you import contacts continuously, this pr will be reproduce;

Comment 8

[Jason Smith [:jsmith]]

(In reply to buri.blff from comment #7)
> the reproduce steps is as introduce in the pr description;
> When you import contacts continuously, this pr will be reproduce;

Ah, so what you are doing here is trying to import Contacts from SIM more than once. Can you give some more details here? Specific questions I'd like to know are:

1. Did a crash reporter dialog appear during the reproduction of this bug?

2. How many contacts were imported for each import from SIM? 

It sounds like the bug here is that we're pushing the edges of how many contacts are allowed to be stored on the device. When that edge is hit and you try to use the contacts app, the problems you've described above occur.

Something that would be worthwhile testing here is to take a fresh-flashed device and write a very large amount of contacts to the device that takes up a large amount of storage available on the device. Then, try using the contacts app.

Comment 9

[buri.blff]

1 there is no reporter dialog appear;
2 500 contacts for each time;

Comment 10

[buri.blff]

but there is no consistence number of imported contacts, when it will be reproduce;

we find it :
when import contacts with device free space from 181M to 92.1MB, it will be reproduce;
when import from 150M to 110M, it also will be reproduce;
so i think it's not memory kill problem;
Maybe there is some error on communication between gecko and gaia.

Comment 11

[Jason Smith [:jsmith]]

Can you provide a logcat when this bug reproduces?

Comment 12

[buri.blff]

Attachment: log.txt

the log.txt is the log for reproduce when freespace down to 92.1MB

Comment 13

[Jason Smith [:jsmith]]

That log reads off as this being issues related to being out of memory.

What's concerning here is the issues cited in comment 0 imply that running out of memory in the Contacts app is actually causing the whole phone to reboot, rather then the app process. Additionally, we seem to run into a lot of trouble when a background process is doing heavy memory consumption while in the background while trying to use a different app.

On the QA side - if someone can confirm this bug on our side, that would be helpful.

Someone (either Moz QA or Buri) also needs to follow the steps in https://wiki.mozilla.org/B2G/Debugging_OOMs#Debugging_an_OOM_crash to diagnose this.

Comment 14

[buri.blff]

Attachment: log.txt

this is the log when this pr is reproduce with freespace 92.1M

Comment 15

[buri.blff]

Attachment: log2.txt

log.txt

this is the log when this pr is reproduce with freespace 81M

Comment 16

[buri.blff]

Attachment: log2.txt

log.txt

this is the log when this pr is reproduce with freespace 81M

Comment 17

[buri.blff]

(In reply to Jason Smith [:jsmith] from comment #13)
> That log reads off as this being issues related to being out of memory.
> 
> What's concerning here is the issues cited in comment 0 imply that running
> out of memory in the Contacts app is actually causing the whole phone to
> reboot, rather then the app process. Additionally, we seem to run into a lot
> of trouble when a background process is doing heavy memory consumption while
> in the background while trying to use a different app.
> 
> On the QA side - if someone can confirm this bug on our side, that would be
> helpful.
> 
> Someone (either Moz QA or Buri) also needs to follow the steps in
> https://wiki.mozilla.org/B2G/Debugging_OOMs#Debugging_an_OOM_crash to
> diagnose this.

we find this is not OOM problem;

there is 109M freespace;

but when we click the simimport button the device reboot; the adb give log as follow;


E/memalloc( 1513): /dev/pmem: No more pmem available
E/msm7627a.gralloc( 1513): gralloc failed err=Out of memory
W/GraphicBufferAllocator( 1513): alloc(320, 460, 1, 00000133, ...) failed -12 (Out of memory)
I/Gecko   ( 1513): 
I/Gecko   ( 1513): ###!!! [Parent][AsyncChannel] Error: Channel error: cannot send/recv
I/Gecko   ( 1513): 
I/Gecko   ( 1513): _fireFatalError()
E/GeckoConsole( 1513): [JavaScript Error: "[Exception... "'Error: no message manager set' when calling method: [nsIObserver::observe]"  nsresult: "0x8057001c (NS_ERROR_XPC_JS_THREW_JS_OBJECT)"  location: "native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0"  data: no]"]

Comment 18

[buri.blff]

so the b2g process is killed, then the device reboot;

Comment 19

[buri.blff]

this is the b2g process reboot. not the device reboot; because the adb process is not kill;

Comment 20

[buri.blff]

the process memory is as follow:

User 57%, System 26%, IOW 15%, IRQ 0%
User 67 + Nice 0 + Sys 31 + Idle 0 + IOW 18 + IRQ 0 + SIRQ 0 = 116

  PID PR CPU% S  #THR     VSS     RSS PCY UID      Name
 1325  0  77% R    17  91268K  32488K  fg root     /system/b2g/b2g
 1318  0   6% R     1   1028K    320K  fg root     top

Comment 21

[buri.blff]

we find the memory is only 91M

Comment 22

[buri.blff]

32M

Comment 23

[buri.blff]

Dear jsmith:
is there any suggestion??

Comment 24

[buri.blff]

Dear jsmith:
 Maybe this pr  is similar to  bug 888284, which is resolved.

Comment 25

[Ben Kelly [:bkelly, not reviewing]]

I am working to reproduce this, but have gotten hung up on bug 896698.

I have imported many thousands of contacts so far without a crash.  The import does slow down as the database gets larger, though.

Can you give me a rough estimate of how many times you import your 500 contact file?  10 times?  100 times?  1000 times?  Right now I don't know if I should have expected the crash or not which is kind of an open ended process.

Having the exact file containing your 500 contacts would of course be ideal in case it is related to data content, etc.

(In reply to buri.blff from comment #10)
> we find it :
> when import contacts with device free space from 181M to 92.1MB, it will be
> reproduce;
> when import from 150M to 110M, it also will be reproduce;

Can you clarify if the "150M to 110M" is also free memory?

(In reply to buri.blff from comment #20)
> the process memory is as follow:
> 
> User 57%, System 26%, IOW 15%, IRQ 0%
> User 67 + Nice 0 + Sys 31 + Idle 0 + IOW 18 + IRQ 0 + SIRQ 0 = 116
> 
>   PID PR CPU% S  #THR     VSS     RSS PCY UID      Name
>  1325  0  77% R    17  91268K  32488K  fg root     /system/b2g/b2g
>  1318  0   6% R     1   1028K    320K  fg root     top

This looks like you are running top to get the memory?  Can you you paste the output of procrank or b2g-procrank when this problem occurs?

Comment 26

[Ben Kelly [:bkelly, not reviewing]]

(In reply to buri.blff from comment #24)
>  Maybe this pr  is similar to  bug 888284, which is resolved.

As far as I can tell bug 888284 was only in trunk and never hit the b2g18 code.  So I don't think this is related.

Comment 27

[Ben Kelly [:bkelly, not reviewing]]

Hmm, I am also importing contacts from an sdcard, not a SIM card.  I don't currently have a SIM card with any contacts on it.

Comment 28

[Ben Kelly [:bkelly, not reviewing]]

(In reply to buri.blff from comment #17)
> E/msm7627a.gralloc( 1513): gralloc failed err=Out of memory
> W/GraphicBufferAllocator( 1513): alloc(320, 460, 1, 00000133, ...) failed
> -12 (Out of memory)

I assume gralloc pulls from a graphics memory region separate from normal system memory?  I'm trying to understand under what conditions we would expect this failure.

Comment 29

[Ben Kelly [:bkelly, not reviewing]]

The fine folks on #gfx inform me that if we are somehow creating a large number of layers (for example one for each contact or something) then we could trigger the gralloc failure.

I will try testing with "layers.draw-borders" enabled later today to see how the layers are structured in contacts app.

Comment 30

[Justin Lebar (not reading bugmail)]

What's "Performation" (in the bug summary)?

Comment 31

[buri.blff]

(In reply to Ben Kelly [:bkelly] from comment #25)
> I am working to reproduce this, but have gotten hung up on bug 896698.
> 
> I have imported many thousands of contacts so far without a crash.  The
> import does slow down as the database gets larger, though.
> 
> Can you give me a rough estimate of how many times you import your 500
> contact file?  10 times?  100 times?  1000 times?  Right now I don't know if
> I should have expected the crash or not which is kind of an open ended
> process.
> 
> Having the exact file containing your 500 contacts would of course be ideal
> in case it is related to data content, etc.
> 
> (In reply to buri.blff from comment #10)
> > we find it :
> > when import contacts with device free space from 181M to 92.1MB, it will be
> > reproduce;
> > when import from 150M to 110M, it also will be reproduce;
> 
> Can you clarify if the "150M to 110M" is also free memory?
> 
> (In reply to buri.blff from comment #20)
> > the process memory is as follow:
> > 
> > User 57%, System 26%, IOW 15%, IRQ 0%
> > User 67 + Nice 0 + Sys 31 + Idle 0 + IOW 18 + IRQ 0 + SIRQ 0 = 116
> > 
> >   PID PR CPU% S  #THR     VSS     RSS PCY UID      Name
> >  1325  0  77% R    17  91268K  32488K  fg root     /system/b2g/b2g
> >  1318  0   6% R     1   1028K    320K  fg root     top
> 
> This looks like you are running top to get the memory?  Can you you paste
> the output of procrank or b2g-procrank when this problem occurs?




yes "150M to 110M" is free memory;

Comment 32

[buri.blff]

(In reply to Ben Kelly [:bkelly] from comment #25)
> I am working to reproduce this, but have gotten hung up on bug 896698.
> 
> I have imported many thousands of contacts so far without a crash.  The
> import does slow down as the database gets larger, though.
> 
> Can you give me a rough estimate of how many times you import your 500
> contact file?  10 times?  100 times?  1000 times?  Right now I don't know if
> I should have expected the crash or not which is kind of an open ended
> process.
> 
> Having the exact file containing your 500 contacts would of course be ideal
> in case it is related to data content, etc.
> 
> (In reply to buri.blff from comment #10)
> > we find it :
> > when import contacts with device free space from 181M to 92.1MB, it will be
> > reproduce;
> > when import from 150M to 110M, it also will be reproduce;
> 
> Can you clarify if the "150M to 110M" is also free memory?
> 
> (In reply to buri.blff from comment #20)
> > the process memory is as follow:
> > 
> > User 57%, System 26%, IOW 15%, IRQ 0%
> > User 67 + Nice 0 + Sys 31 + Idle 0 + IOW 18 + IRQ 0 + SIRQ 0 = 116
> > 
> >   PID PR CPU% S  #THR     VSS     RSS PCY UID      Name
> >  1325  0  77% R    17  91268K  32488K  fg root     /system/b2g/b2g
> >  1318  0   6% R     1   1028K    320K  fg root     top
> 
> This looks like you are running top to get the memory?  Can you you paste
> the output of procrank or b2g-procrank when this problem occurs?

when the contacts imported with speed one contact in 1s, then you stop the import;
you close the contact app, then open the contact app, the mobile device will reboot.
or you go to contact form view, and click the import sim contact button, the mobile will reboot.

Comment 33

[buri.blff]

(In reply to Ben Kelly [:bkelly] from comment #29)
> The fine folks on #gfx inform me that if we are somehow creating a large
> number of layers (for example one for each contact or something) then we
> could trigger the gralloc failure.
> 
> I will try testing with "layers.draw-borders" enabled later today to see how
> the layers are structured in contacts app.

we should to known, is there limit of the number of div in the list.

Comment 34

[Ben Kelly [:bkelly, not reviewing]]

Unfortunately b2g18 does not offer the layers.draw-borders pref.  Running v1-train gaia on mozilla-central with layers.draw-borders shows a reasonable number of layers.  One for each letter group title, one for the letter group list, etc.  I will continue trying to reproduce this problem.

Comment 35

[Gregor Wagner [:gwagner]]

We might just run out of memory in the parent. Currently we have to load all the contacts once in order to sort them.
I don't think this should block.

Comment 36

[buri.blff]

the device will reboot, i think this is import pr.
it is block.
Once there is too much contacts,  user can't go in to contacts app and can't recovery.
What  should we tell user to do??

Comment 37

[Gregor Wagner [:gwagner]]

What's the number we are talking about? How many contacts do you import?
Afaik a SIM card can hold 500 contacts and not 10000. Why should we block on this unrealistic scenario where a user keeps clicking import?

Comment 38

[buri.blff]

if there are still 10000 contacts, then we import contact form simcard, there will problem.

Comment 39

[buri.blff]

or we should know the limit?
there is no consist number when it will crash?

Comment 40

[Lukas Blakk [:lsblakk] use ?needinfo]

Putting this on Product team's radar for upcoming releases. Let's get a real spec here for what our ceiling should be, how to warn the user about it, and also look at setting limits in other areas of the software in a consistent manner.  This particular use case is not likely to come up in most real-world usage.

Comment 41

[Ben Kelly [:bkelly, not reviewing]]

Taking this off my list for now based on leo- prioritization.  Also, I don't have equivalent sim card with contacts and have not been able to reproduce with my sdcard setup.  I did have more ideas for trying to improve my scard load, though, so if its still a priority to investigate I can pick this up again.

Comment 42

[Wilfred Mathanaraj [:WDM]]

adding to comms discussion for Monday. This should form part of v1.2 contacts import design plus contacts management.

Comment 43

[Ben Kelly [:bkelly, not reviewing]]

Re-apply my field changes after mid-air collision.

Comment 44

[Marcia Knous [:marcia]]

I have attempted to test this on the Buri device, 7-25 partner build. I am currently running into several issues, including the fact that when I am able to import contacts from the SIM card they don't show reliably in the UI, and I cannot import contacts successfully from the SD card. However, during my testing I have not seen any crashes.

As has been suggested having the test file you are using would be useful.

Comment 45

[Marcia Knous [:marcia]]

In addition, I never had to pull out the battery. I probably was able to get 2 imports of 1000 contacts into the phone from the SIM card, but as noted the contacts were only showing numbers and not the contact name.

Comment 46

[Ben Kelly [:bkelly, not reviewing]]

Does this partner build include the uplift from bug 894751?

  https://github.com/mozilla-b2g/gaia/commit/47b513a2ee5ed285c5793dd487613403f03c846a

Also, if you feel comfortable sending me the vcard file you are using I could see if it would hit the issues fixed by that uplift.

Comment 47

[Jon Hylands [:jhylands]]

Developers can use reference workloads to test reasonable loads on contacts. Available contact counts are 200, 500, 1000, and 2000. If larger contact workloads are needed for a specific bug, please get in touch with me, and I can arrange that.

See https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Platform/Gaia/Hacking_Tips_And_FAQ#Reference_Workloads for details. You need a master gaia repo in order to apply reference workloads to a device. The device can be master or v1-train.

Comment 48

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Jon Hylands [:jhylands] from comment #47)
> Developers can use reference workloads to test reasonable loads on contacts.
> Available contact counts are 200, 500, 1000, and 2000. If larger contact
> workloads are needed for a specific bug, please get in touch with me, and I
> can arrange that.

I've tested with these and not seen the behavior reported in this bug.  My impression was that the repeated imports were taking us to much higher, less reasonable numbers of contacts.

I never found a good, existing tool to convert our reference-workload data from csv or json to vcard.  I also did not build a tool to do that.  It would be great if we had it at some point in order to do more standard tests during imports.

Comment 49

[Joe Cheng [:jcheng] (please needinfo)]

QAWANTED for Master

Comment 50

[Joe Cheng [:jcheng] (please needinfo)]

for results from Master/M-C

Comment 51

[Hubert Figuiere [:hub]]

With the reference workload (ie how we load the contacts on the device), 2000 contacts, the only problem encountered is the time it takes to upgrade the database: a one time operation that seem to take several minutes.

Comment 52

[John Hammink]

tcl/Buri
Base build: 7/25, rooted
gecko: 4dd254baf424523d3256ae47bdc20f9b2b8d5217
gaia: b527d7406b4eed3bacdee78597bb4ff21bf9f7d0
Version: 26.0a1
Build ID: 20130813040222

Tested from mozilla central build by repeating import from SIM of 106 contacts, continually.
After repeating this three times, I found that
A.) Contacts froze and could not be scrolled;
B.) Only one instance of each contact was shown.  On other devices, repeatedly importing from the same SIM card shows multiple instances of each contact.  Not sure what the desired behavior is here.

Note that the phone doesn't freeze, only the contacts app freezes and becomes unresponsive.  After 3 imports (totalling about 318 contacts) I could no longer import any more, as the app was responsive, but I was able to reboot the device with the power button and access the homescreen normally.

Also, after the reboot, I could scroll contacts again but it was very choppy.  I could access the import contacts dialog, but could not import any more contacts.

Comment 53

[Ben Kelly [:bkelly, not reviewing]]

(In reply to John Hammink from comment #52)
> tcl/Buri
> Base build: 7/25, rooted
> gecko: 4dd254baf424523d3256ae47bdc20f9b2b8d5217
> gaia: b527d7406b4eed3bacdee78597bb4ff21bf9f7d0
> Version: 26.0a1
> Build ID: 20130813040222
> 
> Tested from mozilla central build by repeating import from SIM of 106
> contacts, continually.
> After repeating this three times, I found that
> A.) Contacts froze and could not be scrolled;
> B.) Only one instance of each contact was shown.  On other devices,
> repeatedly importing from the same SIM card shows multiple instances of each
> contact.  Not sure what the desired behavior is here.

It looks like you were testing with gaia/master and m-c here.  In gaia/master new code has been added to remove duplicates.  So it will not be possible to reproduce the original issue in this bug in 1.2.  There may still be an issue with large numbers of contacts, but importing duplicates won't work to get there.

That being said, it sounds like maybe you hit a new/different bug in the de-duplication code that froze contacts.  I went ahead an opened bug 905167 to track that.

Comment 54

[Hubert Figuiere [:hub]]

Blocking on bug 905656 that prevent me from importing contacts from a .vcf.

Comment 55

[Hubert Figuiere [:hub]]

I found https://marketplace.firefox.com/app/vcardin to workaround bug 905656 and import a rather large .vcf with 25000 contacts. The all got, albeit imperfectly imported.

Then when I open contacts, and I start scrolling, things move unexpectingly.

What is happening is that it takes a while to populate the whole list and therefore items get added to the list. This happen for a bit then all is good.

This lead to a problem that is two folder.

1. can we load and populate the list faster?

2. can we do it in a way so that the user doesn't get confused with the items "order" changing while being displayed?

Comment 56

[Hubert Figuiere [:hub]]

Bug 909935 is about loading contacts progressively

Comment 57

[Hubert Figuiere [:hub]]

Also another slowliness is with the reference workload (x-heavy) it takes a long (several seconds) time to be able to scroll directly to the letter Z.

The effect is that nothing happens. We can see that things are being loaded alphabetically.

Comment 58

[Kevin Grandon :kgrandon]

(In reply to Hubert Figuiere [:hub] from comment #57)
> Also another slowliness is with the reference workload (x-heavy) it takes a
> long (several seconds) time to be able to scroll directly to the letter Z.

Hi Hub, FYI we're tracking this issue in bug 865747.

Comment 59

[Hubert Figuiere [:hub]]

> Hi Hub, FYI we're tracking this issue in bug 865747.

Marking as blocking this.

Comment 60

[Preeti Raghunath(:Preeti)]

Mike,

Any progress here?

Comment 61

[Preeti Raghunath(:Preeti)]

Hub

Any progress here?

Comment 62

[Hubert Figuiere [:hub]]

(In reply to Preeti Raghunath(:Preeti) from comment #61)
> Hub
> 
> Any progress here?

Still waiting on the blocking bugs that should solve identified problems to see if there is still issues.

Comment 63

[Hubert Figuiere [:hub]]

Related: bug 927348 that is about writing to test to import from SD card.

Comment 64

[Alex Keybl [:akeybl]]

Not a regression from 1.1, so we won't be taking a fix in 1.2 a this point (critical blockers only). We should get a fix into future releases instead.

Comment 65

[Eric Rahm [:erahm]]

Closing out old Firefox OS specific memshrink bugs.