Improve the performance of importing contacts from SD card

RESOLVED WONTFIX

Status

Firefox OS
Gaia::Contacts
P2
major
RESOLVED WONTFIX
4 years ago
3 months ago

People

(Reporter: Na Hao, Unassigned)

Tracking

({perf})

unspecified
ARM
Gonk (Firefox OS)

Firefox Tracking Flags

(tracking-b2g:backlog, b2g-v1.3T affected, b2g-v1.4 affected)

Details

(Whiteboard: [sprd322750][sprd325604][c=progress p= s= u=][perf])

Attachments

(1 attachment)

4.48 KB, application/zip
Details
(Reporter)

Description

4 years ago
Steps to reproduce:
-------------------------------------------------------
1. Launch Contact
2. Tap on the setting icon on the top right corner
3. Select 'Import Contacts'
4. Select the Memory Card


Actual result:
-------------------------------------------------------
Import contacts from SD card which total space is 16G and 7G left,It takes over 1min to find the available vcf files which contain 488 contacts before import them.It's longer when the left space is smaller.

root cause:
-------------------------------------------------------
The retrieveFiles method of SdCard in communications/contacts/js/utilities/sdcard.js takes the most of time to find the possible vcard type files,than the filearray is delivered to getTextFromFiles method to import the contacts.
(Reporter)

Updated

4 years ago
status-b2g-v1.3T: --- → affected
status-b2g-v1.4: --- → affected
Whiteboard: [sprd322750]
Flags: needinfo?(fabrice)
Do you mind attaching the test file you use for imports?
Flags: needinfo?(fabrice)
(Reporter)

Comment 2

4 years ago
(In reply to Fabrice Desré [:fabrice] from comment #1)
> Do you mind attaching the test file you use for imports?

sorry,I'm a novice programmer,would you mind helping me to write the test file? I don't how to start it?
(In reply to na.hao from comment #2)
> (In reply to Fabrice Desré [:fabrice] from comment #1)
> > Do you mind attaching the test file you use for imports?
> 
> sorry,I'm a novice programmer,would you mind helping me to write the test
> file? I don't how to start it?

I just need the file with the 488 contacts that you talk about in comment 0 to reproduce the issue on my side. Feel free to email it to me if you don't want it to be public. thanks!
Flags: needinfo?(na.hao)
blocking-b2g: --- → 1.3T?
Whiteboard: [sprd322750] → [sprd322750][sprd325604]
(Reporter)

Comment 4

4 years ago
Created attachment 8447800 [details]
4vcard.zip

the 488 contacts
Flags: needinfo?(na.hao)
(Reporter)

Comment 5

4 years ago
(In reply to Fabrice Desré [:fabrice] from comment #3)
> (In reply to na.hao from comment #2)
> > (In reply to Fabrice Desré [:fabrice] from comment #1)
> > > Do you mind attaching the test file you use for imports?
> > 
> > sorry,I'm a novice programmer,would you mind helping me to write the test
> > file? I don't how to start it?
> 
> I just need the file with the 488 contacts that you talk about in comment 0
> to reproduce the issue on my side. Feel free to email it to me if you don't
> want it to be public. thanks!

I have post the vcf files,and also send an email to you,thank u.
(Reporter)

Comment 6

4 years ago
In contrast,with the same SD card, the tcl one touch takes 20s and flame_v1.4 takes over 2min to find the vcf files.I found that most of the time is spent on retrieveFiles method.

Taking an example of one test, the total time spent on retrieveFiles method is 68.647s and 66.775s of them is used for searching and judging if it is the vcf file.so when there are many small files,its search time is longer than the big ones even though they take the same space in SD card.So the perfomance of importing sort of depends on the SD status.
Keywords: perf
Priority: -- → P2
Hardware: Other → ARM
Whiteboard: [sprd322750][sprd325604] → [sprd322750][sprd325604][c=progress p= s= u=]
not blocking as function works and its not frequently used.
add to backlog
blocking-b2g: 1.3T? → backlog
It seems Buri is faster on retrieving vcf files. Following is my test results with the same SD card(4G, 2.6G free). 
Both tests are based on the same steps: Insert SD card -> restart device)

Buri v1.4: 
-> It only took less than 10s to find the vcf file.
-> Total time is 1 min 20s

Dolphin:  
-> It took about 20s to find the vcf file
-> Total time is 1 min 40s
(Reporter)

Comment 9

4 years ago
(In reply to pcheng from comment #8)
> It seems Buri is faster on retrieving vcf files. Following is my test
> results with the same SD card(4G, 2.6G free). 
> Both tests are based on the same steps: Insert SD card -> restart device)
> 
> Buri v1.4: 
> -> It only took less than 10s to find the vcf file.
> -> Total time is 1 min 20s
> 
> Dolphin:  
> -> It took about 20s to find the vcf file
> -> Total time is 1 min 40s

hi pcheng,
thank u for your test.Could you please try an 8G or 16G SD card (more usually be used)with the same percent of left space as mine ? Because when there is lots of small files,it really takes a long time to find vcf file,the test condition is important.
Flags: needinfo?(pcheng)
I agree that with more small files, the performance is worse. Here is my test results.

8G SD card (7G free). 
---------------------------------------------------------------------------------
Buri v1.4: 
-> It only took about 6s to find the vcf file.
-> Total time is 1 min 25s
Dolphin:  
-> It took about 12s to find the vcf file
-> Total time is 1 min 20s. 

8G SD card (5.9G free): I copied some filed to the same sd card.
--------------------------------------------------------------------------------------
Buri v1.4: 
-> It only took about 20s to find the vcf file.
-> Total time is 2 min
Dolphin:  
-> It took about 40s to find the vcf file
-> Total time is 2 min 10s.
Flags: needinfo?(pcheng)
Whiteboard: [sprd322750][sprd325604][c=progress p= s= u=] → [sprd322750][sprd325604][c=progress p= s= u=][perf]

Comment 11

4 years ago
i have tried this case. The slow API is mozContact.save, it takes 150ms average per contact on 6821.

@Fabrice, do you think it is possible to improve it?
Flags: needinfo?(fabrice)
(In reply to renfeng.mei from comment #11)
> i have tried this case. The slow API is mozContact.save, it takes 150ms
> average per contact on 6821.
> 
> @Fabrice, do you think it is possible to improve it?

I don't think there is any easy gain there. ni? on people that worked on that to confirm.
Flags: needinfo?(reuben.bmo)
Flags: needinfo?(fabrice)
Flags: needinfo?(anygregor)
It's a trade-off between lookup and save performance. We want fast lookup and create many indexes we can query for. We can optimize the save time by removing indexes but with the drawback that people can't search for middle-names, nick names, case-insensitive or local phone numbers for example.
Flags: needinfo?(anygregor)
Has anyone profiled where we spend those 150ms? If there's significant time spent in IPC, we could optimize some of it by figuring out a way to batch save operations.
Flags: needinfo?(reuben.bmo)
(In reply to Reuben Morais [:reuben] from comment #14)
> Has anyone profiled where we spend those 150ms? If there's significant time
> spent in IPC, we could optimize some of it by figuring out a way to batch
> save operations.

We looked at it in other bugs and basically all of it is spent in sqlite committing memory.

Comment 16

4 years ago
(In reply to Reuben Morais [:reuben] from comment #14)
> Has anyone profiled where we spend those 150ms? If there's significant time
> spent in IPC, we could optimize some of it by figuring out a way to batch
> save operations.

I quite agree with you. Is it possible to make another thread to build indexes? to save performance, do not build indexes so immediately. but for query, we can build indexes somewhere such as when idle, or before query, etc
Note here - comment 0 claims the time seeking the file is long rather than the actual importing.

Given the size/content of the sd card in the description, is this really a problem?
(Reporter)

Comment 18

4 years ago
(In reply to Wayne Chang [:wchang] from comment #17)
> Note here - comment 0 claims the time seeking the file is long rather than
> the actual importing.
> 
> Given the size/content of the sd card in the description, is this really a
> problem?

It's already given 16G and 7G free.
(Assignee)

Updated

3 years ago
blocking-b2g: backlog → ---
tracking-b2g: --- → backlog

Comment 19

3 months ago
Firefox OS is not being worked on
Status: NEW → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.