888666 - Contacts API: mozContacts.getAll() seems slow

Status: RESOLVED WONTFIX

(Core Graveyard :: DOM: Contacts, defect, P5)

Description

[Ben Kelly [:bkelly, not reviewing]]

Currently mozContacts.getAll() seems abnormally slow.  In a stripped down app that does nothing but call getAll() with no sorting or filtering options, it takes ~3.5 seconds to load 1000 contacts on my Buri.

While I know that a lot of good work has already gone into optimizing Contacts API, I can't help feeling that things are still unusually slow.

In an attempt to quantify this feeling I thought perhaps we could look at it in terms of bandwidth.  If we call JSON.stringify().length on the 1000 contacts in reference-workload-heavy, we can approximate the amount information to be accessed at 645,628 bytes.  This means we are accessing contact data at about 1.4Mbit/s from the app.

For comparison:

  mozContacts.getAll():    1.4   Mbit/s
  802.11b:                 11    Mbit/s
  File system:             50    Mbit/s
  Memory copy:             650   Mbit/s

(The file system was estimated using dd on a cold cache right after device boot.  Memory bandwidth was estimated from dd on a hot cache.)

I understand this comparison is a bit unfair since contacts has to provide more functionality than simple data access, but I just wanted to show the differences for some rough perspective.

Considering that all of the contacts data is local to the device, the current mozContacts.getAll() numbers look a bit out of whack to me.  They might seem reasonable if we were running over the network, but we're not.

This was all on a Buri using mozilla-central at 134166:12cdc8931e48.

In any case, I was not aware of a top level bug discussing this so I thought I would write my thoughts up.  I hope this bug is not too controversial.  From discussing the topic with various people it seems like its a known issue that we would like to address.  It just seemed that it might be helpful to separate it out into its own bug.

Comment 1

[Ben Kelly [:bkelly, not reviewing]]

Attachment: Link to simple app used to measure contacts API.

The code for the simple test app I used to measure the contacts API on its own can be found here:

  https://github.com/wanderview/gaia/tree/contacts-api-bench/test_apps/contacts-bench

Comment 2

[Kevin Grandon :kgrandon]

Just making a note that we should never ever use, or encourage people to use, getAll for the contacts API. Perhaps some performance investigation wouldn't hurt if we could speed up cursor methods though :)

Comment 3

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Kevin Grandon :kgrandon from comment #2)
> Just making a note that we should never ever use, or encourage people to
> use, getAll for the contacts API. Perhaps some performance investigation
> wouldn't hurt if we could speed up cursor methods though :)

mozContacts.getAll() does return a cursor.  Are you thinking of IDB's mozGetAll()?

Comment 4

[Kevin Grandon :kgrandon]

Ah right, I must be - ignore me. Sorry for the comment spam!

Comment 5

[Ben Kelly [:bkelly, not reviewing]]

One current working theory is that the locale based sorting outside of IDB is contributing to the slowdown.  To fix that, however, we would need bug 871846.

Comment 6

[Ben Kelly [:bkelly, not reviewing]]

Bug 872741 would also help with these load times as it helps optimize the loading of the sort cache.

Comment 7

[Ben Kelly [:bkelly, not reviewing]]

Bug 873873 shows that IPC costs from parent to child process is a non-trivial part of the picture.

Comment 8

[Ben Kelly [:bkelly, not reviewing]]

I re-ran the benchmark with a recent m-c today:

  Rev:      139136:4eeefa65fffa
  Results:  3459ms, 3513ms, 3502ms, 3603ms, 3477ms
  Content:  1000 contacts (reference-workload-heavy)

Comment 9

[Ben Kelly [:bkelly, not reviewing]]

I rebased my contacts API bench app here today:

  https://github.com/wanderview/gaia/tree/contacts-api-bench/test_apps/contacts-bench

And re-ran the benchmark again m-c:

  Rev:      150676:3a486112072b
  Results:  3900ms, 3913ms, 3915ms
  Content:  1000 contacts (reference-workload-heavy)

It seems we have regressed about 10%.

Comment 10

[Ben Kelly [:bkelly, not reviewing]]

Gregor, any idea what could have caused mozContacts.getAll() to regress by about 10%?  Unfortunately I have quite a wide starting range between comment 8 and comment 9.

Comment 11

[Ben Kelly [:bkelly, not reviewing]]

Note, I did change the contacts bench app to wait 15 seconds instead of 5 seconds before starting the test.  I wanted to make sure there was no conflict with the pre-allocated process launch (which occurs on a 5 second delay).

Comment 12

[Ben Kelly [:bkelly, not reviewing]]

I opened bug 927028 track the regression from comment 9.

Comment 13

[Ben Kelly [:bkelly, not reviewing]]

We have regressed further due to the webidl change.  See bug 931164.  Recent numbers:

  152235:2f2a45f04e7c + workaround patch from bug 929652:  4056ms, 4014ms, 4061ms

Comment 14

[Ben Kelly [:bkelly, not reviewing]]

Speculatively setting a dependency on bug 768074 since it seems probably it could help here.

Also, dropping this from my current queue as I need to focus on 1.2 bugs.

Comment 15

[James Zhang (Spreadtrum)]

Can we improve this API performance for tarako only?

Comment 16

[Gregor Wagner [:gwagner]]

(In reply to James Zhang from comment #15)
> Can we improve this API performance for tarako only?

What do you want to improve? Whats the STR thats slow? Is it the first contact returned, the time to fetch all contacts? And whats the workload?

Comment 17

[Joe Cheng [:jcheng] (please needinfo)]

perf improvement. let's not block on this for tarako release unless it is extremely slow

Comment 18

[Firefox Bug Husbandry Bot]

Bulk priority change, per :mdaly

Comment 19

[Sylvestre Ledru [:Sylvestre]]

DOM: Contacts isn't used anymore. 
Closing all remaining bugs.