933374 - HUGE Performance regression, Deleting from history takes 14times as long since 20130612

Status: RESOLVED DUPLICATE of bug 734643

(Toolkit :: Places, defect, P3)

Description

[Elbart]

Attachment: ProcessExplorer performance graph for good and bad

STR:
New profile made with 23.0, added a places.sqlite-database with ~150k entries (normal browsing history with favicons etc.), integrity_check result is "ok".
Search for "." (dot) in the history, select the first ~4,000 entries and delete them.

Before first bad: I/O wouldn't go higher than ~500MB
After first bad: I/O would reach ~1.8GB, deletion takes twice as long (90s vs 200s on my machine)

See image for a comparison of the performance graphs

good:
[Build]
BuildID=20130611031140
Milestone=24.0a1
SourceStamp=9115d8b717e1
SourceRepository=http://hg.mozilla.org/mozilla-central

bad:
[Build]
BuildID=20130612031138
Milestone=24.0a1
SourceStamp=0414d6d0f60d
SourceRepository=http://hg.mozilla.org/mozilla-central

Regression range: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=9115d8b717e1&tochange=0414d6d0f60d

Latest Nightly (2013-10-30) is also affected.

Comment 1

[Elbart]

Attachment: sqlite3_analyzer log of the database

Comment 2

[Alice0775 White]

Attachment: places.7z sample places.sqlite

I can reproduce the huge performance regression.
Firefox24 is 14 times slower than Firefox23.


Steps To Reproduce:
1. Extract attachment and Copy places.sqlite into profile folder
2. Start Firefox

3. Open Library  History > Show All History (Ctrl+Shify+H)
4. Click "August" in the keft side pane
4. Click "Organize" menu and Choose "Select All"  (selected 1000 items )
5. Click "Organize" menu and Choose "Del"
    ---- Observe deletion timeRegression window(m-c)
Good: 10s/1000items
http://hg.mozilla.org/mozilla-central/rev/d79910d9e251
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20130610 Firefox/24.0 ID:20130610171414
Bad: 140sec/1000items
http://hg.mozilla.org/mozilla-central/rev/81b227f1a522
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20130611 Firefox/24.0 ID:20130611013316
Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d79910d9e251&tochange=81b227f1a522


Regression window(m-i)
Good: 10s/1000items
http://hg.mozilla.org/integration/mozilla-inbound/rev/21e3c2610814
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20130610 Firefox/24.0 ID:20130610142333
Bad: 140sec/1000items
L:\trunk\2013\06\inbound\firefox inbound 10-Jun-2013 1627 2327 1370902155
http://hg.mozilla.org/integration/mozilla-inbound/rev/b0afcbcafb72
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20130610 Firefox/24.0 ID:20130610150915
Pushlog:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=21e3c2610814&tochange=b0afcbcafb72

Suspected:
85af7479df1c	Marco Bonardo — Bug 769348 - Change selection algorithm for autofilled URLs prefixes. This new variant limits selection to typed pages and suggests only prefixes that match all of those addresses. r=paolo

Comment 3

[Alice0775 White]

Err
s/Firefox24 is 14 times slower than Firefox23/Firefox23 is 14 times slower than Firefox22

Comment 4

[Elbart]

I assume this code hasn't made it into Nightly until 0612, because 0610 and 0611 don't show the same slowness here:

Top:
BuildID=20130610031147
Milestone=24.0a1
SourceStamp=252b1ac4d537

Middle:
BuildID=20130611031140
Milestone=24.0a1
SourceStamp=9115d8b717e1

Bottom:
BuildID=20130612031138
Milestone=24.0a1
SourceStamp=0414d6d0f60d

Thank you for confirming.

Comment 5

[Elbart]

Attachment: 0610 vs 0611 vs 0612 with Alice0775 White's places.sqlite and STR

Comment 6

[Alice0775 White]

(In reply to elbart from comment #4)
> I assume this code hasn't made it into Nightly until 0612.

Yes, 
You can also find the changeset of comment#2 in your Regression range of commnet#0.

Comment 7

[Alex Keybl [:akeybl]]

The tracking flag is for new regressions that are critical (with a fairly high bar of losing us users). We'd take a low risk uplift, but wouldn't track.

Comment 8

[Marco Bonardo [:mak]]

(In reply to Alice0775 White from comment #2)
> Suspected:
> 85af7479df1c	Marco Bonardo — Bug 769348 - Change selection algorithm for
> autofilled URLs prefixes. This new variant limits selection to typed pages
> and suggests only prefixes that match all of those addresses. r=paolo

This is very likely, there's more work to do to keep prefixes up-to-date and have autocomplete work as expected.
Though, we never optimized for edge cases like "select thousands entries and delete them" so it's not top priority, bug 913160 is a more important bug to fix atm. We'll look what we can do to improve things here later.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Comment 10

[Mathew Hodson]

Should this be marked as a duplicate of bug 734643, which is closed now?

Comment 11

[Marco Bonardo [:mak]]

Makes sense, but also because the code around origins update changed so much that the original regrange doesn't help anymore.