527956 - Performance Regression upgrading to SQLite 3.6.20

Status: RESOLVED FIXED

(Toolkit :: Storage, defect)

Description

[Shawn Wilsher :sdwilsh]

Ts Shutdown MAX Dirty Profile regressed 0.94% on Leopard
Ts Shutdown MAX Dirty Profile regressed 3.09% on Linux

Windows showed no regression, and no other shutdown test showed a regression.

Comment 1

[Shawn Wilsher :sdwilsh]

The database that is used can be found here (warnings, it's close to 200 MB):
http://files.shawnwilsher.com/2009/11/11/dirtyMaxDB.zip

Comment 2

[Shawn Wilsher :sdwilsh]

Marco - was the log you generated based on this places.sqlite file?

Comment 3

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

no, it was based on a small profile, i can make one for this

Comment 4

[D. Richard Hipp]

Marco - In the previous log, host parameters in queries (ex: ?3) were replaced by the values actually bound to those parameters.  However, the values replaced where not quoted.  This caused invalid SQL, which we had to go in and hand-edit.  Is there any easy fix for this, or is it something we'll just have to work around?

Comment 5

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

the log was just generated through a stupid script using storage prlog data (with some custom addition), so that was a blind find and replace (some param could be weird, especially those binded through name could have a bad replace ordering).
i can fix that though, if you need to change the log data should not be a problem, i'll generate a new one with the above db and quoted string params.

Comment 6

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

Attachment: new log from max dirty profile

this is a 200MB txt log with quoted params... good luck :\

Comment 7

[Shawn Wilsher :sdwilsh]

Looking at the graphs, I think the upgrade killed these two regressions.  The automatic scripts haven't shown anything yet though...

Comment 8

[Shawn Wilsher :sdwilsh]

Now we are talking!

Ts Shutdown, MAX Dirty Profile decrease 7.85% on Linux Firefox

Before the regression, it was at 6253.42 (raised to 6446.89).  After the upgrade, it ended up at 5965.58 (from 6473.79).  I'd say this is fixed, with room to spare on Linux.  We'll see if the scripts notice anything on Leopard.

Comment 9

[Shawn Wilsher :sdwilsh]

Ts Shutdown, MAX Dirty Profile decrease 4.84% on Leopard Firefox

Before the regression, it was at 6943.47 (raise to 7008.68).  After the upgrade, it ended up at 6886.95 (from 6553.63).  This bug is now fixed.