Closed Bug 530667 (SQLite3.6.22) Opened 10 years ago Closed 10 years ago

Upgrade to SQLite 3.6.22

Categories

(Toolkit :: Storage, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla1.9.3a1
Tracking Status
blocking1.9.2 --- -
status1.9.2 --- .4-fixed

People

(Reporter: sdwilsh, Assigned: sdwilsh)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

We'll need this to pick up a fix to how SQLITE_SECURE_DELETE works (so it works right in all cases), as well as fixing the performance regression we hit when upgrading to 3.6.20.
Blocks: 531122
3.6.22 is out very soon.  I got a copy from drh and it passed the try server, so we are going to take that instead.
Summary: Upgrade to SQLite 3.6.21 → Upgrade to SQLite 3.6.22
Alias: SQLite3.6.22
Attached patch Mozilla ChangesSplinter Review
rs=mconnor for SQLite library upgrades.
Pushed this to the try server Monday without issue.  Trying mozilla-central today.
Lets hope the try server is right. :)

http://hg.mozilla.org/mozilla-central/rev/17c42b6b184c
http://hg.mozilla.org/mozilla-central/rev/b2d8a922901c
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9.3a1
I'd like to start talking about getting this into 1.9.2.  This release has a number of stability fixes, plus a bunch of performance wins for us (especially on shutdown).  Right now we have 3.6.16.1 on 1.9.2.  This has baked on mozilla-central for two weeks without any issues reported against it.
blocking1.9.2: --- → ?
OK, so let's start talking. I'll go first:

 - what is the state of automated testing for SQLite usage in Firefox?
 - where have we gotten caught when doing these upgrades before?
 - have we thought if this will have any adverse effects on mobile?
(In reply to comment #6)
>  - what is the state of automated testing for SQLite usage in Firefox?
Better than most of the code base.  The recent LCOV reports look suspect because I know there are tests for some of the stuff that it says is not tested, but you can view that report here:
http://people.mozilla.com/~mnandigama/codecoverage_html/storage/src/index.html
Additionally, SQLite has 100% branch test coverage, so the likelihood of introducing a regression is low (although still possible).  See http://www.sqlite.org/testing.html for more details on their testing procedures.

>  - where have we gotten caught when doing these upgrades before?
Performance regressions, but that is easy enough to back out.

>  - have we thought if this will have any adverse effects on mobile?
It should not.  In fact, it should give us a performance and stability win across the board.
Thunderbird would also like to see SQLite 3.6.22 on 1.9.2 to be able to minimize forkage.

We released TB 3.1a1 with 3.6.22, and TB 3.0 with 3.6.20 if that provides any reassurance.  (I'd go with the insane level of testing the SQLite people perform, myself.)
:asuth when are you planning on shipping a milestone off of 1.9.2? Trying to determine if we need this for 1.9.2.2 or can wait for a future one.

I don't think this blocks, per se, but we're definitely interested. Is there a changelist that drivers can look over?
blocking1.9.2: ? → -
(In reply to comment #9)
> I don't think this blocks, per se, but we're definitely interested. Is there a
> changelist that drivers can look over?
http://www.sqlite.org/changes.html

1.9.2 is running SQLite 3.6.16.1.
(In reply to comment #9)
> :asuth when are you planning on shipping a milestone off of 1.9.2? Trying to
> determine if we need this for 1.9.2.2 or can wait for a future one.

Thunderbird 3.1b1 is shipping this week-ish and is using a mozilla-1.9.2 release branch where we have patched 3.6.22 into place.  (The reason for the patching is actually for the storage component more than SQLite; but since we already had to patch...)

Thunderbird 3.1b2 will ship at the end of the March or early April.  We will no longer be using a specially patched mozilla-1.9.2 release for storage purposes and will instead be using a copy of mozilla-central's storage transplanted into comm-central.  That landing will likely happen this week so that our nightlies are usable.  We will transplant SQLite 3.6.22 as well if it's not already in mozilla-1.9.2 since that will eliminate a lot of crash-stats debates for us.

It's not a big deal if we have to do it, it's just preferable not to have to do it.
Comment on attachment 420363 [details] [diff] [review]
Mozilla Changes

Pretty pretty please can we land SQLite 3.6.22 for 1.9.2.3?

The motivation is that Thunderbird needs a more recent release of SQLite than 3.6.16.1 and it turns out that my claim that we could easily fork it into comm-central underestimated the difficulty.  We would otherwise need to turn to a release branch.

As discussed earlier on the bug, the risks of upgrading the SQLite version are very low.
Attachment #420363 - Flags: approval1.9.2.3?
Since today there is SQLite 3.6.23 in m-c, see Bug 551260. So wouldn't it be better to update 1.9.2.3 to 3.6.23? Or is this to early?
No, it hasn't backed in a beta release so we do not know for sure how stable it is.
Just to make an extension to comment 8 - Thunderbird has shipped 3.6.22 since 3.0.2 (since Gecko 1.9.1.8) and we've not seen any significant problems as a result.
(In reply to comment #15)
> Just to make an extension to comment 8 - Thunderbird has shipped 3.6.22 since
> 3.0.2 (since Gecko 1.9.1.8) and we've not seen any significant problems as a
> result.
I am going to prod beltzner in person about this today since here is on-site.  I won't be able to land it until next week, but I bet asuth would be happy to do so since Thunderbird really wants it.
beltzner, can you provide an ETA on when 1.9.2 drivers would be willing to accept 3.6.22 into mozilla-1.9.2?  If it's not imminently, can we get approval to change the sqlite.def file to expose a function we need for our comm-central-only backport of mozStorage?
Comment on attachment 420363 [details] [diff] [review]
Mozilla Changes

a=beltzner for 1.9.2.4 who is hoping like hell this isn't the bug that comes back to bite him in the ass
Attachment #420363 - Flags: approval1.9.2.4? → approval1.9.2.4+
This is effectively the same patch as for trunk except that we also update the sqlite.def file to export sqlite3_db_mutex.

Stuff that trunk has that mozilla-1.9.2 does not have with these changes:
1) On trunk the library has been renamed to "mozsqlite3" (versus "sqlite3") and also has a SHORT_LIBNAME because of os/2.
2) On trunk "NO_PROFILE_GUIDED_OPTIMIZE = 1" is only the case for SOLARIS_SUNPRO_CC, whreas on mozilla-1.9.2 it is the case for everybody.
3) On trunk we have more thorough configure.in checks when system SQLite is requested.

I believe this is all correct, but it bears explicitly calling out so beltzner does not hunt me down.
Attachment #437131 - Flags: review?(sdwilsh)
Comment on attachment 437131 [details] [diff] [review]
SQLite 3.6.22 mozilla changes for mozilla-1.9.2 v1

>--- Shawn Wilsher <me@shawnwilsher.com>, 10/2009
>+-- Shawn Wilsher <me@shawnwilsher.com>, 01/2010
You should update the date to this month, and put your name since you are actually doing that upgrade.

r=sdwilsh
Attachment #437131 - Flags: review?(sdwilsh) → review+
I landed this on mozilla-1.9.2 and think it's sticking.  (There are a few outstanding ridiculous slow Windows optimized builds yet to report in, but they are unlikely to have anything interesting other than random oranges.)  I also needed to remove that pesky obsolete test_storage_combined_sharing.js unit test that sdwilsh gave r+ to remove long ago when the Thunderbird release branches ran into the same trouble:

SQLite 3.6.22:
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/89f29d9aea91
nuked test:
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/c2a5d27c0c5b

beltzner, Thunderbird thanks you for your approval granting.  As a downstream consumer of stable mozilla trees, we do have a nice set of chompers.
Blocks: 524706
Blocks: 524712
You need to log in before you can comment on or make changes to this bug.