Closed Bug 1339321 (SQLite3.17.0) Opened 3 years ago Closed 3 years ago

Upgrade to SQLite 3.17.0

Categories

(Toolkit :: Storage, defect)

defect
Not set

Tracking

()

RESOLVED FIXED
mozilla54
Tracking Status
firefox52 --- fixed
firefox-esr52 --- fixed
firefox53 --- fixed
firefox54 --- fixed

People

(Reporter: RyanVM, Assigned: RyanVM)

References

()

Details

Attachments

(1 file)

+++ This bug was initially created as a clone of Bug #1328161 +++

2017-02-13 - Release 3.17.0

SQLite version 3.17.0 is a regularly scheduled maintenance release.

Most of the changes in this release are performance optimizations. Optimizations to the R-Tree extension are especially noticeable.

In this release, the default size of the lookaside buffer allocated for each database connection is increased from 64,000 to 120,000 bytes. This provides improved performance on many common workloads in exchange for a small increase in memory usage. Applications that value a small memory footprint over raw speed can change the lookaside buffer size back to its old value (or to zero) using the SQLITE_DEFAULT_LOOKASIDE compile-time option, or the sqlite3_config(SQLITE_CONFIG_LOOKASIDE) start-time setting, or the sqlite3_db_config(SQLITE_DBCONFIG_LOOKASIDE) run-time setting.

Full changelog:
http://sqlite.org/releaselog/3_17_0.html
Note, we may want to uplift this, at least to Aurora, per https://bugzilla.mozilla.org/show_bug.cgi?id=1309696#c29

The specific change:
"Fixed performance problems and potential stack overflows when creating views from multi-row VALUES clauses with hundreds of thousands of rows."
Looks like on MacOS that limit is much lower than hundred thousands, and it's just a few thousands.
Duplicate of this bug: 1339110
(In reply to Marco Bonardo [::mak] from comment #3)
> Note, we may want to uplift this, at least to Aurora, per
> https://bugzilla.mozilla.org/show_bug.cgi?id=1309696#c29

Whether it's via uplifting the 3.17.0 release or getting a custom 3.15.3 build, I still think we should do something for 52 as well since it's going to be the next ESR release.
https://hg.mozilla.org/mozilla-central/rev/b3404ec6a4f0
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
(In reply to Ryan VanderMeulen [:RyanVM] from comment #6)
> Whether it's via uplifting the 3.17.0 release or getting a custom 3.15.3
> build, I still think we should do something for 52 as well since it's going
> to be the next ESR release.

3.16 and 3.17 were mostly maintenance releases, without large breaking changes, so we could even try to uplift this. It's not something we should do normally, but considering the large amount of perf and stability improvements in these versions, plus the crash fix, it sounds like a good idea.

We should also try to fix bug 1309696 separately, so that in case we should ever need to undo this, we'll have a separate proper fix.
(In reply to Wes Kocher (:KWierso) from comment #7)
> https://hg.mozilla.org/mozilla-central/rev/b3404ec6a4f0

Approval Request Comment
[Feature/Bug causing the regression]: n/a
[User impact if declined]: crashes from bug 1309696
[Is this code covered by automated tests?]: yes, both in our tree and extensively upstream
[Has the fix been verified in Nightly?]: as much as a third-party library update can be :)
[Needs manual test from QE? If yes, steps to reproduce]: no
[List of other uplifts needed for the feature/fix]: none
[Is the change risky?]: not really
[Why is the change risky/not risky?]: as Marco pointed out, none of the recent SQLite updates have had any major breaking changes in them. We also have a lot of in-tree tests that exercise SQLite and I'm confident that any major issues would be caught by them.
[String changes made/needed]: none
Attachment #8837610 - Flags: approval-mozilla-beta?
Attachment #8837610 - Flags: approval-mozilla-aurora?
Depends on: 1340028
A note for future updates: the single sqlite.c source is, in essence, a unified build of the normal sqlite multi-file source. Since we have unified builds of our own, we could import the multi-file source and use UNIFIED_SOURCES.
(In reply to Mike Hommey [:glandium] from comment #10)
> A note for future updates: the single sqlite.c source is, in essence, a
> unified build of the normal sqlite multi-file source. Since we have unified
> builds of our own, we could import the multi-file source and use
> UNIFIED_SOURCES.

Apart from the obvious benefit of not blocking the UI when one tries to see part of the Sqlite code in dxr, what are the advantages?
note: for uplift we'll also need the upstream patch for bug 1340028. We can take upstream patches, we just don't take mozilla-only patches.
I just landed an import of the upstream patch on inbound with Marco's blessing. I'll fold it into the uplift in this bug when/if it's approved.
Comment on attachment 8837610 [details]
Upgrade SQLite to version 3.17.0

update sqlite to fix some recursion issue on osx, aurora53+, beta52+
Attachment #8837610 - Flags: approval-mozilla-beta?
Attachment #8837610 - Flags: approval-mozilla-beta+
Attachment #8837610 - Flags: approval-mozilla-aurora?
Attachment #8837610 - Flags: approval-mozilla-aurora+
Blocks: SQLite3.18.0
You need to log in before you can comment on or make changes to this bug.