909382 - (SQLite3.8.0.2) Upgrade to SQLite 3.8.0.2

Status: RESOLVED FIXED

(Core :: SQLite and Embedded Database Bindings, defect)

Description

[NVD]

http://www.sqlite.org/releaselog/3_8_0.html

http://www.sqlite.org/news.html

"SQLite version 3.8.0 might easily have been called "3.7.18" instead. However, this release features the cutover of the next generation query planner or NGQP, and there is a small chance of breaking legacy programs that rely on undefined behavior in previous SQLite releases, and so the minor version number was incremented for that reason. But the risks are low and there is a query planner checklist is available to application developers to aid in avoiding problems.

SQLite version 3.8.0 is actually one of the most heavily tested SQLite releases ever. Thousands and thousands of beta copies have be downloaded, and presumably tested, and there have been no problem reports.

In addition to the next generation query planner, the 3.8.0 release adds support for partial indices, as well as several other new features. See the change log for further detail."

Comment 1

[Ryan VanderMeulen [:RyanVM]]

https://tbpl.mozilla.org/?tree=Try&rev=91cbd25ad6ac

Build on top of m-c revision 14b1e8c2957e for performance comparisons.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Attachment: Upgrade to SQLite 3.8.0 - Mozilla changes

Try run isn't fully complete yet, but it's green enough that I think we're OK moving forward with a review.

Comment 3

[Marco Bonardo [:mak] (Away 10-21 August)]

I think we are going to wait the begin of a release cycle to take this, considered the new query planner, we want a full cycle of baking to detect eventual perf regressions and have the time to work on them.

Comment 4

[Boaz Dodin]

SQLite Release 3.8.0.1 On 2013-08-29 (3.8.0.1):
http://www.sqlite.org/releaselog/3_8_0_1.html

* Fix an off-by-one error that caused quoted empty string at the end of a CRNL-terminated line of CSV input to be misread by the command-line shell.
* Fix a query planner bug involving a LEFT JOIN with a BETWEEN or LIKE/GLOB constraint and then another INNER JOIN to the right that involves an OR constraint.
* Fix a query planner bug that could result in a segfault when querying tables with a UNIQUE or PRIMARY KEY constraint with more than four columns.

Comment 5

[Ryan VanderMeulen [:RyanVM]]

lulz

Comment 6

[Ryan VanderMeulen [:RyanVM]]

Attachment: Upgrade to SQLite 3.8.0.1 - Mozilla changes

https://tbpl.mozilla.org/?tree=Try&rev=a4c38a2ae384

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Looks like the WinXP xpcshell failure is real. Doesn't happen on the parent m-c changeset.
https://tbpl.mozilla.org/php/getParsedLog.php?id=27223434&tree=Try

Comment 8

[Marco Bonardo [:mak] (Away 10-21 August)]

may be a timing issue, related to the new planner. is it reproducible locally?

Comment 9

[Ryan VanderMeulen [:RyanVM]]

I don't have XP handy to look, sorry.

Comment 10

[Ryan VanderMeulen [:RyanVM]]

It is interesting that the 3.8.0.0 Try run was green.

Comment 11

[Boaz Dodin]

SQLite Release 3.8.0.2 On 2013-09-03 (3.8.0.2):
http://www.sqlite.org/changes.html

* Fix a bug in the optimization that attempts to omit unused LEFT JOINs.

Ryan - does this fixed bug related to the WinXP xpcshell failure?

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Worth a shot!

https://tbpl.mozilla.org/?tree=Try&rev=a82c8cd8f465

Comment 13

[Ryan VanderMeulen [:RyanVM]]

Attachment: Upgrade to SQLite 3.8.0.2 - Mozilla changes

Though with the rate of bug fixes this release is getting, we may be up to version 3.8.1 before we take this :)

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Well, it's green on XP now. So maybe.

Comment 15

[Marco Bonardo [:mak] (Away 10-21 August)]

I don't think it's related cause the fix in 3.8.0.2 was a segfault. It's likely just intermittent.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 799406 [details] [diff] [review]
Upgrade to SQLite 3.8.0.2 - Mozilla changes

Ready to give this a shot? Last try run was green, but I can push it again if you like. I've been running with in my own builds without issue since it was released.

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Mak, ping? :)

Comment 18

[Marco Bonardo [:mak] (Away 10-21 August)]

Comment on attachment 799406 [details] [diff] [review]
Upgrade to SQLite 3.8.0.2 - Mozilla changes

Review of attachment 799406 [details] [diff] [review]:
-----------------------------------------------------------------

yeah, the changelog sounds good, even if I wonder what's the difference between pragma query_only and read-only mode :)

it's likely we may need to update to 3.8.1 soon, though we should take this now to have more testing in Nightly of the new query planner.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b8d8404a4721
https://hg.mozilla.org/integration/mozilla-inbound/rev/65c4f1221204

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/b8d8404a4721
https://hg.mozilla.org/mozilla-central/rev/65c4f1221204