Closed Bug 928251 (SQLite3.8.1) Opened 8 years ago Closed 8 years ago

Upgrade to SQLite 3.8.1

Categories

(Toolkit :: Storage, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla28

People

(Reporter: boaz.dodin, Assigned: RyanVM)

References

()

Details

(Whiteboard: [qa-])

Attachments

(1 file)

http://www.sqlite.org/news.html

SQLite version 3.8.1 is a regularly scheduled maintenance release. Upgrading from the previous release is optional, though you should upgrade if you are using partial indices as there was a bug related to partial indices in the previous release that could result in an incorrect answer for count(*) queries.

The next generation query planner that was premiered in the previous release continues to work well. The new query planner has been tweaked slightly in the current release to help it make better decisions in some cases, but is largely unchanged. Two new SQL functions, likelihood() and unlikely(), have been added to allow developers to give hints to the query planner without forcing the query planner into a particular decision.

Version 3.8.1 is the first SQLite release to take into account the estimated size of table and index rows when choosing a query plan. Row size estimates are based on the declared datatypes of columns. For example, a column of type VARCHAR(1000) is assumed to use much more space than a column of type INT. The datatype-based row size estimate can be overridden by appending a term of the form "sz=NNN" (where NNN is the average row size in bytes) to the end of the sqlite_stat1.stat record for a table or index. Currently, row sizes are only used to help the query planner choose between a table or one of its indices when doing a table scan or a count(*) operation, though future releases are likely to use the estimated row size in other contexts as well. The new PRAGMA stats statement can be used to view row size estimates.

Version 3.8.1 adds the SQLITE_ENABLE_STAT4 compile-time option. STAT4 is very similar to STAT3 in that it uses samples from indices to try to guess how many rows of the index will be satisfy by WHERE clause constraints. The difference is that STAT4 samples all columns of the index whereas the older STAT3 only sampled the left-most column. Users of STAT3 are encouraged to upgrade to STAT4. Application developers should use STAT3 and STAT4 with caution since both options, by design, violate the query planner stability guarantee, making it more difficult to ensure uniform performance is widely-deployed and mass-produced embedded applications.
Depends on: SQLite3.8.0.2
http://www.sqlite.org/releaselog/3_8_1.html

* Added the unlikely() and likelihood() SQL functions to be used as hints to the query planner.
* Enhancements to the query planner:
   Take into account the fact WHERE clause terms that cannot be used with indices still probably reduce the number of output rows.
   Estimate the sizes of table and index rows and use the smallest applicable B-Tree for full scans and "count(*)" operations. 
* Added the soft_heap_limit pragma.
* Added support for SQLITE_ENABLE_STAT4
* Added support for "sz=NNN" parameters at the end of sqlite_stat1.stat fields used to specify the average length in bytes for table and index rows.
* Avoid running foreign-key constraint checks on an UPDATE if none of the modified columns are associated with foreign keys.
* Added the SQLITE_MINIMUM_FILE_DESCRIPTOR compile-time option
* Added the win32-longpath VFS on windows, permitting filenames up to 32K characters in length.
* The Date And Time Functions are enhanced so that the current time (ex: julianday('now')) is always the same for multiple function invocations within the same sqlite3_step() call.
* Add the "totype.c" extension, implementing the tointeger() and toreal() SQL functions.
* FTS4 queries are better able to make use of docid<$limit constraints to limit the amount of I/O required.
* Added the hidden fts4aux languageid column to the fts4aux virtual table.
* The VACUUM command packs the database about 1% tighter.
* The sqlite3_analyzer utility program is updated to provide better descriptions and to compute a more accurate estimate for "Non-sequential pages"
* Refactor the implementation of PRAGMA statements to improve parsing performance.
* The directory used to hold temporary files on unix can now be set using the SQLITE_TMPDIR environment variable, which takes precedence over the TMPDIR environment variable. The sqlite3_temp_directory global variable still has higher precedence than both environment variables, however.
* Added the PRAGMA stats statement.
* Bug fix: Return the correct answer for "SELECT count(*) FROM table" even if there is a partial index on the table. Ticket a5c8ed66ca.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Alias: SQLite3.8.1
Summary: (SQLite3.8.1) Upgrade to SQLite 3.8.1 → Upgrade to SQLite 3.8.1
https://tbpl.mozilla.org/?tree=Try&rev=5c3b0813d546

Pushed on top of m-c parent 54e8c6492dc4 for non-PGO perf comparisons.
Assignee: nobody → ryanvm
Status: NEW → ASSIGNED
Attachment #830868 - Flags: review?(mak77)
One scary-looking failure on Windows, but I couldn't hit it again after 50 runs.
https://tbpl.mozilla.org/php/getParsedLog.php?id=30482763&full=1&branch=try#error0
it's not due to the update, looks like it's this code

http://mxr.mozilla.org/mozilla-central/source/storage/src/mozStorageService.cpp#899

894     if (gShutdownChecks == SCM_CRASH) {
895       nsTArray<nsRefPtr<Connection> > connections;
896       getConnections(connections);
897       for (uint32_t i = 0, n = connections.Length(); i < n; i++) {
898         if (connections[i]->ConnectionReady()) {
899           MOZ_CRASH();
900         }
901       }
902     }

there is a connection that is not yet closed on xpcom-shutdown-threads, something related to threads scheduler, that would explain why it didn't reproduce easily
Comment on attachment 830868 [details] [diff] [review]
Upgrade to SQLite 3.8.1 - Mozilla changes

Review of attachment 830868 [details] [diff] [review]:
-----------------------------------------------------------------

thank you
Attachment #830868 - Flags: review?(mak77) → review+
https://hg.mozilla.org/mozilla-central/rev/43993ae52c19
https://hg.mozilla.org/mozilla-central/rev/441be0ffc5eb
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Flags: in-testsuite-
Resolution: --- → FIXED
Target Milestone: --- → mozilla28
Blocks: SQLite3.8.2
Whiteboard: [qa-]
You need to log in before you can comment on or make changes to this bug.