1969325 - (SQLite3.50.1) Upgrade to SQLite 3.50.1

Status: RESOLVED FIXED

(Core :: SQLite and Embedded Database Bindings, task)

Description

[Ryan VanderMeulen [:RyanVM]]

SQLite Release 3.50.0 On 2025-05-29

• Add the sqlite3_setlk_timeout() interface which sets a separate timeout, distinct from the sqlite3_busy_timeout(), for blocking locks on builds that support blocking locks.
• The SQLITE_DBCONFIG_ENABLE_COMMENTS constraint (added in the previous release) is relaxed slightly so that comments are always allowed when reading the schema out of a pre-existing sqlite_schema table. Comments are only blocked in new SQL.
• New SQL functions:
  • unistr()
  • unistr_quote()
• For the %Q and %q conversions in the built-in printf() (which covers the sqlite3_mprintf() API and the format() SQL function and similar) the alternate-form-1 flag ("#") causes control characters to be converted into backslash-escapes suitable for unistr().
• CLI enhancements:
  • Avoids direct output of most control characters.
  • The output of the .dump command makes use of the new unistr() SQL funtion to encode special characters, unless the --escape mode is set to off.
  • Better formatting of complex partial indexes in the output from the ".schema --indent" command.
• Enhancements to sqlite3_rsync:
  • The requirement that the database be in WAL mode has been removed.
  • The sync protocol is enhanced to use less network bandwidth when both sides start out being very similar to one another.
  • The sqlite3_rsync program now works on Macs without having to specify the full pathname of the sqlite3_rsync executable on the remote side as long as you install the sqlite3_rsync executable in one of these directories: $HOME/bin:/usr/local/bin:/opt/homebrew/bin
• Changes to JSON functions:
  • Bug fix: Enforce the JSON5 restriction that the "\0" escape must not be followed by a digit.
  • Bug fix: When the LABEL argument to json_group_object(LABEL,VALUE) is NULL, that element of the resulting object is omitted.
  • Optimization: If the jsonb_set() or jsonb_replace() functions make a change in the interior of a large JSONB object, they strive to keep the size of the JSONB object unchanged and to modify as few bytes as possible on the interior of the object. This helps reduce I/O as it allows SQLite to write only the page that contains the changed bytes and not all the surrounding pages.
• Improved support for building on Cygwin and MinGW and similar, as well as Termux.
• Typo fixes in the documentation and in the source code comments.
• Miscellaneous performance improvements.
• JavaScript/WASM:
  • Fix a long-standing filename digest calculation bug in the OPFS SAHPool VFS. Databases created in that VFS by 3.50.0+ cannot be read by older versions of the VFS, but 3.50.0 can backwards-compatibly work with existing databases created by older versions.

Looks like this is hitting FTS5 UBSan failures on Try, however.
https://treeherder.mozilla.org/logviewer?job_id=510632328&repo=try&lineNumber=4404

Comment 1

[Marco Bonardo [:mak]]

The stack from the failure

fts5_index.c:6787:15: runtime error: member access within null pointer of type 'Fts5Structure' (aka 'struct Fts5Structure')
#0 0x7f2950986f39 in sqlite3Fts5IndexReinit /builds/worker/workspace/obj-build/third_party/sqlite3/ext/fts5_index.c:6787:15
#1 0x7f2950986f39 in sqlite3Fts5IndexOpen /builds/worker/workspace/obj-build/third_party/sqlite3/ext/fts5_index.c:6832:14
#2 0x7f2950986f39 in fts5InitVtab /builds/worker/workspace/obj-build/third_party/sqlite3/ext/fts5_main.c:425:10
#3 0x7f2950977f26 in fts5CreateMethod /builds/worker/workspace/obj-build/third_party/sqlite3/ext/fts5_main.c:485:10
#4 0x7f2950772177 in vtabCallConstructor /builds/worker/checkouts/gecko/third_party/sqlite3/src/sqlite3.c:158087:8
#5 0x7f2950740d40 in sqlite3VtabCallCreate /builds/worker/checkouts/gecko/third_party/sqlite3/src/sqlite3.c:158264:10
#6 0x7f295070c347 in sqlite3VdbeExec /builds/worker/checkouts/gecko/third_party/sqlite3/src/sqlite3.c:102494:10
#7 0x7f295066654d in sqlite3Step /builds/worker/checkouts/gecko/third_party/sqlite3/src/sqlite3.c:92276:10
#8 0x7f295066654d in sqlite3_step /builds/worker/checkouts/gecko/third_party/sqlite3/src/sqlite3.c:92337:16
#9 0x7f296be4e923 in rusqlite::raw_statement::RawStatement::step::hb31b4397b6915dbb /builds/worker/checkouts/gecko/third_party/rust/rusqlite/src/raw_statement.rs:115:30
#10 0x7f296be4f38e in rusqlite::statement::Statement::step::h5fe21dfc6fb582c0 /builds/worker/checkouts/gecko/third_party/rust/rusqlite/src/statement.rs:849:15
#11 0x7f296be4f38e in rusqlite::Connection::execute_batch::h4120697a731f3eec /builds/worker/checkouts/gecko/third_party/rust/rusqlite/src/lib.rs:567:40

maybe related to this change?
https://github.com/sqlite/sqlite/commit/bd0e3ed522a10f32d689bda1991068e35add65fb

Richard, any thougts about this?

Comment 2

[D. Richard Hipp]

Preliminary analysis suggests that the problem is that y'all are compiling FTS5 separately, rather than using the FTS5 that is normally built into the amalgamation. (Why?) Apparently, when FTS5 is part of the amalgamation, it sees the "#include <stddef.h>" contained within the main body of SQLite and therefore does not try to create its own implementation of offsetof() at https://sqlite.org/src/info/bffbd0acdcdf5098996?ln=82. Clang seems to be unhappy with that definition of offsetof(). There is now a check-in https://sqlite.org/src/info/838deb7f3423df84 that adds "#include <stddef.h>" to FTS5 and might work around your error.

We think that a better solution, however, is to build FTS5 as part of the amalgamation, because that is the way we test it since it never occurred to us that somebody would build FTS5 separately from the rest of the system.

Comment 3

[Marco Bonardo [:mak]]

(In reply to D. Richard Hipp from comment #2)

Preliminary analysis suggests that the problem is that y'all are compiling FTS5 separately, rather than using the FTS5 that is normally built into the amalgamation. (Why?)

We build it as a loadable extension, for connection that need it. That seems to be supported according to https://www.sqlite.org/fts5.html#building_a_loadable_extension. We package all SQLite loadable extensions together, apart from SQLite core.

Comment 4

[Ryan VanderMeulen [:RyanVM]]

Attachment: Bug 1969325 - Upgrade to SQLite 3.50.1. r=mak

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Green on Try with that change included.
https://treeherder.mozilla.org/jobs?repo=try&revision=4a8bd55410a0e59bf39b3dc27a091244a31bdb25

Comment 6

[Ryan VanderMeulen [:RyanVM]]

Changes in this specific patch release, version 3.50.1 (2025-06-06):

• Fix a long-standing bug in jsonb_set() and similar that was exposed by new optimizations added in version 3.50.0.
• Fix an apparently harmless ASAN warning that can occur on builds that use -DSQLITE_DEFAULT_MEMSTATUS=0.
• Fix an off-by-one bug in sqlite3_rsync that can result in the last page not being transferred for the replicate database.
• Query planner optimization: Allow the right-hand side of a LEFT JOIN to be flattened even if it is a virtual table.
• Fix sqlite3_setlk_timeout() to use a blocking lock when opening a snapshot transaction and when block by another process running recovery.
• Other minor fixes that were reported after the 3.50.0 release.

Comment 7

[D. Richard Hipp]

Just to be clear, the problem reported in comment #1 above should also be resolved by the 3.50.1 patch. I did not write a separate bullet point in the release notes for that. It comes under the "Other minor fixes".

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/60e831201b6a
https://hg.mozilla.org/integration/autoland/rev/4e83647673a5
Upgrade to SQLite 3.50.1. r=mak

Comment 9

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/4e83647673a5