1611386 - Unship support for system Sqlite

Status: RESOLVED FIXED

(Core :: SQLite and Embedded Database Bindings, task, P1)

Description

[Marco Bonardo [:mak]]

Considered:

1. the reduced resources we have. In general keeping this support has always requested additional time and resources.
2. Bugs like bug 1607902 are impossible to handle with system Sqlite, and may cause us to lose users. We must ensure Firefox stays usable even if Sqlite has to make breaking changes.
3. we should stop enforcing Firefox database preferences onto system Sqlite (things like the tokenizers, secure delete, disabling unused/dangerous APIs)
4. custom memory allocators complexities
5. the benefits are minimal

We should be re-evaluating this, and if the internal evaluation is to drop, we should write an intent to unship to dev-platform and move from there.

Comment 1

[Jan Varga [:janv]]

Yeah, the benefits seem to be minimal. What are they actually ?

1. Smaller installer size
2. Some memory savings

Comment 2

[Marco Bonardo [:mak]]

It mostly boils down to what you said, plus the "good habit" reasons on Unix systems in general.

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

It's worth noting that there seems to be some divergence from the historical packaging strategy with the introduction of Snaps/Snappy by Canonical/Ubuntu and Flatpak by Fedora/Redhat. As I understand it, these packaging mechanism include all of their dependencies as part of the app distribution. Firefox is a high-enough level application that it seems like it would be an appropriate candidate for packaging in this fashion, assuming that the sandboxing mechanism doesn't impede its core operation.

Comment 4

[Andrew Sutherland [:asuth] (he/him)]

Note that for our plans for private browsing support through encryption of SQLite databases may also necessitate requiring building SQLite in a way that's not compatible with system distributions. Namely, under a candidate plan using per-page AEAD, we will need to reserve portions of the VFS pages for crypto metadata which currently requires defining SQLITE_HAS_CODEC which itself or other required changes might require diverging from what system SQLite can/will ship.

Comment 5

[Viktor Jägersküpper]

2. Bugs like bug 1607902 are impossible to handle with system Sqlite, and may cause us to lose users. We must ensure Firefox stays usable even if Sqlite has to make breaking changes.

As a user affected by bug 1607902, I don't understand this. As far as I know, sqlite 3.31 was uploaded to Debian unstable (Firefox, Firefox ESR and Thunderbird affected) and to the testing repository (not enabled by default) of Arch Linux (only Thunderbird affected), probably other rolling distributions also upgraded to this release. I used Debian unstable in the past, now I am using Arch Linux with the testing repository, and I think that users like me are aware that this kind of bug might occur when upstream (sqlite and/or Mozilla) doesn't manage to find and/or solve this bug in time. I also think that downgrading a package (sqlite in this case) isn't a problem for users of rolling distributions. So I really think that you are not going to lose those users by giving downstream the possibility to use system sqlite.

Comment 6

[Marco Bonardo [:mak]]

(In reply to Viktor Jägersküpper from comment #5)

So I really think that you are not going to lose those users by giving downstream the possibility to use system sqlite.

There are a few points to consider:

1. it may be unclear for the user why Firefox starts crashing, there's no direct relation with Sqlite and the case where the user only upgraded Sqlite is uncommon. that could mean users will just give up and move to a different browser, or suffer frustration to debug the problem.
2. you may have another product that instead requires a newer Sqlite version, and then you're stuck with having to pick one
3. All the costs we have to maintain and write new features, because of this. If the costs overweight the benefits, plus include breaking bugs, then it's not good

Comment 7

[Marco Bonardo [:mak]]

Attachment: Bug 1611386 - Drop support for --enable-system-sqlite. r=asuth,glandium

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5022efe33088
Drop support for --enable-system-sqlite. r=asuth,glandium

Comment 9

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/5022efe33088

Comment 10

[Landry Breuil (:gaston)]

question from a downstream.. does it affect the fact that nss can be built with system sqlite, and firefox can be built against system nss ? might lead to conflicting symbols ?

Comment 11

[Marco Bonardo [:mak]]

(In reply to Landry Breuil (:gaston) from comment #10)

question from a downstream.. does it affect the fact that nss can be built with system sqlite, and firefox can be built against system nss ? might lead to conflicting symbols ?

AFAIK, our Sqlite library is compiled into the nss3 library when nss is built into Firefox, we don't use the Sqlite version that nss provides.
If instead Firefox uses --with-system-nss, then system nss can either have its own Sqlite built-in or use system Sqlite.
There should be no conflict in these cases.
I'm not sure whether it's possible to build nss into firefox with NSS_USE_SYSTEM_SQLITE, but I doubt it.