1513320 - SQLite in some Docker images doesn't support common table expressions

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Gregory Szorc [:gps]]

Mercurial's new SQLite backend uses common table expressions (CTEs) for a query (specifically WITH RECURSIVE).

SQLite doesn't support this feature until version 3.8.3. Apparently some of our Docker images are running an older SQLite version because tasks like https://taskcluster-artifacts.net/dN8YrSwkSbqiyXTssFYI_Q/0/public/logs/live_backing.log fail with an error related to WITH not being supported.

We'll need to upgrade to a modern SQLite before we can use the modern Mercurial feature.

Comment 1

[Gregory Szorc [:gps]]

According to Python, sqlite3.sqlite_version reports 3.7.13.

We'll want to build a backported SQLite for the debian 7 based images. I'll get on that.

Comment 2

[Gregory Szorc [:gps]]

Attachment: Bug 1513320 - SQLite package backport for Debian 7; r?glandium

The SQLite in Debian 7 (3.7.13) lacks support for common table
expressions (the WITH keyword), which was introduced in SQLite
3.8.3. The Mercurial SQLite storage backend currently relies on
CTEs. Even if a future Mercurial doesn't require CTE, it is likely
that it will still use CTE if available for performance reasons.
So, it is in our best interest to give Mercurial access to a
modern SQLite. Plus, using a modern SQLite and avoiding potential
bugs in old versions seems prudent.

This commit introduces a SQLite package backport for Debian 7
so we can use the new SQLite feature. We had to minimally patch
the build to work with an older version of TCL that isn't using
multiarch.

I observed libsqlite3 being installed in various other packages.
So I added the package dependency. I doubt this matters too much
since SQLite should be dynamically linked. But I don't think it
will hurt. And for the case of Python, I wouldn't be surprised if
Python is detecting SQLite features at build time, so having the
modern SQLite installed then makes sense.

Comment 3

[Justin Wood (:Callek)]

I've gone ahead and scheduled this to autoland

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ea0c6363ebc4
SQLite package backport for Debian 7; r=glandium

Comment 5

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/ea0c6363ebc4

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/0dcfb6c107ea
Port bug 1513320 - SQLite package backport for Debian 7. rs=bustage-fix

Comment 7

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/b48379043d27

Comment 8

[Jorg K (CEST = GMT+2)]

TB 60.6 ESR:
https://hg.mozilla.org/releases/comm-esr60/rev/c5db8c7bace938ccedec3a327353c9c2b83ed42d