1026648 - abort: no module named sqlite3 using mozext

Status: RESOLVED WORKSFORME

(Developer Services :: Mercurial: mozext, defect)

Description

[Benjamin Smedberg]

New win7 computer, stock mozillabuild, enabled mozext pulled from latest 
hg.mozilla.org/hgcustom/version-control-tools.

$ hg pull --rebase inbound
pulling from https://hg.mozilla.org/integration/mozilla-inbound
searching for changes
adding changesets
adding manifests
adding file changes
added 427 changesets with 1807 changes to 1026 files (+1 heads)
abort: No module named sqlite3

I know I've seen a but about including sqlite3 in mozillabuild-python, but in the meantime this should either early-abort or succeed. Is there a workaround?

Is this the import from pylib/mozautomation/mozautomation/changetracker.py? It seems odd that the error happens that late in the process, since mozext/__init__.py has the changetracker import at toplevel.

Comment 1

[Benjamin Smedberg]

workaround from code inspection:

[mozext]
disable_local_database = 1

Comment 2

[Gregory Szorc [:gps]]

This is similar in nature to bug 1023129. It would be really nice if the hg in MozillaBuild used the full Python and not a stripped down one.

Comment 3

[Benjamin Smedberg]

Also now "error: commit.critic hook raised an exception: No module named ConfigParser"
workaround:
[mozext]
noautocritic = True

Although basically we've disabled most of mozext except for the default paths ;-)

Comment 4

[Gregory Szorc [:gps]]

ConfigParser is likely also not packaged with Mercurial's Python. Boo.

We need to invest in making MozillaBuild not suck as much.

Comment 5

[Nick Alexander :nalexander [he/him]]

This appears to be an issue trying to make |mach artifact| work under Windows as well.  If my digging is correct, the chain is that |mach artifact| requires mozext to use the pushlog and mozext requires sqlite3 to maintain the local database.

Perhaps this would be addressed by fixing Bug 1236618, or perhaps we should just ship a functioning sqlite3 in Mozilla Build.

gps, RyanVM: can you comment on the relative difficulty of making this better?

Comment 6

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander [Vacation PTO from Dec 23--Jan 4] from comment #5)
> This appears to be an issue trying to make |mach artifact| work under
> Windows as well.  If my digging is correct, the chain is that |mach
> artifact| requires mozext to use the pushlog and mozext requires sqlite3 to
> maintain the local database.

Worth noting: disable_local_database = 1 is not sensible since |mach artifact| really needs the pushlog.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

As far as I can tell, the sqlite3 library we ship with MozillaBuild (as bundled with Python 2.7) appears to work when invoked:

$ python
Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec  5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> print sqlite3.sqlite_version
3.6.21
>>>

This is with a test build that includes Python 2.7.11, but it appears that SQLite hasn't been updated in ages anyway. I'm happy to make any changes needed to support |mach artifact|, but I'm afraid that I'm not sure where to start at this point.

Comment 8

[Gregory Szorc [:gps]]

Did you actually reproduce this recently with current MozillaBuild? Back in the day, `hg` had its own, minimal Python distribution which was lacking stdlib modules like sqlite3. This should no longer be an issue with modern versions of MozillaBuild, as it runs `hg` from a full Python distribution.

This bug should be marked WORKSFORME because I'm pretty sure I have mozext running w/ sqlite on my home Windows machine just fine.

Comment 9

[Nick Alexander :nalexander [he/him]]

(In reply to Gregory Szorc [:gps] from comment #8)
> Did you actually reproduce this recently with current MozillaBuild? Back in
> the day, `hg` had its own, minimal Python distribution which was lacking
> stdlib modules like sqlite3. This should no longer be an issue with modern
> versions of MozillaBuild, as it runs `hg` from a full Python distribution.

The end of this story is PEBKAC:

I am sharing my srcdir across my Mac and a Windows VM.  My Mac has

nalexander@chocho ~/M/gecko> python -c 'import sqlite3; print(sqlite3.sqlite_version)'
3.8.10.2

but the Windows VM has MozillaBuild with Python 2.7.10 and the *ancient*

Administrator@experien-19087a /z/gecko
$ python -c 'import sqlite3; print(sqlite3.sqlite_version)'
3.6.21

That produces the error I see, which is like http://www.obsidianforensics.com/blog/upgrading-python-sqlite.

> This bug should be marked WORKSFORME because I'm pretty sure I have mozext
> running w/ sqlite on my home Windows machine just fine.

Agreed.