1023129 - Use a version of Mercurial in MozillaBuild that includes the Python json module (so bzexport works)

Status: RESOLVED FIXED

(Firefox Build System :: MozillaBuild, task)

Description

[Marco Zehe (:MarcoZ)]

Right now, if trying to use hg bzexport to attach a patch to a bug, an error message is given that "json" is not defined or not found. From a brief chat in #developers from some time ago, I learned that the version of Python used in MozillaBuild doesn't include json.

Comment 1

[Gregory Szorc [:gps]]

It's actually the Python in Mercurial in MozillaBuild that's the problem: the Python in MozillaBuild will import json just fine.

This issue has been discussed before. I'm pretty sure we already have a bug on file. RyanVM probably knows off hand.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Not sure there's much we can do about this until we migrate to a version of Hg that's not using py2win - i.e. a native MSYS version.

Comment 3

[Ed Morley [:emorley]]

json is included in the py2exe version of Mercurial bundled in TortoiseHg (http://tortoisehg.bitbucket.org) - see:
https://bitbucket.org/tortoisehg/thg/src/default/win32/setup.cfg#cl-4

As is sqlite3 (which is useful for bzexport since it can access the Firefox profile cookie DB for Bugzilla credentials, which otherwise is unsupported on Windows), after I added it to setup.cfg in:
https://bitbucket.org/tortoisehg/thg/issue/2519/please-add-sqlite3-to-tortoisehg-windows

It seems like we need to either:
1) Encourage the Hg project to include json (and any others that may be of use) in their py2exe runs.
2) Build py2exe versions of Mercurial ourselves.
3) Switch to native MSYS Mercurial.

Comment 4

[Ed Morley [:emorley]]

See also bug 728117.

Comment 5

[Ed Morley [:emorley]]

(In reply to Ed Morley [:edmorley UTC+0] from comment #3)
> 1) Encourage the Hg project to include json (and any others that may be of
> use) in their py2exe runs.

Actually, this would be as simple as adding 'json' to the list of packages specified here (they don't use setup.cfg and instead append the options in the distutils setup.py file):
http://selenic.com/hg/file/0f73ed629362/setup.py#l581

However there may be resistance, given:
http://bz.selenic.com/show_bug.cgi?id=2577

> 2) Build py2exe versions of Mercurial ourselves.
> 3) Switch to native MSYS Mercurial.

4) Use the MSI from https://bitbucket.org/tortoisehg/thg-winbuild/downloads/ , which is Mercurial without TortoiseHg, but with the additional Python modules that TortoiseHg ships with.

Comment 6

[Ed Morley [:emorley]]

(In reply to Ed Morley [:edmorley UTC+0] from comment #5)
> > 1) Encourage the Hg project to include json (and any others that may be of
> > use) in their py2exe runs.
> 
> Actually, this would be as simple as adding 'json' to the list of packages
> specified here (they don't use setup.cfg and instead append the options in
> the distutils setup.py file):
> http://selenic.com/hg/file/0f73ed629362/setup.py#l581
> 
> However there may be resistance, given:
> http://bz.selenic.com/show_bug.cgi?id=2577

I posted to the Mercurial mailing list asking if they would be up for this:
http://www.selenic.com/pipermail/mercurial/2014-June/047226.html

Comment 7

[Ryan VanderMeulen [:RyanVM]]

As bug 545432 has nicely documented, the Mercurial build situation is a trainwreck put nicely. I spent a couple hours the other day fighting with various build documentation I found and still ended up without a working build.

At this point, I think I'm inclined to try to go the way of the Mercurial for Python method as discussed in bug 545432 comment 14. I have the necessary tools for diffing a directory before and after an installer runs, so I'm going to try heading down that path and see where it gets me. I may need to push this off to MozillaBuild 2.0, though as I'd like to get 1.10 out the door sooner rather than later.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Given that this is the only issue blocking 1.10 at the moment and me being quite constrained for time right now, I'm going to punt this to 2.0 so I can get candidate builds out the door.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

Bug 545432 should resolve this once and for all.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Bug 545432 has landed. A test build that includes this is available at the link below. Feedback welcome :)
http://people.mozilla.org/~rvandermeulen/MozillaBuildSetup2.0.0pre3.exe