972329 - Install setuptools as part of bootstrap

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Panos Astithas (he/him) [:past] (please ni?)]

I get this error all the time on my Linux box, but not on my OS X box:

fx-team$ hg qref
error: commit.critic hook raised an exception: distribute

Disabling mozext makes it go away.

Comment 1

[Gregory Szorc [:gps]]

No stack trace?

That error message isn't a lot to go on. I suspect this may be an issue with Python packaging, as "distribute" is a [legacy] Python package manager thingy.

It would be amazing if you could provide reproduce steps.

Comment 2

[Panos Astithas (he/him) [:past] (please ni?)]

I'm on Ubuntu 13.10, Python 2.7.5+ (as reported by --version) and mercurial 2.8.2. In general I keep everything up to date ('mach mercurial-setup' every few days, etc.). I've always been getting this on fx-team tip every time I simply qrefresh the topmost patch without any changes. Is there anything else I could provide?

Comment 3

[Gregory Szorc [:gps]]

Try running the hg command with --traceback --debug?

Comment 4

[Panos Astithas (he/him) [:past] (please ni?)]

Certainly:

$ hg qref
error: commit.critic hook raised an exception: distribute
$ hg qref --traceback --debug
removing unknown node cfe39af7826e from 1-phase boundary
invalid branchheads cache (served): tip differs
js/src/jit-test/tests/debug/Frame-executionReason-01.js
js/src/jsapi.cpp
js/src/jsapi.h
js/src/jscntxt.cpp
js/src/jscntxt.h
js/src/shell/js.cpp
js/src/vm/Debugger.cpp
js/src/vm/Stack-inl.h
js/src/vm/Stack.h
running hook commit.whitespace: hg export tip | (! grep -C 3 --color=always -n '^+.*[ ]$')
calling hook commit.critic: <function critic_hook at 0x7f085dbc8398>
error: commit.critic hook raised an exception: distribute
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hook.py", line 81, in _pythonhook
    r = obj(ui=ui, repo=repo, hooktype=name, **args)
  File "/home/past/.mozbuild/version-control-tools/hgext/mozext/__init__.py", line 706, in critic_hook
    critique(ui, repo, node=node, **opts)
  File "/home/past/.mozbuild/version-control-tools/hgext/mozext/__init__.py", line 405, in critique
    style = get_style_guide(parse_argv=False, ignore='E128')
  File "/home/past/.mozbuild/version-control-tools/pylib/flake8/flake8/engine.py", line 75, in get_style_guide
    kwargs['parser'], options_hooks = get_parser()
  File "/home/past/.mozbuild/version-control-tools/pylib/flake8/flake8/engine.py", line 38, in get_parser
    (extensions, parser_hooks, options_hooks) = _register_extensions()
  File "/home/past/.mozbuild/version-control-tools/pylib/flake8/flake8/engine.py", line 24, in _register_extensions
    checker = entry.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2014, in load
    if require: self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2027, in require
    working_set.resolve(self.dist.requires(self.extras),env,installer))
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 594, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: distribute

Comment 5

[Julien Wajsberg [:julienw]]

I got something similar with "setuptools" instead of "distribute" and I resolved it by installing the python-setuptools package in my Linux.

Maybe a more useful feedback could be good, but I don't know enough Python to know how to do it :)

Comment 6

[Gregory Szorc [:gps]]

I'd say installing some combination of "pip" "setuptools" and "virtualenv" Python packages should resolve this. I would expect those to get installed as dependencies of other system packages. Kind of crazy these aren't available when Python is installed :/

Comment 7

[Panos Astithas (he/him) [:past] (please ni?)]

Installing python-setuptools fixed it for me, too! Should we close this bug or is there a way to make sure that package gets installed from 'mach bootstrap' or 'mach mercurial-setup'?

Comment 8

[Julien Wajsberg [:julienw]]

Ideally I'd like a better error message than "DistributionNotFound" that's not so useful for non-Python developers.

Comment 9

[Gregory Szorc [:gps]]

Let's install setuptools as part of bootstrap.

Code here in in python/mozboot. The hardest part here is digging up package names for all the operating systems.

Comment 10

[Theodore Kokkoris [:tgkokk]]

I'd like to try this one. I guess that I have to change some of the files here: https://hg.mozilla.org/mozilla-central/file/9d3da41ad0b6/python/mozboot/mozboot, right?

I don't have to change anything for OS X (when using Homebrew, at least), since setuptools gets automatically installed. I'm going to check about MacPorts.

About Linux (and *BSD), it's going to be as "simple" as adding the appropriate package to the package list.

Comment 11

[Zorro]

Hello!
I have learned Python basics and would like to test them by trying to fix this bug. Can I get assigned to it with some starting pointers? I am completely new in open source.

Thanks!

Comment 12

[Gregory Szorc [:gps]]

There are files in python/mozboot/mozboot/ that contain lists of packages to install. You want to spin up some virtual machines, run the one-line bootstrapper from [1] and verify "setuptools" is a valid command in your shell when it completes.

[1] https://developer.mozilla.org/en-US/docs/Simple_Firefox_build/Linux_and_MacOS_build_preparation

Comment 13

[Zorro]

Thank you for your reply. So I completed the steps in [1]. Now do I need to build Firefox to modify the package list or just download the source? What do I do next?

Comment 14

[Zorro]

Attachment: This file present in python/mozboot/mozboot installs setuptools as a part of bootstrap build

This is only for Debian distros.

Comment 15

[Gregory Szorc [:gps]]

Comment on attachment 8434551 [details] [diff] [review]
This file present in python/mozboot/mozboot installs setuptools as a part of bootstrap build

Could you please upload the attachment as a diff (as opposed to the full file)? Please see https://developer.mozilla.org/en-US/docs/Mercurial_FAQ#How_can_I_generate_a_patch_for_somebody_else_to_check-in_for_me.3F

Comment 16

[Mike Hommey [:glandium]]

virtualenv is supposed to be installing pip and setuptools. They are shipped as support files in python/virtualenv/virtualenv_support. I'd rather look why those aren't picked before adding dependencies on system libraries that shouldn't be used in the first place.

Comment 17

[Mike Hommey [:glandium]]

Forget comment 16, I missed the fact that this was not errors with the build system. That being said, if installing mercurial doesn't pull the right packages on a distro, bugs should be filed on that distro.

Comment 18

[Zorro]

Attachment: Imports setuptools in Debian.py

Comment 19

[Zorro]

Sorry for late reply I was bed ridden

Comment 20

[Gregory Szorc [:gps]]

Comment on attachment 8438593 [details] [diff] [review]
Imports setuptools in Debian.py

Review of attachment 8438593 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good. I'll check it in for you.

Comment 21

[Gregory Szorc [:gps]]

Attachment: Install setuptools as part of bootstrap

Updated patch ready for checkin (added DONTBUILD to commit message).

Comment 22

[Gregory Szorc [:gps]]

Trees are closed. Adding checkin-needed. This is NPOTB, so no try run should be required.

Comment 23

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e931f9e105f0

Comment 24

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/e931f9e105f0