942275 - Provide an entry point for 3rd party modules implementing mach commands

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, enhancement)

Description

[Andrew Halberstadt [:ahal]]

In bug 794509 a method which searches sys.path for command providers in 'mach/commands/*.py' files was implemented. This method doesn't lend itself very well to python packages. In python packages, I don't believe it is possible to install an entire directory to the path (e.g no way to automatically setup a mach_commands.py under a 'mach/commands' subdirectory on the path).

One solution is to implement a .machrc where command modules/providers can be registered. Though this wouldn't be auto-magical.

Another is to create a plugin to setuptools' entry_points. I know this was discussed in bug 794509 and rejected because that's not how in-tree packages are set up.. but it would still be useful for out of tree modules.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Patch 1.0 - add ability to load commands from entry points

Not sure if this is quite there yet, but it's simple. An example of how a third party module would hook in with this is:
https://github.com/ahal/b2g-commands

Under this system, all the user has to do is 'pip install b2g-commands' and everything works.

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8337100 [details] [diff] [review]
Patch 1.0 - add ability to load commands from entry points

Review of attachment 8337100 [details] [diff] [review]:
-----------------------------------------------------------------

Nice! I always wanted support for entry points in mach!

I have plans to make the Python virtualenv in m-c an actual virtualenv with installed Python packages. So doing things more properly like this will be good for the long-term.

FWIW, I have an internal monologue every time the subject of arbitrary "mach_commands.py" files come up. I like the idea in theory. However, I despise the thought of some developers having access to some commands that others don't know about. Productivity (via automatic command discovery and availability) should be the default. I don't want a world of haves and have-nots.

Comment 3

[Andrew Halberstadt [:ahal]]

(In reply to Gregory Szorc [:gps] from comment #2)
> FWIW, I have an internal monologue every time the subject of arbitrary
> "mach_commands.py" files come up. I like the idea in theory. However, I
> despise the thought of some developers having access to some commands that
> others don't know about. Productivity (via automatic command discovery and
> availability) should be the default. I don't want a world of haves and
> have-nots.

I agree. Though a world of haves and have-nots is better than a world of nothing but have-nots (e.g B2G). Ideally I'd like all the commands to land in the repo, but barring that, an installable module is the next best thing. I could also picture a set of mach commands that are highly specialized to a single team, and might not make sense being available to everyone by default (though I see the counter argument to this).

Comment 4

[Gregory Szorc [:gps]]

I guess we can cross the bridge of out-of-tree commands if it becomes a problem. Worst case, we could probably maintain an in-tree mapping of mach commands to PyPI packages and have mach automatically download and install PyPI packages when said command is executed. Makes things harder for offline use. But, we could probably devise some clever caching inside ~/.mozbuild.

Comment 5

[Andrew Halberstadt [:ahal]]

Attachment: Patch 2.0 - add entry point support to mach

Same as last patch except with tests and updated README

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 8341267 [details] [diff] [review]
Patch 2.0 - add entry point support to mach

Review of attachment 8341267 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mach/mach/test/providers/basic.py
@@ +1,1 @@
> +from __future__ import unicode_literals

Need license.

::: python/mach/mach/test/test_entry_point.py
@@ +8,5 @@
> +import sys
> +
> +from mach.base import MachError
> +from mach.test.common import TestBase
> +from mock import patch

I think this is the first time we used mock in mach. We'll need to add mock to setup.py.

Comment 7

[Andrew Halberstadt [:ahal]]

Pushed with comments fixed:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5cb5fdb72e55

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Backed out for mach bustage.
https://hg.mozilla.org/integration/mozilla-inbound/rev/80f74b6e3644

https://tbpl.mozilla.org/php/getParsedLog.php?id=31382355&tree=Mozilla-Inbound

Comment 9

[Andrew Halberstadt [:ahal]]

Attachment: Patch 2.1 - ignore entry point if setuptools not present

Comment 10

[Andrew Halberstadt [:ahal]]

Folded the fix:
https://hg.mozilla.org/integration/mozilla-inbound/rev/be257c6d6d24

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/be257c6d6d24