Closed Bug 1253651 Opened 5 years ago Closed 5 years ago

|mach artifact install| fails when using git with "No module named mercurial"

Categories

(Firefox Build System :: General, defect)

47 Branch
defect
Not set
normal

Tracking

(firefox47 affected, firefox48 fixed)

VERIFIED FIXED
mozilla48
Tracking Status
firefox47 --- affected
firefox48 --- fixed

People

(Reporter: whimboo, Assigned: chmanchester)

References

Details

Attachments

(1 file)

After bug 1234913 got fixed I tried to use artifact build with my git repository by using cinnabar. Sadly it failed with the following error:

Traceback (most recent call last):
 0:24.94   File "/mozilla/code/git-cinnabar/git-cinnabar", line 9, in <module>
 0:24.94     from cinnabar.githg import (
 0:24.94   File "/mozilla/code/gecko/build/mach_bootstrap.py", line 470, in __call__
 0:24.94     module = self._original_import(name, globals, locals, fromlist, level)
 0:24.94   File "/mozilla/code/git-cinnabar/cinnabar/githg.py", line 38, in <module>
 0:24.94     from mercurial import mdiff
 0:24.94   File "/mozilla/code/gecko/build/mach_bootstrap.py", line 470, in __call__
 0:24.94     module = self._original_import(name, globals, locals, fromlist, level)
 0:24.94 ImportError: No module named mercurial

I did a look into the obj/_virtualenv/ folder and noticed that mercurial has not been installed for that venv which mach seems to use. Manually installing mercurial via pip fixed the problem for me. 

So it looks like that we miss to install mercurial for git users?
For reference, https://bugzilla.mozilla.org/show_bug.cgi?id=1234913#c19 and below is the relevant discussion from the original bug. This has worked for me and a few others without installing mercurial in the mozbuild virtualenv. We should figure out what's going on here.
I see that the first step for setting up git-cinnabar is to install mercurial, which sounds like it solved Henrik's problem. I'll be sure to call this out in the mdn docs when updating.

Closing for now.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WORKSFORME
Mercurial is installed on my system but the virtualenv which is affected here doesn't seem to use the packages from dist-packages.
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
I wasn't able to reproduce because I, too, had mercurial installed in my tree virtualenv. Nick's comment from https://bugzilla.mozilla.org/show_bug.cgi?id=1234913#c19 is still the exact problem we're having (git cinnabar picks up the virtualenv python).

So we can go back to installing mercurial into the mach virtualenv, or we can invoke cinnabar with sys.executable, which gets set before activating the virtualenv.
Assignee: nobody → cmanchester
https://reviewboard.mozilla.org/r/38323/#review34819

::: python/mozbuild/mozbuild/artifacts.py:829
(Diff revision 1)
> -            self._cinnabar, 'git2hg'
> +            sys.executable, self._cinnabar, 'git2hg'

Don't call cinnabar with python directly. Use "git", "cinnabar". And there's no reason to do the lookup on your own, let git do it.
https://reviewboard.mozilla.org/r/38323/#review34819

> Don't call cinnabar with python directly. Use "git", "cinnabar". And there's no reason to do the lookup on your own, let git do it.

Specifically, there is no guarantee that the command will stay in python forever.
https://reviewboard.mozilla.org/r/38323/#review34819

> Specifically, there is no guarantee that the command will stay in python forever.

I tried this, it fails in the same fashion. When the virtualenv is active, /usr/bin/env python2.7 is the virtualenv python.
Comment on attachment 8726919 [details]
MozReview Request: Bug 1253651 - Undo changes to PATH made by activating virtualenv in mach artifact. r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/38323/diff/1-2/
Attachment #8726919 - Attachment description: MozReview Request: Bug 1253651 - Use system Python rather than in-tree virtualenv Python when invoking cinnabar to avoid errors import mercurial libaries. r=gps → MozReview Request: Bug 1253651 - Install mercurial into the in-tree virtualenv to allow invoking git cinnabar in a subprocess. r=gps
https://reviewboard.mozilla.org/r/38323/#review34863

::: python/mozbuild/mozbuild/mach_commands.py:1495
(Diff revision 2)
> +        self.virtualenv_manager.install_pip_package('mercurial==3.7.1')

I'd rather save os.environ['PATH '] before running _activate_virtualenv() and restore it after.
Comment on attachment 8726919 [details]
MozReview Request: Bug 1253651 - Undo changes to PATH made by activating virtualenv in mach artifact. r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/38323/diff/2-3/
Attachment #8726919 - Attachment description: MozReview Request: Bug 1253651 - Install mercurial into the in-tree virtualenv to allow invoking git cinnabar in a subprocess. r=gps → MozReview Request: Bug 1253651 - Undo changes to PATH made by activating virtualenv in mach artifact. r=glandium
Attachment #8726919 - Flags: review?(gps) → review?(mh+mozilla)
https://reviewboard.mozilla.org/r/38323/#review34863

> I'd rather save os.environ['PATH '] before running _activate_virtualenv() and restore it after.

Works for me.
Attachment #8726919 - Flags: review?(mh+mozilla)
Attachment #8726919 - Flags: review?(mh+mozilla)
Comment on attachment 8726919 [details]
MozReview Request: Bug 1253651 - Undo changes to PATH made by activating virtualenv in mach artifact. r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/38323/diff/3-4/
https://reviewboard.mozilla.org/r/38323/#review35199

::: python/mozbuild/mozbuild/mach_commands.py:1497
(Diff revision 4)
> +        os.environ['PATH'] = original_path

If you are going to hack up the environment, you should have a try..finally block to unhack it.
Comment on attachment 8726919 [details]
MozReview Request: Bug 1253651 - Undo changes to PATH made by activating virtualenv in mach artifact. r=glandium

https://reviewboard.mozilla.org/r/38323/#review35283
Attachment #8726919 - Flags: review?(mh+mozilla) → review+
https://reviewboard.mozilla.org/r/38323/#review35199

> If you are going to hack up the environment, you should have a try..finally block to unhack it.

It's virtualenv_activate that is hacking the environment. This is *restoring* it.
https://hg.mozilla.org/mozilla-central/rev/1fc581388ffd
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
Works like a charm! Thanks for the fix.
Status: RESOLVED → VERIFIED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.