1604074 - mach vcs-setup fails to install the hg histedit or evolve extensions

Status: RESOLVED FIXED

(Developer Services :: Mercurial: configwizard, defect)

Description

[Chris Peterson [:cpeterson]]

mach vcs-setup does to install the hg histedit or evolve extensions even though I selected y:

Would you like to enable these history editing extensions (Yn)?  y
...
Would you like to enable the evolve extension? (Yn)  y

In fact, mach vcs-setup will actually remove histedit and evolve and all my other hg extensions from my ~/.hgrc:

 [extensions]
-histedit =
-progress =
-purge =
-rebase =
-shelve =
-firefoxtree = /Users/chris/.mozbuild/version-control-tools/hgext/firefoxtree
-clang-format = /Users/chris/.mozbuild/version-control-tools/hgext/clang-format
-js-format = /Users/chris/.mozbuild/version-control-tools/hgext/js-format
-push-to-try = /Users/chris/.mozbuild/version-control-tools/hgext/push-to-try
+blackbox =

However, mach vcs-setup did successfully download evolve to ~/.mozbuild/evolve. I don't know if this next problem is a related or a different bug, but if I rerun mach vcs-setup, the script throws an exception because the ~/.mozbuild/evolve directory already exists:

Would you like to enable the evolve extension? (Yn)  y
** unknown exception encountered, please report by visiting
** https://mercurial-scm.org/wiki/BugTracker
** Python 3.7.5 (default, Nov  1 2019, 02:16:23) [Clang 11.0.0 (clang-1100.0.33.8)]
** Mercurial Distributed SCM (version 5.2)
** Extensions loaded: firefoxtree, clang-format, js-format, push-to-try, configwizard
Traceback (most recent call last):
  File "/Users/chris/.mozbuild/version-control-tools/hgext/configwizard/__init__.py", line 986, in _checkevolve
    hg.clone(ui, {}, remote_evolve_path, branch=(b'stable',), dest=local_evolve_path)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/hg.py", line 659, in clone
    raise error.Abort(_(b"destination '%s' is not empty") % dest)
mercurial.error.Abort: destination '/Users/chris/.mozbuild/evolve' is not empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/hg", line 36, in <module>
    dispatch.run()
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 111, in run
    status = dispatch(req)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 250, in dispatch
    ret = _runcatch(req) or 0
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 424, in _runcatch
    return _callcatch(ui, _runcatchfunc)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 433, in _callcatch
    return scmutil.callcatch(ui, func)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/scmutil.py", line 177, in callcatch
    return func()
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 414, in _runcatchfunc
    return _dispatch(req)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 1174, in _dispatch
    lui, repo, cmd, fullargs, ui, options, d, cmdpats, cmdoptions
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 862, in runcommand
    ret = _runcommand(ui, options, cmd, d)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 1185, in _runcommand
    return cmdfunc()
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/dispatch.py", line 1171, in <lambda>
    d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/util.py", line 1843, in check
    return func(*args, **kwargs)
  File "/Users/chris/.mozbuild/version-control-tools/hgext/configwizard/__init__.py", line 555, in configwizard
    _checkevolve(ui, cw, hgversion)
  File "/Users/chris/.mozbuild/version-control-tools/hgext/configwizard/__init__.py", line 992, in _checkevolve
    ui.write(str(hg_err))
  File "/usr/local/Cellar/mercurial/5.2_1/lib/python3.7/site-packages/mercurial/ui.py", line 1100, in write
    msg = b''.join(args)
TypeError: sequence item 0: expected a bytes-like object, str found

Comment 2

[Connor Sheehan [:sheehan]]

This is strange - and obviously totally unexpected.

There were some recent changes made to configwizard around how evolve is managed that may be causing this problem. I also notice you are using Python 3.7 to install Mercurial, and Py3 support has only landed recently. You may be one of the first to switch to Py3 for Mercurial and uncover bugs. It's possible these two things are interacting.

I'll need to dig into this a bit more to say for certain.

Comment 3

[Chris Peterson [:cpeterson]]

I'm using macOS 10.15 Catalina with zsh, if it matters. I experienced this problem with the histedit and evolve extensions a couple months ago. I don't recall how I fixed the problem then or know why it returned just yesterday.

Comment 4

[Michael Froman [:mjf]]

I can confirm this behavior on a pristine macOS install of Mojave (10.14), so this is not Catalina specific. I used the stock python (2.7.17) from Mojave to run the bootstrap.py script from here[1] as documented in MDN for macOS builds[2].

[1] https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py
[2] https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Mac_OS_X_Prerequisites

Comment 5

[Michael Froman [:mjf]]

Interestingly, what is installed in ~/.mozbuild/evolve is the mercurial repository for evolve. It doesn't seem to try installing the extension (or fails to do so without surfacing the error) because running 'pip list' doesn't show that it is installed.

Comment 6

[Connor Sheehan [:sheehan]]

(In reply to Michael Froman [:mjf] from comment #5)

Interestingly, what is installed in ~/.mozbuild/evolve is the mercurial repository for evolve. It doesn't seem to try installing the extension (or fails to do so without surfacing the error) because running 'pip list' doesn't show that it is installed.

Mercurial extensions don't need to be installed into the same Python interpreter as Mercurial to work, you can specify a path to the extension in your hgrc and it will load. That's what the setup wizard is doing currently. I have an upcoming patch that changes this behaviour slightly, FWIW.

Comment 7

[Michael Froman [:mjf]]

Additional info: if you delete ~/.mozbuild/evolve between runs of ./mach vcs-setup, it alternates between:

[extensions]
shelve =
firefoxtree = /Users/chris/.mozbuild/version-control-tools/hgext/firefoxtree
clang-format = /Users/chris/.mozbuild/version-control-tools/hgext/clang-format
js-format = /Users/chris/.mozbuild/version-control-tools/hgext/js-format
push-to-try = /Users/chris/.mozbuild/version-control-tools/hgext/push-to-try

and

[extensions]
blackbox =

Comment 8

[Zeid [:zeid]]

Attachment: extensions: fix bug that clears extensions (Bug 1604074) r=sheehan,mhentges

Do not land yet, WIP.

This change fixes a check that erased cw.c['extensions'] if b'extensions'is not present incw.c`. Main thing to note is that str and bytes can form
distinct keys in a dictionary, even though their string representation is the
same.

Comment 9

[Zeid [:zeid]]

@Michael @Chris FYI the fix pushed also fixes the (separate) bug which caused the exception to be thrown when trying to print/surface an error if evolve failed to be enabled for whatever reason.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/cfa24b4fdc96
extensions: fix bug that clears extensions r=sheehan