1190218 - Homebrew not using own Python for Mercurial

Status: REOPENED

(Developer Services :: General, task)

Description

[aleth [:aleth]]

I've seen this a few times recently:

...
remote: Follow the progress of your build on Treeherder:
remote:   https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=cfeeb77f8042
/Users/user/.mozbuild/version-control-tools/pylib/requests/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
recording push at https://bugzilla.mozilla.org/show_bug.cgi?id=1188780

Comment 1

[Gregory Szorc [:gps]]

If you follow the link to https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning, it will say the same thing what I'm about to tell you: Python < 2.7.9 didn't do various parts of SSL/TLS properly and it wasn't secure. To get a secure SSL/TLS implementation on these older versions of Python, you needed to install pyOpenSSL, which provides APIs into OpenSSL needed to make the SSL/TLS context sufficient.

urllib3 introduced this InsecurePlatformWarning to inform people when their Python SSL/TLS "stack" isn't secure. While it is possible to disable this warning, I'd rather we let people know when their Python is insecure.

The workaround is to upgrade to Python >= 2.7.9 (preferred) or to install pyOpenSSL.

If you are on Windows, the latest MozillaBuild has Python 2.7.9. On Mac, Homebrew or MacPorts should be offering Python 2.7.9+. On Linux, you probably want to install your own Python in userland. I recommend https://github.com/yyuu/pyenv.

Comment 2

[aleth [:aleth]]

(In reply to Gregory Szorc [:gps] from comment #1)
> If you follow the link to
> https://urllib3.readthedocs.org/en/latest/security.
> html#insecureplatformwarning, it will say the same thing what I'm about to
> tell you: Python < 2.7.9 didn't do various parts of SSL/TLS properly and it
> wasn't secure. To get a secure SSL/TLS implementation on these older
> versions of Python, you needed to install pyOpenSSL, which provides APIs
> into OpenSSL needed to make the SSL/TLS context sufficient.
> 
> urllib3 introduced this InsecurePlatformWarning to inform people when their
> Python SSL/TLS "stack" isn't secure. While it is possible to disable this
> warning, I'd rather we let people know when their Python is insecure.
> 
> The workaround is to upgrade to Python >= 2.7.9 (preferred) or to install
> pyOpenSSL.

I have Python 2.7.10 installed via homebrew, and a current "mach mercurial-setup" hgrc configuration. 
I just checked and rerunning "mach bootstrap" didn't add any further homebrew packages either.

I certainly have openssl installed (not sure if it was "mach boostrap" that did that), so maybe pyOpenSSL is required to avoid this warning even with a current python?

Comment 3

[aleth [:aleth]]

I should add the current homebrew openssl is 1.0.2d_1.

Comment 4

[Gregory Szorc [:gps]]

Your Mercurial is almost certainly pointing at a Python that isn't Homebrew's. Try reinstalling Mercurial (via Homebrew).

You can verify this assumption by looking at the first line (shebang) of `hg`. I bet it is pointing at /usr/bin/python instead of /usr/local/bin/python.

`hg debuginstall` may also say something useful.

Comment 5

[aleth [:aleth]]

(In reply to Gregory Szorc [:gps] from comment #4)
> Your Mercurial is almost certainly pointing at a Python that isn't
> Homebrew's. Try reinstalling Mercurial (via Homebrew).
> 
> You can verify this assumption by looking at the first line (shebang) of
> `hg`. I bet it is pointing at /usr/bin/python instead of
> /usr/local/bin/python.

Thanks! You're absolutely right.  Maybe "mach mercurial-setup" could warn about this mismatch?

It's strange though, I wonder how this happened, as "which python" correctly points at /usr/local/bin/python.
 
> `hg debuginstall` may also say something useful.
checking Python executable (/usr/bin/python)
checking Python version (2.7.6)
checking Python lib (/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7)...
checking installed modules (/usr/local/Cellar/mercurial/3.5/lib/python2.7/site-packages/mercurial)...
checking templates (/usr/local/Cellar/mercurial/3.5/lib/python2.7/site-packages/mercurial/templates)...

Unfortunately,

brew uninstall --force mercurial
brew install mercurial

did not have any effect.

Comment 6

[aleth [:aleth]]

This issue seems to match
http://stackoverflow.com/questions/28512350/homebrew-path-env-var-is-modified-before-installation-of-formulae-mercurial-wi
and the answer there appears to work.

Looks like it is 1) non-obvious and 2) will affect everyone using homebrew.

Comment 7

[aleth [:aleth]]

(In reply to aleth [:aleth] from comment #6)
> This issue seems to match
> http://stackoverflow.com/questions/28512350/homebrew-path-env-var-is-
> modified-before-installation-of-formulae-mercurial-wi
> and the answer there appears to work.

While it works, it does end up pointing directly at /usr/local/opt/python/bin/python2.7 (as opposed to /usr/local/bin/python). And I suspect the next mercurial update will reset it.

Comment 8

[Gregory Szorc [:gps]]

Homebrew and/or Mercurial's installer is buggy and should be fixed. I'm reopening this to track that, since this will only bite more and more people as time progresses.

Comment 9

[Gregory Szorc [:gps]]

I filed https://github.com/Homebrew/homebrew/issues/42416. Also raised this in #mercurial on Freenode. I may do a little debugging later if the GitHub issue doesn't see an immediate response.