908065 - [SUMO][-dev,-stage,-prod] Verify (install?) pyopenssl version 0.13

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Community Platform, task, P4)

Description

[Ricky Rosario [:rrosario, :r1cky]]

Back in bug 830496, :cturra did some magic for us to have pyopenssl v 0.13 on our servers. It works fine on the admin nodes, our cron jobs have been happy. But we just added code that depends on it from the webservers and we are getting the errors that seem to indicate the wrong version is being picked up from the web processes:

https://errormill.mozilla.org/support/support/group/87877/

Help?

Comment 1

[Ricky Rosario [:rrosario, :r1cky]]

I logged on to stage and the right version seems to be there. Maybe the wsgi processes aren't picking it up for some reason? Different PYTHONPATH?

[[e-mail]1 ~]$ PYTHONPATH=/usr/local/lib64/python2.6/site-packages/ python
Python 2.6.6 (r266:84292, Oct 12 2012, 14:23:48)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import OpenSSL
>>> OpenSSL.__version__
'0.13'
>>> exit()

Comment 2

[Daniel Maher [:phrawzty]]

I can confirm that all of the nodes in the Support cluster have the "pyOpenSSLmoz-0.13-1.x86_64" package, so no problem on that front.

I'm trying to get myself set up with an errormill login now.  Any help you might be able to provide would be appreciated. :)

Comment 3

[Daniel Maher [:phrawzty]]

What in that error report indicates that the wrong SSL lib is being used (I believe you - I'm just curious).  Also, is there a dev (or you) that's awake in European time that I can work with to debug this issue ?

Comment 4

[Daniel Maher [:phrawzty]]

07:02:31 < willkg> phrawzty: mmm... i'm looking at the oauth2client code that's erroring out and it's using OpenSSL.crypto and it's lacking "sign".
07:02:41 < willkg> phrawzty: i'm guessing that's why r1cky suspects pyopenssl.
07:03:29 < willkg> i've got 0.13 and it has a sign.
07:04:06 < willkg> i don't know the details, but i'm a quick study. i can work through this with you now.

Comment 5

[Daniel Maher [:phrawzty]]

Unfortunately, it would appear that the WSGIPythonPath directive was not being respected, meaning that /usr/local/lib64/python2.6/site-packages wasn't being added to sys.path as far as the webapp was concerned.

I commented that line out, and then appended a python-path option directly to the WSGIDaemonProcess directive with the appropriate paths.  This had the desired effect.

I have committed this change to Puppet for dev *only*.  Please let me know when/if you want to have this change made in stage and/or prod.

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Daniel Maher [:phrawzty] from comment #5)
> I have committed this change to Puppet for dev *only*.  Please let me know
> when/if you want to have this change made in stage and/or prod.

Can we do stage now?

Comment 7

[Daniel Maher [:phrawzty]]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #6)
> Can we do stage now?

Committed to Puppet and live now (stage).

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

Stage looks good so far! Let's let it sit until Monday to do prod, being friday and mfbt and all that.

Comment 9

[Daniel Maher [:phrawzty]]

Prod has been committed.

Comment 10

[Daniel Maher [:phrawzty]]

r1cky sez "it looks good; resolve it and I'll verify"

Comment 11

[Ricky Rosario [:rrosario, :r1cky]]

working great! thanks :phrawzty!