1129717 - Investigate libffi deploy failure on SUMO

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Michael Cooper [:mythmon]]

Recently, we upgraded some libraries in SUMO, and are now using the python library cffi. It seems to rely on libffi. At least, that is what I was able to make out of it.

This Chief deploy log is where the errors are: http://supportadm.private.phx1.mozilla.com/chief/support.stage/logs/92288e2671ba248d5fe1fd9b700f4c9764f1a732.1423093903

I think that either we need to install libffi on the admin node, the celery nodes, and the webnodes (I think that's everything), or we need to back this change out (which I'd rather not do).

The reason for this change: the Python library pyOpenSSL version 0.13 does not support openssl 1.0.2 (which just came out). pyOpenSSL version 0.14 supports openssl 1.0.2 (as well as older versions). In interest of being compatible with developer systems, we upgraded. 0.13 linked directly to openssl, which was kind of annoying. 0.14 using the python library cffi instead. That's why we are running in to this issue now.

Comment 1

[Philippe M. Chiasson (:gozer)]

libffi-devel installed on that admin node, try deploying again.

Comment 2

[Philippe M. Chiasson (:gozer)]

I re-deployed that commit, and looks like it worked this time around:

http://supportadm.private.phx1.mozilla.com/chief/support.stage/logs/8525cee31cd43219fc424502742751fd4baa3a19.1423249464

Comment 3

[Michael Cooper [:mythmon]]

The commit you deployed doesn't exhibit the problem. In particular, it still has pyOpenSSL==0.13. We rolled back to get the stage site back online. The commit I originally deployed that has the problem is a460dda31d34ddc0dce555495dc7f752f8b7bf02.

I just deployed a460dd again, and it seems to have fixed this problem.