We can't use lxml on dev, stage or prod. We have code that depends on lxml to download transcripts. This is done using PyQuery, which depends on lxml.html which depends on libxslt.so (amongst other things). To reproduce, in the airmozilla virtualenv, try to run:: >>> import lxml The error we get is:: ImportError: libxslt.so.1: cannot open shared object file: No such file or directory
python-lxml is installed on the admin node but not the webheads. lxml is installed by pip but I'm not sure what it requires. Should I install python-lxml at the OS level everywhere?
(In reply to Eric Ziegenhorn :ericz from comment #1) > python-lxml is installed on the admin node but not the webheads. lxml is > installed by pip but I'm not sure what it requires. Should I install > python-lxml at the OS level everywhere? The python-package lxml (https://pypi.python.org/pypi/lxml) is not as powerful as the OS-level package, python-lxml in this case. The OS level one will make sure that a bunch of other important dependencies are available. The right thing to do is to make sure the python-lxml yum package is also installed on the webheads. It used to be. Because the functionality has regressed. Perhaps the first time python-lxml was installed on the webheads someone forgot to update some puppet or something and just did it manually to make the problem go away.
python-lxml installed on python prod, stage and dev clusters via puppet so this won't come up again.
Verified on dev. (at least)
When I found out about this error the first time, I decided to upgrade the python package "lxml". See https://github.com/mozilla/airmozilla/commit/3bc74c3ea977f1dbfe542206436a42f595743e8c I landed that but didn't notice that it failed in Chief. Later you fixed the existing running version in dev (by installing the os level package). Now lxml can't be built in Chief. See latest build attempt: http://genericadm.private.phx1.mozilla.com/chief/air.dev/logs/75c7194490adf3b93dfb64ef26025da9a8e30b90.1464360242 As you can see here, the first time Chief started to have problems is after that above mentioned commit. See https://whatsdeployed.io/s-6pt
Added libxml2-devel and libxslt-devel to all the webheads through puppet, try building it again please.
I *just* pushed something to github. Let's see if it changes things in Chief in 5 min.
Discussed in IRC, solved as the packages also need to be on the admin node.