848985 - [WhistlePig] enable offline compression so lessc and stuff can be run on admin node

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P3)

Description

[Peter Bengtsson [:peterbe]]

A pull request has already been created for this here:
https://github.com/rtucker-mozilla/WhistlePig/pull/3

Without this, the poor apache worker needs to run the compression stuff in runtime on the web head(s).

Comment 1

[Rob Tucker [:rtucker]]

I've pulled and merged, however still not able to get compression to work correctly.

Going to try a few different angles here.

Thanks so much for the pull request.

Comment 2

[Rob Tucker [:rtucker]]

I'm getting this error on the admin node:

Error: An error occured during rendering /data/genericrhel6-dev/src/whistlepig-dev.allizom.org/WhistlePig/whistlepig/base/templates/whistlepig_base.html: Error while importing lxml: No module named BeautifulSoup

The vendor.pth appears to be setup correctly and contains:
src/django-compressor

[[e-mail]1 WhistlePig]# find ./ -name 'beautifulsoup*'
./vendor/src/django-compressor/compressor/parser/beautifulsoup.pyc
./vendor/src/django-compressor/compressor/parser/beautifulsoup.py

peterbe,
Any additional thoughts?

Comment 3

[Peter Bengtsson [:peterbe]]

I've heard of this but never been able to reproduce it locally. I suspect there's something to do with lxml because otherwise it would be more predictable. 

See:

(whistlepig)peterbe@mpb:~/dev/MOZILLA/WhistlePig (enable-django-jingo-offline-compressor %)$ ./manage.py shell
/Users/peterbe/dev/MOZILLA/WhistlePig/vendor/lib/python/django/conf/__init__.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
  warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)
Python 2.6.6 (r266:84292, Dec  5 2011, 09:38:23)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import BeautifulSoup
Traceback (most recent call last):
  File "<console>", line 1, in <module>
ImportError: No module named BeautifulSoup
>>> import lxml
Traceback (most recent call last):
  File "<console>", line 1, in <module>
ImportError: No module named lxml

(whistlepig)peterbe@mpb:~/dev/MOZILLA/WhistlePig (enable-django-jingo-offline-compressor %)$ pip freeze
Jinja2==2.5.5
MySQL-python==1.2.3c1
py-bcrypt==0.2
wsgiref==0.1.2

Clearly it works on my virtualenv without BeautifulSoup or lxml. 
It *could* be that if you have lxml installed, django-compressor then also requires BeautifulSoup.

I don't really know how django-compressor chooses which parsers to use.

Comment 4

[Rob Tucker [:rtucker]]

I'm gonna mark this as complete. I just created a static file for the css. We're not going to get a benefit of compressing our 1 css asset much anyway.

peterbe,
Really appreciate the effort here. Perhaps some day it will make sense to worry about compressing for whistlepig, I just don't think this warrants any more of our time.

Comment 5

[Peter Bengtsson [:peterbe]]

That's cop out! :)

If you have problems on your local system, just install BeautifulSoup then. It might not be a problem on the dev server.

Comment 6

[Rob Tucker [:rtucker]]

It works fine on my local system as well as my developer system. It does not however work on the developer system that is available to others, where others are testing etc.

I know it's a cop out, we just don't have the cycles to justify the time to figure it out right now.

Really do appreciate all your help though and at some point I do intend on revisiting this.