894950 - Decide best way to handle psutil

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[John Hopkins (:jhopkins)]

psutil is a compiled Python package which wants to be built against the same version of compiler that the python binary was built with.

In our new win64 setup (in testing), we've currently got VS10 installed, no VS9 installed, and the Python binary was built against VS9.  This configuration fails to build psutil since it's looking for VS9's vcvarsall.bat and doesn't find it.

Some options:

1) rebuild python 2.7 using VS10.  

2) install VS9.

3) hack around the VS9 dependency and build psutil with VS10.  The compile itself passes but I'm not sure whether this will cause other problems.

4) other?

Comment 1

[Justin Wood (:Callek)]

imo option 2 is a lot more work for little gain and should be off the table

Comment 2

[(not currently active) Ted Mielczarek]

(In reply to John Hopkins (:jhopkins) from comment #0)
> 3) hack around the VS9 dependency and build psutil with VS10.  The compile
> itself passes but I'm not sure whether this will cause other problems.

I'm not sure this is safe, psutil has binary components that will be loaded in the Python process, and I don't know how the memory management works.

Comment 3

[Gregory Szorc [:gps]]

If the easiest route is for Release Engineering to deploy pre-built binary packages to the machine (via Puppet or similar), then I think downstream consumers can work around this by first checking for whether psutil is available on the system before attempting to install it in the virtualenv.

That being said, that feels like a one-off and there is the potential for other binary Python packages in the future. It would be nice to devise a solution that scales.

Comment 4

[John Hopkins (:jhopkins)]

Thanks for all your input - deploying a pre-built binary package sound like the best route.  I built psutil on a w64-ix-slaveXX machine and asked for it to be deployed here https://bugzilla.mozilla.org/show_bug.cgi?id=781277#c36