1256676 - Webrtc testing needs some new python packages on the testers

Status: RESOLVED WORKSFORME

(Release Engineering :: General, task)

Description

[Byron Campen [:bwc]]

Bug 1231981 is going to introduce some python dependencies on the testers, here are the import statements for the stuff that isn't part of python's standard library:

import ipaddr
import passlib.utils # for saslprep
from twisted.internet import reactor, protocol
from twisted.internet.task import LoopingCall
from twisted.internet.address import IPv4Address, IPv6Address
import txws

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

This should be in a releng/ateam component, and CC the relevant people.

Comment 2

[Byron Campen [:bwc]]

The copy of twisted that is already at http://puppetagain.pub.build.mozilla.org/data/python/packages/ is probably going to work fine, but ipaddr, passlib, and txws are still going to be necessary.

Comment 3

[Byron Campen [:bwc]]

It looks like txws has a dependency that pypi doesn't know about:

six

Comment 4

[Byron Campen [:bwc]]

Ah, we seem to have six already.

Comment 5

[Byron Campen [:bwc]]

Who needs to be assigned to this?

Comment 6

[Justin Wood (:Callek)]

(In reply to Byron Campen [:bwc] from comment #5)
> Who needs to be assigned to this?

I'm handling this for catlee.

From IRC chat, I think we're done here... I was told to make sure we had `ipaddress` available on internal pypi, and then we also needed the following:

callek@Centaurus:~/tmp2$ for i in *.whl *.gz; do echo $i; done
passlib-1.6.5-py2.py3-none-any.whl
passlib-1.6.5.tar.gz
txWS-0.9.1.tar.gz

(well I wasn't told to do .tar.gz and .whl for passlib, but figured I might as well upload both).

If there are more packages to install, please reopen and n-i me, with their names and I'll get them up.

http://pypi.pub.build.mozilla.org/pub/

Comment 7

[Byron Campen [:bwc]]

It looks like the versions of twisted in our local pypi won't do the trick. 12.3.0 won't install on the windows testers (because that version of twisted has c code it needs to compile), and 10.2.0 doesn't work because it lacks IPv6 support. I see a couple of options:

1. Put a precompiled twisted wheel on our internal pypi.

2. Make sure the windows testers have the necessary software to install the copy of twisted we have.

If this is going to take a while, I can temporarily disable IPv6 in our ICE server, and see if that is enough to un-stick this. It may turn out that more stuff is missing though.

Comment 8

[Justin Wood (:Callek)]

Greg/Dustin, can you provide "us" advice on this bug. I'm not sure I'm particularly comfortable putting things up internally that are neither our own software creations nor on pypi.

However I'm also not certain how to proceed here (I know doing 'builds' of python stuff on windows is harder than it should be in our automation)

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

also, "precompiled" for windows is a slippery thing, as it ends up being precompiled for a specific python version and, I think, for a specific path to that binary.

Tooltool might be a good choice for such things?

Comment 10

[Gregory Szorc [:gps]]

We should aim to use Python wheels on Windows. These target a specific Python/Windows version, so there should be no binary compatibility problem. The Windows wheel at https://pypi.python.org/pypi/Twisted should "just work" (at least on 64 bit Python installs - not sure why they don't make a 32-bit wheel available).

We should avoid building Python 2.7 packages with C extensions on Windows in automation because this will require the VS2008 toolchain because Python 2.7 is built with VS2008. (https://www.microsoft.com/en-ca/download/details.aspx?id=44266 can be installed to provide this toolchain without the rest of Visual Studio 2008.)

The version of pip we have installed in automation may not be new enough to install wheels properly. We should upgrade to the latest version of pip throughout automation to ensure wheels work properly.

Comment 11

[Julien Cristau [:jcristau] (back April 22)]

Presumably this happened at some point.