1608937 - Upgrade NSS win2012r2 workers to python3

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: Windows OS, enhancement)

Description

[Kevin Jacobs [:kjacobs]]

Setuptools v45.0.0 removes support for Python2, which our NSS windows workers are running.

This causes build bustages, such as https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=4d04b0c1735e6dfaec42c3b03b46711095591460:

test-env/Scripts/pip install --upgrade setuptools
+ test-env/Scripts/pip install --upgrade setuptools
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in c:\users\task_1578946610\gyp\test-env\lib\site-packages (45.0.0)
ERROR: Package 'setuptools' requires a different Python: 2.7.15 not in '>=3.5'

We've worked around this temporarily by installing setuptools<45.0.0, but the workers should be upgraded to python3 so as to support the latest stable versions of build dependencies.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

These workers are now managed by relops, I think in OpenCloudConfig..

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

hey pete: we don't have nss builders in occ. can you point me at the powershell scripts that boostrap these. i'd like to just take a shot at modifying the existing bootstrap to include python 3 or check if the mozlla-build version on these already contains python 3.

Comment 3

[Pete Moore [:pmoore][:pete]]

I think this is the latest version:

https://github.com/taskcluster/generic-worker/blob/93b4f68fd7f91f3fc1d33e3853b3d89e1bd5f695/worker_types/nss-win2012r2/bootstrap.ps1

It looks like currently the generic-worker config sections of https://firefox-ci-tc.services.mozilla.com/worker-manager/nss-1%2Fwin2012r2 do not contain this link in a workerTypeMetaData section.

@Tom, do you know why the nss-1/win2012r2 worker pool definition (and possibly others, I haven't checked) has been trimmed of the generic-worker config setting workerTypeMetaData? This is super-useful information that was was previously available for people inspecting tasks, to know how the host was set up that runs the task. ci-admin needs to generate this. Thanks!

Comment 4

[Tom Prince [:tomprince]]

That information is tied to the worker-image, not a pool. If we want to include this type of information, we'd need to refactor the handling of worker-image.yml to able to include this kind of metadata. In the mean time, this info can live as comments in the appropriate stanza in worker-images.yml.

Comment 5

[ablew]

believed complete.