1816793 - tooltool.py syntax error on safe_extract (python 2)

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Geoff Brown [:gbrown]]

mcornmesser brought this to my attention, which certainly looks like a regression from bug 1806899.

Fetching...
Executing: ['c:/mozilla-build/python/python.exe', 'c:/builds/tooltool.py', '-c', 'c:/builds/tooltool_cache', '--url', 'http://taskcluster/tooltool.mozilla-releng.net/', '--overwrite', '-m', 'nss/automation/taskcluster/windows/releng.manifest', 'fetch', '-c', 'C:\\builds\\tooltool_cache']
  File "c:/builds/tooltool.py", line 973
    def safe_extract(tar, path=".", members=None, *, numeric_owner=False):
                                                   ^
SyntaxError: invalid syntax
retry: Calling <function run_with_timeout at 0x0000000002CCF908> with args: (['c:/mozilla-build/python/python.exe', 'c:/builds/tooltool.py', '-c', 'c:/builds/tooltool_cache', '--url', 'http://taskcluster/tooltool.mozilla-releng.net/', '--overwrite', '-m', 'nss/automation/taskcluster/windows/releng.manifest', 'fetch', '-c', 'C:\\builds\\tooltool_cache'], 300, None, None, False, True), kwargs: {}, attempt #2
Executing: ['c:/mozilla-build/python/python.exe', 'c:/builds/tooltool.py', '-c', 'c:/builds/tooltool_cache', '--url', 'http://taskcluster/tooltool.mozilla-releng.net/', '--overwrite', '-m', 'nss/automation/taskcluster/windows/releng.manifest', 'fetch', '-c', 'C:\\builds\\tooltool_cache']
  File "c:/builds/tooltool.py", line 973
    def safe_extract(tar, path=".", members=None, *, numeric_owner=False):

from task: https://firefox-ci-tc.services.mozilla.com/tasks/Og5tTxkMSxWaMO1497jizQ

Comment 1

[Geoff Brown [:gbrown]]

First thought: I notice "c:/mozilla-build/python/python.exe" ... is that python2? Would this run better with python3?

Comment 2

[Mark Cornmesser [:markco]]

(In reply to Geoff Brown [:gbrown] from comment #1)

First thought: I notice "c:/mozilla-build/python/python.exe" ... is that python2? Would this run better with python3?

I think these tasks have a general dependency on pyhton2.

It is interesting though, when I last ran these tasks on the Azure workers about 2 weeks ago they had no issue with tooltool, https://firefox-ci-tc.services.mozilla.com/tasks/h9iX-bIDQFCDSO0OEPgWJQ . Which is also the case with them running on the current AWS workers, https://firefox-ci-tc.services.mozilla.com/tasks/GcHxN5DVTYiKn-U_QS8_1g . Both using Python2. The Azure image itself had not changed between the passing and failing, so it may just be a matter of adding something to the Azure image to work with recent changes. I just don't have an idea of what that could be.

Comment 3

[Julien Cristau [:jcristau]]

https://github.com/mozilla-platform-ops/ronin_puppet/blob/cloud_windows/modules/win_mozilla_build/manifests/tooltool.pp#L21 grabs tooltool from the github master branch so it'll presumably have picked up the update from https://github.com/mozilla-releng/tooltool/pull/1066 last week, which uses python3-only syntax.

Comment 4

[Mark Cornmesser [:markco]]

Easy fix then. I will pin to an older version of the tooltool.py for these workers.

Comment 5

[Geoff Brown [:gbrown]]

Maybe that's best? I'm sure I could make that change in a py2 compatible way, but all of the tooltool.py tests would still be py3 only -- we'd probably regress py2 again in short order.

It would be nice if you could switch to py3 (maybe just in tooltool_wrapper.sh?) and use the latest tooltool.py -- if py3 is at all available in that environment.

Comment 6

[Mark Cornmesser [:markco]]

Pinning it worked: https://firefox-ci-tc.services.mozilla.com/tasks/groups/ccHtQpKLTlCvaPh7r_O3SQ

I believe that the NSS repo is in line to modernized including removing the python2 dependencies. For now I am going to move forward with pinning so that it can be migrated out of AWS and away from bit rotted configuration management.

Comment 7

[Geoff Brown [:gbrown]]

Sounds good. Happy to revisit if this is troublesome in the future.