1702819 - ./mach taskgraph target --diff fails on Windows

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, defect, P3)

Description

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

./mach taskgraph target --target-kind test --fast --diff .~1 -p project=autoland

fails for me with the following output

$ ./mach taskgraph target --target-kind test --fast --diff .~1 -p project=autoland
 0:05.23 Generating target_task_set @ 6f3a9007793c
 0:05.48 Loading graph configuration.
 0:05.55 Starting new HTTPS connection (1): firefox-ci-tc.services.mozilla.com
 0:06.28 Loading parameters from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OEjv1dQOShKhjExsHLWbxA/artifacts/public/parameters.yml
 0:06.28 Starting new HTTPS connection (1): firefox-ci-tc.services.mozilla.com
 0:06.41 Starting new HTTPS connection (1): firefoxci.taskcluster-artifacts.net
 0:07.16 Loading kinds
 0:07.16 Limiting kinds to test and dependencies
 0:07.27 Generating full task set
 0:07.57 Generated 83 tasks for kind fetch
 0:07.58 Generated 16 tasks for kind packages
 0:07.87 Generated 144 tasks for kind toolchain
 0:07.88 Generated 42 tasks for kind docker-image
 0:07.89 Generated 6 tasks for kind instrumented-build
 0:07.90 Generated 6 tasks for kind generate-profile
 0:08.46 Generated 112 tasks for kind build
 0:08.52 Generated 1 tasks for kind build-fat-aar
 0:08.52 Generated 0 tasks for kind build-notarization-part-1
 0:08.53 Generated 0 tasks for kind build-notarization-poller
 0:08.56 Generated 28 tasks for kind build-signing
 0:17.89 Generated 10766 tasks for kind test
 0:17.91 Generating full task graph
 0:17.94 Full task graph contains 11204 tasks and 57247 dependencies
 0:20.32 Generating target task set
 0:20.39 Filter filter_target_tasks pruned 9303 tasks (1901 remain)
 0:23.03 Generating target_task_set @ 82681327d7b8
 0:23.16 Loading graph configuration.
 0:23.34 Loading parameters from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OEjv1dQOShKhjExsHLWbxA/artifacts/public/parameters.yml
 0:23.93 Loading kinds
 0:23.93 Limiting kinds to test and dependencies
 0:24.06 Generating full task set
 0:24.25 Generated 83 tasks for kind fetch
 0:24.26 Generated 16 tasks for kind packages
 0:24.58 Generated 144 tasks for kind toolchain
 0:24.60 Generated 42 tasks for kind docker-image
 0:24.61 Generated 6 tasks for kind instrumented-build
 0:24.62 Generated 6 tasks for kind generate-profile
 0:25.14 Generated 112 tasks for kind build
 0:25.18 Generated 1 tasks for kind build-fat-aar
 0:25.19 Generated 0 tasks for kind build-notarization-part-1
 0:25.19 Generated 0 tasks for kind build-notarization-poller
 0:25.25 Generated 28 tasks for kind build-signing
 0:35.03 Generated 10766 tasks for kind test
 0:35.05 Generating full task graph
 0:35.10 Full task graph contains 11204 tasks and 57247 dependencies
 0:37.40 Generating target task set
 0:37.47 Filter filter_target_tasks pruned 9299 tasks (1905 remain)
Error running mach:

    ['taskgraph', 'target', '--target-kind', 'test', '--fast', '--diff', '.~1', '-p', 'project=autoland']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file taskgraph| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

TypeError: __init__() got an unexpected keyword argument 'capture_output'

  File "c:\projects\mozilla\mozilla-unified\taskcluster/mach_commands.py", line 186, in taskgraph_target
    return self.show_taskgraph("target_task_set", options)
  File "c:\projects\mozilla\mozilla-unified\taskcluster/mach_commands.py", line 582, in show_taskgraph
    universal_newlines=True,
  File "c:\mozilla-build\python3\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "c:\projects\mozilla\mozilla-unified\third_party/python/sentry-sdk\sentry_sdk\integrations\stdlib.py", line 190, in sentry_patched_popen_init
    rv = old_popen_init(self, *a, **kw)  # type: ignore

Sentry event ID: a8c8b02da1c1458792ffa917b17fdd76
Sentry is attempting to send 0 pending error messages
Waiting up to 2 seconds
Press Ctrl-Break to quit

Based on the above I suspect this is due to the diff tool in mozilla build not handling all the flags passed.

Comment 1

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

My original suspicion appears incorrect, it looks like it's rather the capture_output argument to the subprocess run call is not supported until python 3.7, and my mozilla build has 3.6.

Comment 2

[Andrew Halberstadt [:ahal]]

Thanks for filing!

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1702819 - [taskgraph] Support Python 3.6 with ./mach taskgraph <sub> --diff, r?#taskgraph-reviewers!

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d336fab8385
[taskgraph] Support Python 3.6 with ./mach taskgraph <sub> --diff, r=taskgraph-reviewers,jmaher

Comment 5

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/0d336fab8385