1752677 - Make it easy to recycle existing tasks with `mach try`

Status: RESOLVED FIXED

(Developer Infrastructure :: Try, enhancement)

Description

[Nick Alexander :nalexander [he/him]]

This ticket tracks teaching some mach try invocation(s) to recycle existing tasks from other places. This is item 2) of https://bugzilla.mozilla.org/show_bug.cgi?id=1561423#c9.

My immediate use cases have been:

• MSIX repackaging
• single-locale repacks

Doing this doesn't require changes to test archives (namely Bug 1628981), which we should also do, but is a longer road.

Comment 1

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1752677 - Allow to recycle existing tasks with `mach try fuzzy. r?ahal

This approach is a path of least resistance.

Comment 2

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

One use case (which may not happen often) I have is to reuse the cpp artifacts from one push in a second push (I changed cpp earlier but now my patch is just iterating on java). I think the value is that my try pushes would take 5min rather than 20min (the test I'm trying to fix with java code takes ~3 minutes and java builds are similarly quick but I spend most of my time waiting for the native build).

Comment 3

[Andrew Halberstadt [:ahal]]

I ended up implementing this but forgot this bug existed (found it from suggested bugs when going to file one for my patch). I also ended up converting everything to the v2 schema as a prerequisite here in bug 1608026.

Comment 4

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1752677 - Add a task config to set the 'existing_tasks' parameter, r?#releng-reviewers!

This can be used from fuzzy or chooser selectors. For example:

./mach try fuzzy -E

This queries the Treeherder API to find your latest try push, then any tasks
that exist on that push, but aren't explicitly selected this time, will be
replaced with those older tasks.

For example, if iterating on a test, this can be used to avoid needing to run
the build task over and over again. Care must be taken not to use this flag
when your changes might actually make a material difference in the dependency
tasks.

Comment 5

[Andrew Halberstadt [:ahal]]

Dang, Nick needinfo'ed me about this 5 years ago:
https://bugzilla.mozilla.org/show_bug.cgi?id=1561423#c0

I feel bad this took so long ><

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1ddc1733cf76
Add a task config to set the 'existing_tasks' parameter, r=releng-reviewers,bhearsum,jcristau

Comment 8

[Cristian Tuns]

Backed out for causing python failures

• Backout link
• Push with failures
• Failure Log
• Failure line: subprocess.CalledProcessError: Command '['/Users/cltbld/.mozbuild/srcdirs/gecko-0c3536da8a55/_virtualenvs/python-test/bin/python', '-m', 'pip', 'install', 'pytest-mock==3.12.0']' returned non-zero exit status 1.

Comment 9

[Andrew Halberstadt [:ahal]]

Looks like pytest-mock needs to be on the internal pypi mirror. I've added it now, I'll re-land once my try push comes back green.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b9172a6f32d4
Add a task config to set the 'existing_tasks' parameter, r=releng-reviewers,bhearsum,jcristau

Comment 11

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/b9172a6f32d4