[tryselect] Implement |mach try chooser|

RESOLVED FIXED in Firefox -esr60

Status

enhancement
RESOLVED FIXED
8 months ago
3 months ago

People

(Reporter: ahal, Assigned: ahal)

Tracking

(Blocks 1 bug)

unspecified
mozilla66
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr60 fixed, firefox66 fixed)

Details

Attachments

(3 attachments)

Assignee

Description

8 months ago
While |mach try fuzzy| is great for a lot of things, one area it is worse that try syntax is that you need to *know* what you are looking for. With try syntax, you can open up the trychooser web page and see at a glance what the options are, whereas there is no such luxury with fuzzy.

However, trychooser isn't great because it needs to be updated manually and is always horribly out of date. If you rely on it, you'll inevitably miss running tasks that you maybe should have been running.

To get the best of both worlds, I'm proposing a new selector |mach try chooser|. This tool would do a few things:

1. Build the task graph (like fuzzy does)

2. Dynamically compute the tasks that can be selected. We can do this by looking at various keys in the task graph like "kind", "suite", "tags", etc. Some hardcoding will be required to manage the different types of tasks here, but once a type is supported, new tasks will be added automatically added.

3. Run a flask app that displays a "trychooser" like page in the browser based on the computed tasks.

4. Provide a "Push" button which will schedule all selected options (no more need for copy/pasting syntax strings).


One thing to note, is that this "trychooser" page would not need to generate any try syntax on the backend. We could filter the actual taskgraph in real time as the user checks/unchecks items, we could even provide a preview of all the task labels the combination of checkboxes would schedule (a la try fuzzy).
Assignee

Updated

7 months ago
Assignee: nobody → ahal
Status: NEW → ASSIGNED
Assignee

Updated

7 months ago
Depends on: 1509048
This sounds like a really cool idea! The set of tasks we run has gotten so large that I think there's a lot of room for experimenting with new ways to visualize and filter them. For my usecases I often want to run either all types of builds for one platform, or builds of a certain type on all platforms. I'd love to have an interface similar to trychooser like:

Build Type
==========
[ ] opt [ ] debug [ ] pgo [ ] asan [ ] valgrind [ ] rusttest ...

Platform
========
[ ] win32 [ ] win64 [ ] linux [ ] linux64 [ ] mac [ ] android ...

Test Suites
===========
[ ] mochitest [ ] xpcshell [ ] reftest ...

Preferably with "Select All" / "Clear All" buttons for each section. I suspect people would probably want a way to filter test suites down by chunk as well. I'm not sure how you'd best present that. If we do chunking differently per-platform / build type it might also be nice to expose the "select tests by path" functionality that `mach try syntax` has. I can see a use for "I'd like to run test_whatever in debug builds on these platforms."
Assignee

Comment 2

7 months ago
I have a working prototype already :). Hoping to polish it up a bit more in time for the demo sessions at the all hands.

My current idea is to use (nicely styled) multiselects instead of checkboxes, so shift/ctrl+clicking will work. There's also a live preview window (similar to fuzzy), which updates with the task labels that would get selected as you click.

I haven't implemented chunking support yet, but my current idea is to have a text input inside the multiselect <option> tags. It would work similar to printer page setups (e.g "1-3,5"). The maximum number would be bounded by whatever platform has the most chunks for that suite.

I'm not sure if mixing build types and build variants is going to be possible using the way the prototype is currently set up, but I think we could add a new section:

Build Type
==========
[] opt [] debug

Build Variant
=============
[] asan [] pgo [] ccov ...

Platform
========
[] win32 [] win64 ...
Assignee

Updated

7 months ago
See Also: → 559506
Assignee

Comment 3

6 months ago
After chatting with Armen, I think it might be better to try and re-write this on react. However since I have things mostly working already, I'm tempted to get what I have landed first (without advertising) so people can start experimenting and providing feedback.

Worth noting that I've never used react before, so this might not be a small undertaking for me.
I agree that landing what you currently have as a first cut and making improvements from there makes sense. (Even if you wind up rewriting it in react.)
Assignee

Comment 5

6 months ago
This will be used by |mach try chooser| to help pull 'android-stuff' tasks
(which aren't really builds) apart from the main build tasks.
Assignee

Comment 6

6 months ago
Usage:

  $ ./mach try chooser

Will start a local flask server and server a "trychooser-like" page
that is dynamically generated from the taskgraph.

Depends on D14902
Assignee

Comment 7

6 months ago
Hey Armen, I know you are heading on your leave soon and this is a large complicated patch. So if you don't have time feel free to unassign yourself and I'll find someone else to review. That being said, I'm not expecting a super thorough review here.

We talked about using React for this at the all hands, and I do still agree that's the way to go. But since my old hacky way is already working, and I'm not sure how long a React re-write will take, I'd like to get what I have landed first.

I also plan on adding docs and tests before landing, but I won't have that done before you leave, so I'll do it in a new commit and get someone else to review that in the new year.

My apologies for making you look at this mess :)
Attachment #9032265 - Attachment description: Bug 1499822 - [tryselect] Implement |mach try chooser|, r?armenzg → Bug 1499822 - [tryselect] Implement |mach try chooser|

Comment 9

5 months ago
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f4bd2ef9b729
[taskgraph] Add a tag to help differentiate 'android-stuff' from builds, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/9a330195eca6
[lint] Enable codespell on tools/tryselect, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/beb286aeae33
[tryselect] Implement |mach try chooser| r=sclements

Backed out for breaking gecko decision task.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=success%2Cpending%2Crunning%2Ctestfailed%2Cbusted%2Cexception&classifiedState=unclassified&searchStr=gecko%2Cdecision%2Ctask%2Copt%2Cgecko%2Cdecision%2Ctask%2C%28d%29&selectedJob=220690676

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=220690676&repo=autoland&lineNumber=3956

Backout: https://hg.mozilla.org/integration/autoland/rev/5ecacfcd6e65fe5a1dee88b40ef00ca20c0282e8

From IRC:

tomprince> apavel|sheriffduty: Should backout https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=9a330195eca6 for that

<apavel|sheriffduty> ok, however the gecko task after that push is green. it failes only on the merge

<tomprince> Yeah, it is only going to fail when android-stuff tasks are run.

<apavel|sheriffduty> tomprince: ok, thank you for looking

<tomprince> But it will fail everytime they are scheduled to run.

Flags: needinfo?(ahal)

Comment 11

5 months ago
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f8bb48d6a644
[taskgraph] Add a tag to help differentiate 'android-stuff' from builds, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/c2b4cd16c551
[lint] Enable codespell on tools/tryselect, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/31e16c2d9429
[tryselect] Implement |mach try chooser| r=sclements

Comment 12

5 months ago
bugherder
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
Assignee

Updated

5 months ago
Flags: needinfo?(ahal)
Assignee

Updated

5 months ago
Blocks: 1519449
You need to log in before you can comment on or make changes to this bug.