Closed Bug 1262165 Opened 4 years ago Closed 3 years ago

Stop using try syntax as a scheduling API

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: ahal, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [try-option-syntax])

Note: This does *not* mean that try syntax will be deprecated. The intent is to provide a more robust and extensible API to schedule jobs. E.g, a "classic try syntax parser" would use this new API under the hood. The difference being the "try syntax parser" now lives on the local machine rather than on a server somewhere.

Currently the only way to easily schedule jobs on try is by appending a magic string to the commit message. As a result, various tools have started abusing this string to accomplish more and more complex functionality. In parallel, the number of different types of jobs being added have ballooned. Both remembering and building try syntax strings (either manually or automated) can be very complicated and cause much confusion and frustration.

Instead of an arbitrarily complex string, we want either a proper API, or at least a well defined, robust and easily extensible data structure. I believe this should block on getting buildbot jobs scheduled via taskcluster, as it will mean one place to update rather than two. We can support this effort in parallel with the old try parsers for the time being.

There are two basic approaches we could take:

1) Have |mach try| (or something) call the build and scheduler API's directly.

This is the preferred method, as there is no need to pass along scheduling information attached to the push. We would first push the job, then make an API call to get the jobs scheduled.  The downside is dealing with permissions and scopes. They'd have to "log in" somehow. Dustin seems to think this is do-able based on bug 1198341 comment 10, but I'm not really sure what's involved here.

2) Still pass scheduling information along with the push, but in a better format.

This method will work similar to the existing try parser. Except instead of a string, it would be a rich data structure. For example, we might use a list of jobs and a dict of configuration to pass in to the tasks. Or possibly a task graph filter string + configuration.
Dustin informs me that I'm the 19th person to file a bug about better in-tree scheduling :). Feel free to either rename and use this as a tracking bug or dupe if one already exists.
Blocks: 1262571
No longer blocks: awesome-try
Blocks: 1197868
Andrew, now that bug 1248597 has landed, maybe it's time to start thinking more about this?
\o/ That's awesome. Yes we should definitely start thinking about this some more. I'll take a look at everything to see how it works when I have a moment.
Component: Discussion → Task Configuration
No longer depends on: 1243759
Whiteboard: [try-option-syntax]
Andrew, is this still useful as a tracker, or should this be closed in preference to bug 1197868?
You probably meant bug 1262571, but yeah I don't really see any purpose in this bug anymore. In fact it might even already be solved.

Figures I'd file a bunch of trackers and then forget they exist and not use them. Thanks for digging this up.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → INVALID
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.