Closed Bug 1397722 Opened 2 years ago Closed Last year

[tryselect] Consider excluding code coverage runs from default set of tasks

Categories

(Firefox Build System :: Try, enhancement, P3)

enhancement

Tracking

(firefox63 fixed)

RESOLVED FIXED
mozilla63
Tracking Status
firefox63 --- fixed

People

(Reporter: ahal, Assigned: ahal)

References

Details

Attachments

(1 file)

Currently pgo and code coverage builds and tests are part of the target set, so show up when running |mach try fuzzy|. I think we should consider pre-filtering these tasks out so people have to run |mach try fuzzy --full| to be able to select them.

I think most people don't particularly care about running on these platforms, but they often get run anyway when people use <ctrl-a>. Personally almost all of my queries start with "!pgo !cov" to filter them out.

For anyone who *does* want to run pgo or code coverage, this just means they'd need to pass in --full. If desired, they can even alias this as a default.

Maybe instead of filtering these out by name, we could filter out tasks that run periodically on central or something.
When we were using try option syntax we kept jsdcov/ccov out of '-p all -u all' pushes through the 'run-on-projects'. Is this the same sort problem? I ask because I'm not sure what |mach try fuzzy| is equivalent to in try option syntax.

You're right though that most people never need to have ccov/jsdcov scheduled by default (and it's a waste of resources).
Flags: needinfo?(ahalberstadt)
Yeah this is sort of the same idea.

When people hit <ctrl-a> to select all jobs, it's kinda like using try -u all (but not really). But e.g, if I typed 'mochitest-browser, then hit <ctrl-a><enter>, that would schedule code coverage jobs. This bug is proposing to require --full to do that.
Flags: needinfo?(ahalberstadt)
Priority: -- → P3
Component: General → Try
Product: Testing → Firefox Build System
Depends on: 1473048
:ahal, do you think we could append all queries with '!pgo !ccov' to prevent those from running when '--full' is supplied or would that cause issues in the queries?
Flags: needinfo?(ahal)
(In reply to Greg Mierzwinski [:sparky] from comment #3)
> :ahal, do you think we could append all queries with '!pgo !ccov' to prevent
> those from running when '--full' is supplied or would that cause issues in
> the queries?

I mean when '--full' is not supplied.
It's definitely possible. I think I'd prefer the solution outlined in comment 0 though. While it may be a bit hacky, I think it'll lead to a better user experience (plus it'll be generic, not just for |mach try fuzzy|).

Actually, if we fix both this bug generically and fix bug 1400295 (which I'd like to do sometime soonish) then bug 1473048 would magically be fixed for free.
Flags: needinfo?(ahal)
Oh, sounds good! That would be nice. :)
Minor correction:
Bug 1473048 would be fixed for free when using `mach try`. It wouldn't be fixed for people using `hg push try`. So it's probably still worth fixing.
Assignee: nobody → ahal
Status: NEW → ASSIGNED
Marco, do you agree that 'ccov tasks should be excluded from the default |mach try fuzzy| task list (and require --full to select them)?
Flags: needinfo?(mcastelluccio)
I think I'm going to ignore pgo for now, as whether or not those should be included in the default task set is a bit less cut and dry.
Summary: [tryselect] Consider excluding pgo and code coverage runs from default set of tasks → [tryselect] Consider excluding code coverage runs from default set of tasks
Usually people don't mean to schedule code coverage tasks on try. But e.g |mach
try fuzzy -q "'mochitest"|, will cause them to be scheduled as a side effect.
This results in wasted resources and superfluous data in ActiveData.

This patch makes it so you need to explicitly pass --full to select ccov tasks
(even though they're technically part of the target task graph).
Comment on attachment 9002905 [details]
Bug 1397722 - [tryselect] Filter 'ccov' tasks out of the target task set (now they must be selected with --full), r=sparky

Greg Mierzwinski [:sparky] has approved the revision.
Attachment #9002905 - Flags: review+
(In reply to Andrew Halberstadt [:ahal] from comment #8)
> Marco, do you agree that 'ccov tasks should be excluded from the default
> |mach try fuzzy| task list (and require --full to select them)?

Sounds good to me. I don't like the fact that "mach try fuzzy" + ctrl-a could still cause backouts because what we run on mozilla-central would not be the same as what ctrl-a selects, but I guess it's OK in order to save resources.
Flags: needinfo?(mcastelluccio)
(In reply to Marco Castelluccio [:marco] (PTO until August 24) from comment #12)
> Sounds good to me. I don't like the fact that "mach try fuzzy" + ctrl-a
> could still cause backouts because what we run on mozilla-central would not
> be the same as what ctrl-a selects, but I guess it's OK in order to save
> resources.

Yeah, that's why I've been punting on this bug for so long. But on the other hand we want to discourage the use of a blank 'ctrl-a' anyway as it's wasteful. So I guess this is better.

Maybe I could add a "--no-filter" flag so you could still get the exact set of tasks needed to be 100% sure of not being backed out (might be useful in extreme situations). I guess in practice this isn't going to be a problem, and if it is we can tackle it in a follow-up.
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/dcedddab6d89
[tryselect] Filter 'ccov' tasks out of the target task set (now they must be selected with --full), r=sparky
https://hg.mozilla.org/mozilla-central/rev/dcedddab6d89
Status: ASSIGNED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
You need to log in before you can comment on or make changes to this bug.