we conditionally run jsreftest and jittest, unfortunately all the time

RESOLVED FIXED in mozilla55

Status

Taskcluster
General
RESOLVED FIXED
9 months ago
9 months ago

People

(Reporter: jmaher, Assigned: jmaher)

Tracking

unspecified
mozilla55

Details

Attachments

(1 attachment)

(Assignee)

Description

9 months ago
for cron tasks like code coverage, and things like mozilla-central/aurora/beta, we will only run jsreftests andjittest when we change js/src/*||js/public/* code.  I would think that we only want to exclude these on inbound/try/autoland.

What options do we have to do branch support or other conditional support for a when clause?
(Assignee)

Comment 1

9 months ago
:dustin, you might have some ideas here :)
Flags: needinfo?(dustin)
I have a few thoughts that might help get you going in the right direction here.

First, it's possible on a per-branch basis to enable or disable optimization of tasks in the target task set. If these particular tasks are in the target task sets for cron tasks, and that parameter is enabled, that might do the trick.

Second, "files changed" really only applies to a push.  In the case of a cron task, there's no push involved -- there's a revision, but the pushlog-id parameter is -1 [1].  So there is no list of files that have changed, so consulting that at all doesn't make much sense.  It would be reasonable for that optimization to always return False, None when params['pushlog-id'] is -1.

[1] https://tools.taskcluster.net/task-inspector/#eGYefOsXSj-_eF9S3btdUg/
Flags: needinfo?(dustin)
(Assignee)

Comment 3

9 months ago
My attempt at interpreting this:
diff --git a/taskcluster/taskgraph/optimize.py b/taskcluster/taskgraph/optimize.py
--- a/taskcluster/taskgraph/optimize.py
+++ b/taskcluster/taskgraph/optimize.py
@@ -223,14 +223,18 @@ def opt_seta(task, params):
         # Always optimize away low-value tasks
         return True, None
     else:
         return False, None


 @optimization('files-changed')
 def opt_files_changed(task, params, file_patterns):
+    # pushlog_id == -1 - this is the case when run from a cron.yml job
+    if params.get('pushlog_id') == -1:
+        return True, None
+
     changed = files_changed.check(params, file_patterns)
     if not changed:
         logger.debug('no files found matching a pattern in `when.files-changed` for ' +
                      task.label)
         return True, None
     return False, None

:dustin, does that look right?
Yep!
(Assignee)

Comment 5

9 months ago
Created attachment 8851054 [details] [diff] [review]
when running from cron.yml do not optimize out tests related to files_changed

if we should look at doing this for other optimizations, let me know as well.
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8851054 - Flags: review?(dustin)
Attachment #8851054 - Flags: review?(dustin) → review+

Comment 6

9 months ago
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/65de1ea0c3ef
do not used files-changed to conditionally exclude tasks when run from a cron job. r=dustin

Comment 7

9 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/65de1ea0c3ef
Status: ASSIGNED → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
You need to log in before you can comment on or make changes to this bug.