Jobs with "file_patterns" run when they shouldn't on the autoland branch

VERIFIED FIXED

Status

VERIFIED FIXED
2 years ago
2 years ago

People

(Reporter: ahal, Assigned: gps)

Tracking

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

(Reporter)

Description

2 years ago
If you look at autoland, you'll notice that often jobs with 'file_patterns' are being run on pushes they shouldn't. For example, the flake8 linter is run even when the push doesn't modify any python files.

I believe this is because the scheduling logic looks at the files touched by all draft ancestor commits. Autoland appears to be non-publishing, which means commits from previous pushes will help trigger the "file_patterns" clause.

So either the scheduler should be smarter and start looking at the pushlog, or autoland should be publishing. I have a hunch the former would be the preferred solution.
(Assignee)

Comment 1

2 years ago
The decision task queries an API on hg.mozilla.org to retrieve info about "relevant" changesets for a push. Changing its behavior on hg.mozilla.org to treat autoland specially is not out of the question.

We /could/ implement this functionality in-tree if we wanted. At the time I implemented it, the decision code was in a much wonkier state and doing it server side was simpler. I'm not opposed to changing it.
(Reporter)

Comment 2

2 years ago
I guess ideally it would be in-tree, but I can't imagine this needing to change *too* much after special treatment in autoland is fixed up. So if leaving it server side is easier that would be fine with me too.
My impression is that json-automationrelevance is pretty much designed to feed info to decision tasks, so it seems most reasonable to just modify that to return the appropriately-shaped results.  I assume moving this to the decision task would require more HTTP requests, and thus be a bit slower..
(Assignee)

Comment 4

2 years ago
Moving json-automationrelevance behavior into the decision task requires the decision task to perform VCS commands. It /may/ require the decision task to query pushlog data, which either requires installing a Mercurial extension to sync the pushlog data or potentially multiple HTTP requests to hg.mozilla.org's JSON APIs. At the point you are making a HTTP request to hg.mozilla.org, you might as well expose a single endpoint so the client gets all the info it needs in one go so things are fast.

I'm not super keen on automation syncing down pushlog data because the support for syncing pushlog data is either a) designed for a straight 1:1 replication system b) in the mozext extension, which is not production quality. On top of that, it adds overhead.
(Assignee)

Updated

2 years ago
Duplicate of this bug: 1315478
(Assignee)

Comment 6

2 years ago
I'll fix this.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Comment hidden (mozreview-request)

Comment 8

2 years ago
mozreview-review
Comment on attachment 8808426 [details]
hgmo: add config option to control and change default behavior of automationrelevant (bug 1297162);

https://reviewboard.mozilla.org/r/91236/#review91576
Attachment #8808426 - Flags: review?(smacleod) → review+

Comment 9

2 years ago
Pushed by gszorc@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/8a6f5516bb64
hgmo: add config option to control and change default behavior of automationrelevant ; r=smacleod
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
(Assignee)

Updated

2 years ago
Component: Task Configuration → Mercurial: hg.mozilla.org
Product: Taskcluster → Developer Services
QA Contact: hwine
(Assignee)

Comment 10

2 years ago
This is deployed. I spot tested that queries to the autoland repo (such as https://hg.mozilla.org/integration/autoland/json-automationrelevance/05db990f8787e761f12418ec2da0f7d4a7cc04fe) are now returning only changesets in the push itself. And queries to Try (whose hgrc I hacked up to use the new config option) are returning draft changesets from previous pushes (e.g. https://hg.mozilla.org/try/json-automationrelevance/586472f6f477).
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.