Add a CI task analyzing source-test issues and reporting them on Phabricator

RESOLVED FIXED in Firefox 67

Status

task
RESOLVED FIXED
4 months ago
4 months ago

People

(Reporter: bastien.abadie, Assigned: bastien)

Tracking

Trunk
mozilla67
Dependency tree / graph

Firefox Tracking Flags

(firefox67 fixed)

Details

Attachments

(2 attachments)

I'm working on moving the Release Management & Quality code review bot from a standalone Taskcluster task to a CI task, in-tree.
The main goal is to rely on the CI infrastructure instead of trying to maintain it on our side.

This patch adds a new task to Mozilla Central CI, that runs after tasks from the source-test kind have analyzed a patch pushed from Try.

It will be triggered by another part of our system: pulselistener that will create a custom push to try on every new Phabricator diff.
Pulselistener will use a custom try_task_config.json file to start the new source-test-report-phabricator task, and provide the Phabricator Diff ID:

{
  "version": 1,
  "tasks": [
    "source-test-report-phabricator"
  ],
  "templates": {
    "env": {
      "ANALYSIS_ID": "PHID-DIFF-abcdef123456"
    }
  }
}

I've also added a new transform and post-optimisation mechanism, called optimized-dependencies in order to be able to run the task after source-test tasks have run, without interfering with the decision task process (we can't depend on all source-test tasks or they will all always run, skipping the optimisations in place).

It provides a list of task labels through the extra payload to the optimisation process, which will only use the remaining ones after the optimisation has removed most of them.

Finally, here is a Treeherder job running this patch + a python typo to trigger flake8 + a sample try_task_config.json.
You can see the resulting source-test-report-phabricator task only depends on 7 source-test tasks, whereas it has all the source-test in optimized-dependencies

Note that the Hooks service in TC can listen for pulse messages. Depending on the complexity of pulselistener, you might be able to implement that as a hook without requiring any additional service!

https://docs.taskcluster.net/docs/reference/core/taskcluster-hooks/docs/firing-hooks

Unfortunately, pulselistener should be renamed apipoller, as it no longer relies on Pulse messages to trigger code reviews: Phabricator does not send any pulse messages !

Assignee

Comment 4

4 months ago

Hello Tom, could you list the changes to make optimized-dependencies a first class component ? Thanks !

Flags: needinfo?(mozilla)

(In reply to Bastien Abadie [:bastien] from comment #4)

Hello Tom, could you list the changes to make optimized-dependencies a first class component ? Thanks !

:dustin outlined them here

Flags: needinfo?(mozilla)
Assignee

Updated

4 months ago
Depends on: 1530376
Attachment #9043880 - Attachment description: Bug 1527895 - Add source-test-report-phabricator task in CI, r=ahal → Bug 1527895 - Add soft-dependencies to taskgraph, r=tomprince

Comment 7

4 months ago
Pushed by babadie@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5caf48a420eb
Add soft-dependencies to taskgraph, r=ahal,marco,tomprince,dustin

Comment 8

4 months ago
Pushed by babadie@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7fdcccd878ad
Add code-review-issues task in CI, r=dustin,marco,tomprince
Attachment #9047021 - Attachment description: Bug 1527895 - Add code-review-issues task in CI, r=ahal → Bug 1527895 - Add code-review-issues task in CI, r=dustin,marco,tomprince

Comment 10

4 months ago
Pushed by babadie@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b58080db50e6
Add code-review-issues task in CI, r=dustin,marco,tomprince

Comment 11

4 months ago
bugherder
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Assignee: nobody → bastien
Assignee

Comment 13

4 months ago

Both patches are now upstream on mozilla-central.

Flags: needinfo?(bastien)
You need to log in before you can comment on or make changes to this bug.