Closed Bug 2084622 Opened 28 days ago Closed 1 day ago

Derive a base revision for git pushes that create a branch so tgdiff runs on code review bot pushes

Categories

(Firefox Build System :: Task Configuration, task)

task

Tracking

(firefox162 fixed)

RESOLVED FIXED
162 Branch
Tracking Status
firefox162 --- fixed

People

(Reporter: nhadji, Assigned: ahal)

References

(Blocks 1 open bug)

Details

Attachments

(2 external files, 1 obsolete file)

Part of the Mercurial -> Git CI migration (blocks gecko2github). Follow-up to bug 2046767.

When a git push creates a branch, GitHub reports the push's before revision as the null SHA and the decision task records it as base_rev. With bug 2046767 the source-test-taskgraph-diff (tgdiff) task skips the diff with a warning in that case instead of failing. That is fine for a developer branch, but it loses the diff where it matters most.

The code review bot (bug 2048352) pushes every Phabricator revision to its own branch on mozilla-firefox/firefox-crb with a force push:

  • the first diff of a revision creates the branch, so base_rev is the null SHA and no taskgraph diff is produced;
  • later diffs of the same revision force-push the branch, so base_rev is the previous diff's tip and the task compares the new patch to the old one instead of to main.

Today, on Mercurial try, the base of the bot's push is the mozilla-central changeset the stack sits on, and the "Tasks were added or removed" review comment comes from that diff. The em-onpush diff reviewbot posted on D335510 is a recent example of it catching a real problem.

Possible fixes:

  1. In get_base_git_rev (decision.py), when base_rev is the null SHA, or whenever the push went to a branch that is not the default one, fetch the default branch of base_repository and use git merge-base head_rev <default branch> as base_git_rev.
  2. Let the bot hand over the base it applied the stack on, since it knows the main commit it cloned.

Verification: a push creating a branch on mozilla-releng/staging-firefox, then a force push to the same branch, should both produce a taskgraph diff against main. D332894 documents the current behaviour: push 1 skips the diff, push 2 diffs against the previous tip.

Assignee: nobody → ahal
Status: NEW → ASSIGNED

Comment on attachment 9683715 [details]
Bug 2084622 - Version bump taskcluster-taskgraph to v24.8.0, r?#taskgraph-reviewers!

Revision D343152 was moved to bug 2092851. Setting attachment 9683715 [details] to obsolete.

Attachment #9683715 - Attachment is obsolete: true

This allows tasks in Gecko to use Taskgraph 24.9.0's new ability to
fetch secrets in run-task (assuming they are using the upstream
run-task).

Taskgraph's run-task replaces <PREFIX>_BASE_REV with the merge base found
through GitHub's compare API and fetches it. Pass the base revision and ref to
the task through GECKO_BASE_REV and GECKO_BASE_REF, and read the base at
runtime so the diff runs against the corrected value instead of the one baked
into the task definition. This makes tgdiff work for pushes that create a
branch or force push one, such as the code review bot's.

Support Taskgraph's secrets key in job descriptions by reusing its
use_secrets transform, and use it on git pushes to give run-task a GitHub
token, so its compare requests aren't rate limited as anonymous ones.

Pushed by ahalberstadt@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/62b733f1ceae https://hg.mozilla.org/integration/autoland/rev/50348879a0a8 Call upstream Taskgraph's `use_secrets` transforms, r=taskgraph-reviewers,bhearsum https://github.com/mozilla-firefox/firefox/commit/90b99095608a https://hg.mozilla.org/integration/autoland/rev/68e78809c2a1 Run source-test-taskgraph-diff on git pushes against the derived base revision, r=taskgraph-reviewers,bhearsum
Status: ASSIGNED → RESOLVED
Closed: 1 day ago
Resolution: --- → FIXED
Target Milestone: --- → 162 Branch
Blocks: 2046767
No longer depends on: 2046767
No longer blocks: 2046767
Depends on: 2046767
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: