1007437 - We need to pull in merge commit refs for gaia-try runs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

Right now, gaia-try is testing against the tip of the contributors branch.  This is neat, but it isn't exactly what we need.  Github does internal merges to determine whether the PR is able to be merged.  The result of that merge is what we want to test against.

That merge commit is available to the try server hook I wrote, so sending it to Gaia-try is simple.  The reason for filing this bug is that a default clone of gaia will not have the commits for the pull requests.


Example: PR#18853

The API (https://api.github.com/repos/mozilla-b2g/gaia/pulls/18853) gives use the following important commits:

mozilla-b2g:master's sha: d62f9f37ed325df7726dd999f0d49f80be16c18b
RickyChien:new-mocha's sha: 70c4c6204768d7db8d7cefef52d54ac77db1583b
merge_commit_sha: 1ffc5fa424179b46ab14ff0124fc59f4782c3784

There are two methods for getting access to pull request heads.  One is to follow https://help.github.com/articles/checking-out-pull-requests-locally, but that's inefficient as we have nearly 20k pull requests.

Below is what's needed to fetch the reference and required objects as well as checking out a working copy.

$ git fetch origin +refs/pull/18853/merge:pr18853
From https://github.com/mozilla-b2g/gaia
 * [new ref]         refs/pull/18853/merge -> pr18853
$ git show pr18853
commit 1ffc5fa424179b46ab14ff0124fc59f4782c3784
Merge: 03cf633 70c4c62
Author: RickyChien <[e-mail]>
Date:   Wed May 7 22:20:03 2014 +0000

    Merge 70c4c6204768d7db8d7cefef52d54ac77db1583b into 03cf6332a410f5b8cda3bc6edbdc56a8d14166af

$ git branch
  master
  pr18853
$ git checkout pr18853

I'm going to start including the pull request number in the 'git' key of the gaia.json file, such that we have:

{
  "git": {
    "git_revision": "abc123",
    "remote": "remote_url",
    "github_pr_number": 11111
  }
}

I've attached a patch that I can't test.


A further optimization that could be made, since we only care about a single reference and are clobbering, is to do a shallow fetch of just that one reference.

That can be done with something like:
mkdir dest && cd dest && git init && git fetch --depth 1 $remote +refs/pull/$pr_num/merge:local && git checkout $revision

Comment 1

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: mozharness-pr-fetch.diff

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

Please let me know as soon as this change happens so I can update the task runner

Comment 3

[Aki Sasaki (not active)]

Comment on attachment 8419089 [details] [diff] [review]
mozharness-pr-fetch.diff

(In reply to John Ford [:jhford] -- please use 'needinfo?' instead of a CC from comment #2)
> Please let me know as soon as this change happens so I can update the task
> runner

You can update the gaia.json at any point, right?  If you start populating github_pr_number before we use it, it'll just be ignored.

Comment 5

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to Aki Sasaki [:aki] from comment #3)
> You can update the gaia.json at any point, right?  If you start populating
> github_pr_number before we use it, it'll just be ignored.

sorta.  The issue is that we are no longer cloning the PR authors remote, so we either hard code the mozharness script to be mozilla-b2g/gaia (bad), add a new real_remote key (terrible) to gaia.json or we time the landings to be roughly the same time (not ideal).

Given that the results we're getting now aren't totally useful, I'm tempted to just alter the hook and submit the new data until this change happens.

Aki, how do we go about landing and deploying this change?  I haven't tested it at all....  If it's totally cool from your end and it's just an hg push, I can do that and land/deploy my changes at the same time if that makes things easier.

Comment 6

[Aki Sasaki (not active)]

Comment on attachment 8419089 [details] [diff] [review]
mozharness-pr-fetch.diff

https://hg.mozilla.org/build/mozharness/rev/8ffa41cbae7a

Ok, then afaict this patch should be able to land in mozharness without the gaia.json changing at all, since it's a noop unless github_pr_number exists.

I've landed and merged.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

Oops, this patch that had zero testing had a bug, I hope that's ok.

I've landed a fix, but the meat of the change is that the remote reference name and the local branch need to be seperated by a colon instead of by a space.

https://hg.mozilla.org/build/mozharness/rev/3fd934f61d17

This change is basically changing a space into a colon

Comment 8

[Chris Cooper [:coop] (he/him)]

Merged to production.

Comment 9

[John Ford [:jhford] CET/CEST Berlin Time]

This is all done.