1962211 - include git-commit information in json-pushes

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, enhancement)

Description

[Julien Cristau [:jcristau]]

For pushes that are synced from git it could be useful for consumers to know about the corresponding git commit.

https://bugzilla.mozilla.org/show_bug.cgi?id=1961341#c3

Comment 2

[:glob ✱]

From a quick look a the code, it looks like the vcsreplicator message that is sent to Kafka is also sent verbatim over Pulse. We'd either need to add the field at the origin or augment the message just prior to sending to Pulse; the former appears to be more in alignment with the existing structure.

Comment 3

[:glob ✱]

Looks like the pulse message only contains a reference to a pushlog: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmo/notifications.html#changegroup-1

…
    "pushlog_pushes": [{
      "time": 14609750810,
      "pushid": 120040,
      "user": "[e-mail]",
      "push_json_url": "https://hg.mozilla.org/try/json-pushes?version=2&startID=120039&endID=120040",
      "push_full_json_url": "https://hg.mozilla.org/try/json-pushes?version=2&full=1&startID=120039&endID=120040"
    }]
…

---

Looking at the json-pushes endpoint (for a different push than the one referenced above):

https://hg.mozilla.org/mozilla-central/json-pushes?changeset=3d00a96b7fa7d3ec8c70c2caf86a1d280eca3f9b&version=2

{
  "lastpushid": 42808,
  "pushes": {
    "42804": {
      "changesets": [
        "bb02b56cc922bf57e39d3d066c0d96c6ebcacdad",
        "8fa5028d6fa298fc65eb0e7387ac022da18da4b3",
        "b2e1cf02b34fbcbd6a30457e0015eca986733501",
        "55b55cd8347175f8de3a061d451f2e3d95184123",
        "daeea2584ae525cb0919e181b4464ceb5b5e3a3d",
        "a1228bfde2135b583671e235c62e7f3be7cdb129",
        "3d00a96b7fa7d3ec8c70c2caf86a1d280eca3f9b",
        "ed46f3ba59ca4c2e831cd81ba131b886163447e0"
      ],
      "date": 1747646989,
      "user": "[e-mail]"
    }
  }
}

So instead of updating the pulse payload, we'll have to update the json-pushes endpoint. We should probably add git_changesets, making the above example:

{
  "lastpushid": 42808,
  "pushes": {
    "42804": {
      "changesets": [
        "bb02b56cc922bf57e39d3d066c0d96c6ebcacdad",
        "8fa5028d6fa298fc65eb0e7387ac022da18da4b3",
        "b2e1cf02b34fbcbd6a30457e0015eca986733501",
        "55b55cd8347175f8de3a061d451f2e3d95184123",
        "daeea2584ae525cb0919e181b4464ceb5b5e3a3d",
        "a1228bfde2135b583671e235c62e7f3be7cdb129",
        "3d00a96b7fa7d3ec8c70c2caf86a1d280eca3f9b",
        "ed46f3ba59ca4c2e831cd81ba131b886163447e0"
      ],
      "date": 1747646989,
      "git_changesets": [
        "7cd4da127c35bec3f3bd1484c0d7692bbbd28ed9",
        "dcb661da9bd272be5732fc5083ce0983fa8cb4e9",
        "501d8ea86c4572ae16e53fb67dcff96c0dffa73b",
        "60a32508c5062980ec80e2426a491db459456649",
        "ac53f457a47204751c85998254a0b14d192bd3d1",
        "4ab3915300d2a48eea2f2605885c3f968c08b21e",
        "466683b7a615744696a5796fdc99b8a622468a1c",
        "9e7131f6df9112288f17666bddbe901dbc3527db",
      ],
      "user": "[e-mail]"
    }
  }
}

---

Similar story for full=1 response:

https://hg.mozilla.org/mozilla-central/json-pushes?changeset=3d00a96b7fa7d3ec8c70c2caf86a1d280eca3f9b&version=2&full=1

{
  "lastpushid": 42808,
  "pushes": {
    "42804": {
      "changesets": [
        {
          "author": "longsonr <[e-mail]>",
          "branch": "default",
          "desc": "Bug 1967155 - Cache parsed href attribute URI r=emilio\n\nDifferential Revision: https://phabricator.services.mozilla.com/D249915\n",
          "files": [
            "dom/svg/SVGFEImageElement.cpp",
            "dom/svg/SVGFEImageElement.h",
            "dom/svg/SVGImageElement.cpp",
            "dom/svg/SVGImageElement.h"
          ],
          "node": "bb02b56cc922bf57e39d3d066c0d96c6ebcacdad",
          "parents": [
            "8e9456975478acd5db2ddf71579916b8871259ff"
          ],
          "tags": []
        },
…

Rather than duplicating the entire changeset array, we should instead add git_node and git_parents:

{
  "lastpushid": 42808,
  "pushes": {
    "42804": {
      "changesets": [
        {
          "author": "longsonr <[e-mail]>",
          "branch": "default",
          "desc": "Bug 1967155 - Cache parsed href attribute URI r=emilio\n\nDifferential Revision: https://phabricator.services.mozilla.com/D249915\n",
          "files": [
            "dom/svg/SVGFEImageElement.cpp",
            "dom/svg/SVGFEImageElement.h",
            "dom/svg/SVGImageElement.cpp",
            "dom/svg/SVGImageElement.h"
          ],
          "git_node": "7cd4da127c35bec3f3bd1484c0d7692bbbd28ed9",
          "git_parents": [
            "8f74e36f7a0730e06c50927f2738654aeb55985b"
          ],
          "node": "bb02b56cc922bf57e39d3d066c0d96c6ebcacdad",
          "parents": [
            "8e9456975478acd5db2ddf71579916b8871259ff"
          ],
          "tags": []
        },
…

Comment 4

[:glob ✱]

hgext.pushlog.feed.py#pushes_worker appears to be where updating is required (in addition to tests and docs, of course)

Comment 5

[Connor Sheehan [:sheehan]]

Attachment: pushlog: display Git commit information from `json-pushes` endpoint (Bug 1962211) r?glob,jcristau

Add the git_commit extra to the json-pushes endpoint as
git_node and git_parents fields. Add a test showing the
expected output with and without ?full=1.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/dc35dc177281
pushlog: display Git commit information from json-pushes endpoint r=glob,jcristau