1196805 - Remove json-info web command

Status: NEW

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Gregory Szorc [:gps]]

Now that we deployed the official JSON style to hg.mozilla.org, the "info" web command that we have implemented is arguably redundant with an upstream feature and can be removed.

We currently have a number of requests hitting this endpoint. I'm not sure who is sending traffic. But we'll need to move them to other APIs before we can remove json-info.

Also, json-info supports multiple-node lookup - something Mercurial's APIs don't do as cleanly. We may need to add a better API upstream before all consumers of json-info can move to something official.

The presence of this web command isn't causing any immediate pain. But I prefer to trend towards using upstream features unmodified unless there is a compelling reason (such as Mozilla-specific functionality).

Comment 1

[Ed Morley [:emorley]]

On Github I found:
https://github.com/klahnakoski/TestLog-ETL/blob/master/testlog_etl/imports/hg_mozilla_org.py#L139
https://github.com/darchons/tbpl-heatmap/blob/master/fetch-data.js
(I'd be surprised if the latter were still working though)

The former would likely be responsible for quite a few requests, since it's used for ActiveData.

CCing Kyle :-)

Comment 2

[Kyle Lahnakoski [:ekyle]]

Thanks Ed, 

What do I replace the `json-info` call with?  I am pulling multiple changesets at once (with "...?node=<rev1>&node=<rev2>&..."

Thank you

Comment 3

[Gregory Szorc [:gps]]

In theory, you can use a revset to select specific changesets. e.g. "d70b350ed568 or 165c878d8dc2".

You can then plug that into a query for the "changelog" web command. e.g. https://hg.mozilla.org/mozilla-central/changelog?rev=d70b350ed568+or+165c878d8dc2

Unfortunately, the JSON style hasn't implemented support for "changelog" yet, so https://hg.mozilla.org/mozilla-central/json-changelog?rev=d70b350ed568+or+165c878d8dc2 doesn't give anything useful. This can be fixed, however.

Comment 4

[Kyle Lahnakoski [:ekyle]]

Give me some time to change ActiveData ingestion to deal with this new request format.

Comment 5

[Kyle Lahnakoski [:ekyle]]

The date appears to be in a new format:  eg [1440148274.0, -28800]

Is that GMT+offset, or localtime with a timezone indicator?  thanks!

Comment 6

[Kyle Lahnakoski [:ekyle]]

The json-changelog endpoint does not return `files` (useful to lookup change history, and for potential codecoverage analysis) or `rev` (useful for a rough ordering of the changesets).


https://hg.mozilla.org/integration/mozilla-inbound/json-changelog?rev=8059df49f5d0

> {
> "node": "8059df49f5d0ba84e48557a7e7eba89482a8a696",
> "query": "8059df49f5d0",
> "entries": [{
> "node": "8059df49f5d0ba84e48557a7e7eba89482a8a696",
> "date": [1483734197.0, 18000],
> "desc": "Bug 1329296 - don't use ExtractSubset on null SkImage when accessing surface pattern fails. r=mchang\n\nMozReview-Commit-ID: I9K07esTA0W",
> "branch": "default",
> "bookmarks": [],
> "tags": ["tip"],
> "user": "Lee Salzman \[e-mail]\u003e",
> "phase": "public",
> "parents": ["bb4089cd9b19f8df190bcd9daa2c20b9bfe446c5"],
> "pushid": "",
> "pushdate": ""
> }]
}


https://hg.mozilla.org/integration/mozilla-inbound/json-info?node=8059df49f5d0

returns

> {
> "8059df49f5d0": {
> "branch": "default",
> "children": [],
> "date": "2017-01-06 15:23 -0500",
> "description": "Bug 1329296 - don't use ExtractSubset on null SkImage when accessing surface pattern fails. r=mchang\n\nMozReview-Commit-ID: I9K07esTA0W",
> "files": ["gfx/2d/DrawTargetSkia.cpp"],
> "node": "8059df49f5d0ba84e48557a7e7eba89482a8a696",
> "parents": ["bb4089cd9b19f8df190bcd9daa2c20b9bfe446c5"],
> "rev": 328320,
> "tags": ["tip"],
> "user": "Lee Salzman \[e-mail]\u003e"
> }
}

Comment 7

[Gregory Szorc [:gps]]

(In reply to Kyle Lahnakoski [:ekyle] from comment #5)
> The date appears to be in a new format:  eg [1440148274.0, -28800]
> 
> Is that GMT+offset, or localtime with a timezone indicator?  thanks!

It is seconds since UNIX epoch and offset of the local timezone (from the author's machine). The pushlog timezone offset should be 0.

Also, the "rev" command (e.g. https://hg.mozilla.org/integration/mozilla-inbound/json-rev/8059df49f5d0) should be used for retrieval of a single changeset.

Comment 8

[Gregory Szorc [:gps]]

(In reply to Kyle Lahnakoski [:ekyle] from comment #6)
> The json-changelog endpoint does not return `files` (useful to lookup change
> history, and for potential codecoverage analysis) or `rev` (useful for a
> rough ordering of the changesets).

The files list we can add (I'm not sure why it isn't in there right now).

"rev" I'm a bit more concerned about, as the integer revisions are not stable (unlike the hashes). That is why they tend not to be exposed. The preferred mechanism to order changesets is by walking revisions and looking at parent nodes to build up a DAG.

Comment 9

[Kyle Lahnakoski [:ekyle]]

(In reply to Gregory Szorc [:gps] from comment #8)
 
> "rev" I'm a bit more concerned about, as the integer revisions are not
> stable (unlike the hashes). That is why they tend not to be exposed. The
> preferred mechanism to order changesets is by walking revisions and looking
> at parent nodes to build up a DAG.

Yes, I am aware of this problem with rev, but it is still useful for finding a shortlist of revisions to be reviewed for chronological proximity.  Without it I must generate some replacement number to perform the rough topological ordering the rev does for me.

Comment 10

[Gregory Szorc [:gps]]

If you are looking for chronological proximity irregardless of the DAG shape, the pushlog can give that to you. There are even some query string parameters to facilitate limited searching. https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmo/pushlog.html documents.

The pushlog isn't part of core Mercurial, so the barrier to change is much lower. If you want e.g. a new query mode added, that  shouldn't be a major problem.