1171250 - Deploy json style to hg.mozilla.org

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Gregory Szorc [:gps]]

Mercurial 3.4 introduced a "json" style that exposes JSON for various endpoints.

When we upgraded hg.mozilla.org to 3.4.1, we didn't pull in this style because it conflicts with stuff that is already there, notably code in pushlog. We should make the new json style work.

Comment 1

[Gregory Szorc [:gps]]

Someone asked me about this today. I've been sitting on it for long enough. Figured I'd put in a few hours to roll this out. The way I see it, having the JSON endpoints will encourage people to build cooler tools.

Comment 2

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: remove "family" web command (bug 1171250); r?smacleod

pushlog: remove "family" web command (bug 1171250); r?smacleod

Server logs reveal nobody uses this. Whatever purpose it may have once
served, it no longer does.

Comment 3

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: remove "webtags" web command (bug 1171250); r?smacleod

pushlog: remove "webtags" web command (bug 1171250); r?smacleod

Server logs reveal this is not used.

Comment 4

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: remove "webheads" web command (bug 1171250); r?smacleod

pushlog: remove "webheads" web command (bug 1171250); r?smacleod

Again, server logs reveal this isn't used.

Comment 5

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: add tests for json-info endpoint (bug 1171250); r?smacleod

pushlog: add tests for json-info endpoint (bug 1171250); r?smacleod

This is used in production. But we have no test coverage for it. Seems
dangerous. Let's add test coverage.

This will also help ensure that imminent refactoring to support the
official JSON style doesn't break this endpoint.

Comment 6

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: remove "printjson" command (bug 1171250); r?smacleod

pushlog: remove "printjson" command (bug 1171250); r?smacleod

Nobody uses this.

Comment 7

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: remove demandimport hack (bug 1171250); r?smacleod

pushlog: remove demandimport hack (bug 1171250); r?smacleod

We now run on Python 2.7, which has the json module. This hack isn't
needed.

Comment 8

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgtemplates: move json map file to own directory (bug 1171250); r?smacleod

hgtemplates: move json map file to own directory (bug 1171250); r?smacleod

In preparation of receiving the official JSON templates, move the
map file to its own directory, like how other styles are defined.

Comment 9

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgtemplates: remove "default" entry from JSON template; (bug 1171250); r?smacleod

hgtemplates: remove "default" entry from JSON template; (bug 1171250); r?smacleod

This isn't needed.

Comment 10

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: pretty format JSON in tests; (bug 1171250); r?smacleod

pushlog: pretty format JSON in tests; (bug 1171250); r?smacleod

Upcoming commits will change how JSON is rendered, mostly with regards
to newlines. By adding json.tool, we not only ensure consistent
formatting but we also ensure the JSON is valid.

Comment 11

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: proper arguments order to jsoncompare (bug 1171250); r?smacleod

pushlog: proper arguments order to jsoncompare (bug 1171250); r?smacleod

While debugging test failures, I noticed that the diffs were in reverse
of what I was expecting. This is because the argument order for
jsoncompare was wrong.

Comment 12

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: use "json" filter on "error" template; (bug 1171250); r?smacleod

pushlog: use "json" filter on "error" template; (bug 1171250); r?smacleod

It behaves more or less the same as "mozjson" for this item.

Comment 13

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: use native templates for json-info command (bug 1171250); r?smacleod

pushlog: use native templates for json-info command (bug 1171250); r?smacleod

Continuing our march towards eliminating the custom "mozjson" template
filter.

Comment 14

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: use native templates for "pushes" web command (bug 1171250); r?smacleod

pushlog: use native templates for "pushes" web command (bug 1171250); r?smacleod

The last big step towards not using our custom JSON template filter is
to convert the "pushes" template to it.

Comment 15

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: remove unused JSON code (bug 1171250); r?smacleod

pushlog: remove unused JSON code (bug 1171250); r?smacleod

Now that we are using native templates for everything, our custom
"mozjson" filter is not used and can be deleted. Good riddance.

Comment 16

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: move "info" web command to hgmo extension (bug 1171250); r?smacleod

pushlog: move "info" web command to hgmo extension (bug 1171250); r?smacleod

hgwebjson.py has slowly been crippled. It is at the point where it only
defines the "info" web command. This command has nothing to do with
pushlog and has no business being in the pushlog-legacy directory. (This
directory held the contents of the old "pushlog" repository.)

We have the "hgmo" extension now, which is where we are placing
Mozilla-specific extensions for the hg.mozilla.org web server. It makes
the perfect host for the "info" web command.

Many files were changed because many things referred to this old
extension!

Comment 17

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgtemplates: add json template (bug 1171250); r?smacleod

hgtemplates: add json template (bug 1171250); r?smacleod

Content is imported from upstream unmodified. Should be a rubber stamp
review.

Comment 18

[Gregory Szorc [:gps]]

Comment on attachment 8646158 [details]
MozReview Request: pushlog: pretty format JSON in tests; (bug 1171250); r?smacleod

pushlog: pretty format JSON in tests; (bug 1171250); r?smacleod

Upcoming commits will change how JSON is rendered, mostly with regards
to newlines. By adding json.tool, we not only ensure consistent
formatting but we also ensure the JSON is valid.

Comment 19

[Gregory Szorc [:gps]]

Comment on attachment 8646159 [details]
MozReview Request: pushlog: proper arguments order to jsoncompare (bug 1171250); r?smacleod

pushlog: proper arguments order to jsoncompare (bug 1171250); r?smacleod

While debugging test failures, I noticed that the diffs were in reverse
of what I was expecting. This is because the argument order for
jsoncompare was wrong.

Comment 20

[Gregory Szorc [:gps]]

Comment on attachment 8646160 [details]
MozReview Request: pushlog: use "json" filter on "error" template; (bug 1171250); r?smacleod

pushlog: use "json" filter on "error" template; (bug 1171250); r?smacleod

It behaves more or less the same as "mozjson" for this item.

Comment 21

[Gregory Szorc [:gps]]

Comment on attachment 8646161 [details]
MozReview Request: pushlog: use native templates for json-info command (bug 1171250); r?smacleod

pushlog: use native templates for json-info command (bug 1171250); r?smacleod

Continuing our march towards eliminating the custom "mozjson" template
filter.

Comment 22

[Gregory Szorc [:gps]]

Comment on attachment 8646162 [details]
MozReview Request: pushlog: use native templates for "pushes" web command (bug 1171250); r?smacleod

pushlog: use native templates for "pushes" web command (bug 1171250); r?smacleod

The last big step towards not using our custom JSON template filter is
to convert the "pushes" template to it.

Comment 23

[Gregory Szorc [:gps]]

Comment on attachment 8646163 [details]
MozReview Request: pushlog: remove unused JSON code (bug 1171250); r?smacleod

pushlog: remove unused JSON code (bug 1171250); r?smacleod

Now that we are using native templates for everything, our custom
"mozjson" filter is not used and can be deleted. Good riddance.

Comment 24

[Gregory Szorc [:gps]]

Comment on attachment 8646164 [details]
MozReview Request: pushlog: move "info" web command to hgmo extension (bug 1171250); r?smacleod

pushlog: move "info" web command to hgmo extension (bug 1171250); r?smacleod

hgwebjson.py has slowly been crippled. It is at the point where it only
defines the "info" web command. This command has nothing to do with
pushlog and has no business being in the pushlog-legacy directory. (This
directory held the contents of the old "pushlog" repository.)

We have the "hgmo" extension now, which is where we are placing
Mozilla-specific extensions for the hg.mozilla.org web server. It makes
the perfect host for the "info" web command.

Many files were changed because many things referred to this old
extension!

Comment 25

[Gregory Szorc [:gps]]

Comment on attachment 8646165 [details]
MozReview Request: hgtemplates: add json template (bug 1171250); r?smacleod

hgtemplates: add json template (bug 1171250); r?smacleod

Content is imported from upstream unmodified. Should be a rubber stamp
review.

Comment 26

[Steven MacLeod [:smacleod]]

Comment on attachment 8646149 [details]
MozReview Request: pushlog: remove "family" web command (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15619/#review14673

Ship It!

Comment 27

[Steven MacLeod [:smacleod]]

Comment on attachment 8646150 [details]
MozReview Request: pushlog: remove "webtags" web command (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15621/#review14675

Ship It!

Comment 28

[Steven MacLeod [:smacleod]]

Comment on attachment 8646151 [details]
MozReview Request: pushlog: remove "webheads" web command (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15623/#review14677

Ship It!

Comment 29

[Steven MacLeod [:smacleod]]

Comment on attachment 8646153 [details]
MozReview Request: pushlog: add tests for json-info endpoint (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15625/#review14679

Ship It!

Comment 30

[Steven MacLeod [:smacleod]]

Comment on attachment 8646154 [details]
MozReview Request: pushlog: remove "printjson" command (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15627/#review14681

Ship It!

Comment 31

[Steven MacLeod [:smacleod]]

Comment on attachment 8646155 [details]
MozReview Request: pushlog: remove demandimport hack (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15629/#review14683

Ship It!

Comment 32

[Steven MacLeod [:smacleod]]

Comment on attachment 8646156 [details]
MozReview Request: hgtemplates: move json map file to own directory (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15631/#review14685

Ship It!

Comment 33

[Steven MacLeod [:smacleod]]

Comment on attachment 8646157 [details]
MozReview Request: hgtemplates: remove "default" entry from JSON template; (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15633/#review14687

Ship It!

Comment 34

[Steven MacLeod [:smacleod]]

Comment on attachment 8646158 [details]
MozReview Request: pushlog: pretty format JSON in tests; (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15635/#review14689

Ship It!

Comment 35

[Steven MacLeod [:smacleod]]

Comment on attachment 8646159 [details]
MozReview Request: pushlog: proper arguments order to jsoncompare (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15637/#review14691

Ship It!

Comment 36

[Steven MacLeod [:smacleod]]

Comment on attachment 8646160 [details]
MozReview Request: pushlog: use "json" filter on "error" template; (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15639/#review14695

Ship It!

Comment 37

[Steven MacLeod [:smacleod]]

Comment on attachment 8646161 [details]
MozReview Request: pushlog: use native templates for json-info command (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15641/#review14703

Ship It!

Comment 38

[Steven MacLeod [:smacleod]]

Comment on attachment 8646162 [details]
MozReview Request: pushlog: use native templates for "pushes" web command (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15643/#review14705

Ship It!

Comment 39

[Steven MacLeod [:smacleod]]

Comment on attachment 8646163 [details]
MozReview Request: pushlog: remove unused JSON code (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15645/#review14707

Ship It!

Comment 40

[Steven MacLeod [:smacleod]]

Comment on attachment 8646164 [details]
MozReview Request: pushlog: move "info" web command to hgmo extension (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15647/#review14709

Ship It!

Comment 41

[Steven MacLeod [:smacleod]]

Comment on attachment 8646165 [details]
MozReview Request: hgtemplates: add json template (bug 1171250); r?smacleod

https://reviewboard.mozilla.org/r/15649/#review14711

Ship It!

Comment 42

[Gregory Szorc [:gps]]

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/a6a339d754bece250cbc85acbb3b31f31f15c175
changeset:  a6a339d754bece250cbc85acbb3b31f31f15c175
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:25:31 2015 -0700
description:
pushlog: remove "family" web command (bug 1171250); r=smacleod

Server logs reveal nobody uses this. Whatever purpose it may have once
served, it no longer does.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/39b0979c892295e543f481fe6899a232263475d9
changeset:  39b0979c892295e543f481fe6899a232263475d9
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:25:43 2015 -0700
description:
pushlog: remove "webtags" web command (bug 1171250); r=smacleod

Server logs reveal this is not used.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/6090374b05188e5867fb1aaf0720585bf5f6c750
changeset:  6090374b05188e5867fb1aaf0720585bf5f6c750
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:25:52 2015 -0700
description:
pushlog: remove "webheads" web command (bug 1171250); r=smacleod

Again, server logs reveal this isn't used.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/40b78d85308a81a05db972f7c3c149efbd561bdc
changeset:  40b78d85308a81a05db972f7c3c149efbd561bdc
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 17:22:29 2015 -0700
description:
pushlog: add tests for json-info endpoint (bug 1171250); r=smacleod

This is used in production. But we have no test coverage for it. Seems
dangerous. Let's add test coverage.

This will also help ensure that imminent refactoring to support the
official JSON style doesn't break this endpoint.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/1a566a750bfea6389a10a23011f546f82dfcfcd1
changeset:  1a566a750bfea6389a10a23011f546f82dfcfcd1
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:23:26 2015 -0700
description:
pushlog: remove "printjson" command (bug 1171250); r=smacleod

Nobody uses this.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/ed7d2c20d229fdae736824c508167f2277419559
changeset:  ed7d2c20d229fdae736824c508167f2277419559
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:24:04 2015 -0700
description:
pushlog: remove demandimport hack (bug 1171250); r=smacleod

We now run on Python 2.7, which has the json module. This hack isn't
needed.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/a6f3d0ed4605e7dda4df8ebe366367166e1be71a
changeset:  a6f3d0ed4605e7dda4df8ebe366367166e1be71a
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:30:41 2015 -0700
description:
hgtemplates: move json map file to own directory (bug 1171250); r=smacleod

In preparation of receiving the official JSON templates, move the
map file to its own directory, like how other styles are defined.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/f1bf6cb3f0c305f4e64afda99109854e0bd08c3e
changeset:  f1bf6cb3f0c305f4e64afda99109854e0bd08c3e
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:31:09 2015 -0700
description:
hgtemplates: remove "default" entry from JSON template; (bug 1171250); r=smacleod

This isn't needed.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/cc6615c5acca197f39c63688df5bc8f06db4228b
changeset:  cc6615c5acca197f39c63688df5bc8f06db4228b
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 22:14:32 2015 -0700
description:
pushlog: pretty format JSON in tests; (bug 1171250); r=smacleod

Upcoming commits will change how JSON is rendered, mostly with regards
to newlines. By adding json.tool, we not only ensure consistent
formatting but we also ensure the JSON is valid.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/fb32d9201f8b964aa6134fca9233c1aaa938f960
changeset:  fb32d9201f8b964aa6134fca9233c1aaa938f960
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 17:51:51 2015 -0700
description:
pushlog: proper arguments order to jsoncompare (bug 1171250); r=smacleod

While debugging test failures, I noticed that the diffs were in reverse
of what I was expecting. This is because the argument order for
jsoncompare was wrong.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/32b05d1f3e5085769aa277c66b657d28ce3c41ed
changeset:  32b05d1f3e5085769aa277c66b657d28ce3c41ed
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 17:33:55 2015 -0700
description:
pushlog: use "json" filter on "error" template; (bug 1171250); r=smacleod

It behaves more or less the same as "mozjson" for this item.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/840272cb0b027bf2ef902b15274a106470a937cc
changeset:  840272cb0b027bf2ef902b15274a106470a937cc
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 16:59:19 2015 -0700
description:
pushlog: use native templates for json-info command (bug 1171250); r=smacleod

Continuing our march towards eliminating the custom "mozjson" template
filter.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/22031500436f702f72c082d596bb013459e0c550
changeset:  22031500436f702f72c082d596bb013459e0c550
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 19:08:44 2015 -0700
description:
pushlog: use native templates for "pushes" web command (bug 1171250); r=smacleod

The last big step towards not using our custom JSON template filter is
to convert the "pushes" template to it.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/dab75cd11c6dfe4767433d65d92a4f2be5a37f43
changeset:  dab75cd11c6dfe4767433d65d92a4f2be5a37f43
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 19:11:03 2015 -0700
description:
pushlog: remove unused JSON code (bug 1171250); r=smacleod

Now that we are using native templates for everything, our custom
"mozjson" filter is not used and can be deleted. Good riddance.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/aa9c71d9b9251a5aa933b0eeacf91d5d51f7389b
changeset:  aa9c71d9b9251a5aa933b0eeacf91d5d51f7389b
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 19:22:16 2015 -0700
description:
pushlog: move "info" web command to hgmo extension (bug 1171250); r=smacleod

hgwebjson.py has slowly been crippled. It is at the point where it only
defines the "info" web command. This command has nothing to do with
pushlog and has no business being in the pushlog-legacy directory. (This
directory held the contents of the old "pushlog" repository.)

We have the "hgmo" extension now, which is where we are placing
Mozilla-specific extensions for the hg.mozilla.org web server. It makes
the perfect host for the "info" web command.

Many files were changed because many things referred to this old
extension!

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/1dfb35c3cd0d7b0899fabc5db55e3478c2b94e5f
changeset:  1dfb35c3cd0d7b0899fabc5db55e3478c2b94e5f
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 10 22:10:34 2015 -0700
description:
hgtemplates: add json template (bug 1171250); r=smacleod

Content is imported from upstream unmodified. Should be a rubber stamp
review.

Comment 43

[Gregory Szorc [:gps]]

And this is deployed!

It did result in some downtime of json-pushes endpoints for a few dozen seconds - possibly up to a minute. Alerted people in #releng in case it caused disruptions. Hopefully clients know how to retry after failures.