1171321 - Show rich metadata on hgweb

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Gregory Szorc [:gps]]

I added a feature to Mercurial 3.4 that makes it easier to inject extra content into the templates used to render web pages. We can leverage this for all kinds of awesome. Let's start by showing more pushlog info.

Comment 1

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: defer imports; r?smacleod

pushlog: defer imports

Mercurial uses an on-demand importer. However, "from X import Y"
short circuits it and causes immediate imports. This adds a slight
performance overhead to load the extension. Don't do that.

Comment 2

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.

Comment 3

[Gregory Szorc [:gps]]

Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.

Comment 4

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

Let's display more useful info on per-changeset pages on hgweb!

This commit introduces support for rendering the list of reviewers and
bugs.

The state of the hgweb extensions on hg.mozilla.org is kinda funky. Most
of the code today lives in the poorly named hgext/pushlog-legacy
directory (named this way because content came from the original
pushlog repository that was imported into v-c-t around the same time
work started on the new pushlog extension, which is under hgext/pushlog).
So, given that the code is in a weird state and I'm about to start work
to pour a lot of love into it, I took the opportunity to create an
"hgmo" extension. My hope is this extension will become the location for
all the hgweb customizations we do on hg.mozilla.org.

This extensions relied on a v-c-t checkout, which we currently don't
have on the hgweb machines. We'll need a follow-up commit with Ansible
changes to teach the hgweb machines how to run from v-c-t. Since
the hgrc for hg.mozilla.org is stored in this repo and we haven't
introduced this extension to it yet, this commit is safe to check in.

Comment 5

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

hgmo: add milestone to changeset output (bug 1171321); r?smacleod

The config/milestone.txt file in Firefox repos identifies the Gecko/app
version of the changeset. By displaying the milestone, people can easily
get an answer for "what was the first version of Firefox this changeset
landed in."

Comment 6

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

What's better than listing reviewers in changeset output? Linking
to other work that reviewer did. This commit implements that.

We implement a "reviewer" revset that looks up changesets that were
reviewed by someone. We mark it as a safe symbol so it can be exposed
to hgweb's search feature. We finally hyperlink reviewers in
changeset output so you can easily find other changesets that they
reviewed.

Sadly, while developing this feature, I realized that the upstream
work I did to expose extra metadata to the templater did not
include search results. So, we're unable to display pushlog
and other metadata in the search results page yet. Boo.

Comment 7

[Gregory Szorc [:gps]]

Let's expand the summary since I had some coding fun while drinking a beer tonight.

I'll try to code up the Ansible foo tomorrow so we can get this deployed.

Comment 8

[Gregory Szorc [:gps]]

Comment on attachment 8615058 [details]
MozReview Request: pushlog: defer imports; r?smacleod

pushlog: defer imports; r?smacleod

Mercurial uses an on-demand importer. However, "from X import Y"
short circuits it and causes immediate imports. This adds a slight
performance overhead to load the extension. Don't do that.

Comment 9

[Gregory Szorc [:gps]]

Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.

Comment 10

[Gregory Szorc [:gps]]

Comment on attachment 8615093 [details]
MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

Let's display more useful info on per-changeset pages on hgweb!

This commit introduces support for rendering the list of reviewers and
bugs.

The state of the hgweb extensions on hg.mozilla.org is kinda funky. Most
of the code today lives in the poorly named hgext/pushlog-legacy
directory (named this way because content came from the original
pushlog repository that was imported into v-c-t around the same time
work started on the new pushlog extension, which is under hgext/pushlog).
So, given that the code is in a weird state and I'm about to start work
to pour a lot of love into it, I took the opportunity to create an
"hgmo" extension. My hope is this extension will become the location for
all the hgweb customizations we do on hg.mozilla.org.

This extensions relied on a v-c-t checkout, which we currently don't
have on the hgweb machines. We'll need a follow-up commit with Ansible
changes to teach the hgweb machines how to run from v-c-t. Since
the hgrc for hg.mozilla.org is stored in this repo and we haven't
introduced this extension to it yet, this commit is safe to check in.

Comment 11

[Gregory Szorc [:gps]]

Comment on attachment 8615094 [details]
MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

hgmo: add milestone to changeset output (bug 1171321); r?smacleod

The config/milestone.txt file in Firefox repos identifies the Gecko/app
version of the changeset. By displaying the milestone, people can easily
get an answer for "what was the first version of Firefox this changeset
landed in."

Comment 12

[Gregory Szorc [:gps]]

Comment on attachment 8615108 [details]
MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

What's better than listing reviewers in changeset output? Linking
to other work that reviewer did. This commit implements that.

We implement a "reviewer" revset that looks up changesets that were
reviewed by someone. We mark it as a safe symbol so it can be exposed
to hgweb's search feature. We finally hyperlink reviewers in
changeset output so you can easily find other changesets that they
reviewed.

Sadly, while developing this feature, I realized that the upstream
work I did to expose extra metadata to the templater did not
include search results. So, we're unable to display pushlog
and other metadata in the search results page yet. Boo.

Comment 13

[Gregory Szorc [:gps]]

Attachment: MozReview Request: mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

It's nice to know if a commit is a backout commit. Let's add some
functionality for parsing backout info from commit messages.

Comment 14

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

Now that we can extract backout information from commit messages, let's
do something useful with it. We implement rendering of links to the
backed out changesets.

Comment 15

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgmo: show that a commit was backed out (bug 1171321); r?smacleod

hgmo: show that a commit was backed out (bug 1171321); r?smacleod

When looking at commits, it would be nice if we could quickly see if the
commit was backed out. This commit implements support for that feature.

"Bad" commits are denoted two ways. First, a pilo of poo is rendered
next to the changeset node in the page header. Second, a "backed out by"
message is rendered at the top of the changeset metadata section. It's
hard to miss.

The styling isn't the greatest. But something is better than nothing.

Comment 16

[Gregory Szorc [:gps]]

Comment on attachment 8615058 [details]
MozReview Request: pushlog: defer imports; r?smacleod

pushlog: defer imports; r?smacleod

Mercurial uses an on-demand importer. However, "from X import Y"
short circuits it and causes immediate imports. This adds a slight
performance overhead to load the extension. Don't do that.

Comment 17

[Gregory Szorc [:gps]]

Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.

Comment 18

[Gregory Szorc [:gps]]

Comment on attachment 8615093 [details]
MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

Let's display more useful info on per-changeset pages on hgweb!

This commit introduces support for rendering the list of reviewers and
bugs.

The state of the hgweb extensions on hg.mozilla.org is kinda funky. Most
of the code today lives in the poorly named hgext/pushlog-legacy
directory (named this way because content came from the original
pushlog repository that was imported into v-c-t around the same time
work started on the new pushlog extension, which is under hgext/pushlog).
So, given that the code is in a weird state and I'm about to start work
to pour a lot of love into it, I took the opportunity to create an
"hgmo" extension. My hope is this extension will become the location for
all the hgweb customizations we do on hg.mozilla.org.

This extensions relied on a v-c-t checkout, which we currently don't
have on the hgweb machines. We'll need a follow-up commit with Ansible
changes to teach the hgweb machines how to run from v-c-t. Since
the hgrc for hg.mozilla.org is stored in this repo and we haven't
introduced this extension to it yet, this commit is safe to check in.

Comment 19

[Gregory Szorc [:gps]]

Comment on attachment 8615094 [details]
MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

hgmo: add milestone to changeset output (bug 1171321); r?smacleod

The config/milestone.txt file in Firefox repos identifies the Gecko/app
version of the changeset. By displaying the milestone, people can easily
get an answer for "what was the first version of Firefox this changeset
landed in."

Comment 20

[Gregory Szorc [:gps]]

Comment on attachment 8615108 [details]
MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

What's better than listing reviewers in changeset output? Linking
to other work that reviewer did. This commit implements that.

We implement a "reviewer" revset that looks up changesets that were
reviewed by someone. We mark it as a safe symbol so it can be exposed
to hgweb's search feature. We finally hyperlink reviewers in
changeset output so you can easily find other changesets that they
reviewed.

Sadly, while developing this feature, I realized that the upstream
work I did to expose extra metadata to the templater did not
include search results. So, we're unable to display pushlog
and other metadata in the search results page yet. Boo.

Comment 21

[Gregory Szorc [:gps]]

Comment on attachment 8615428 [details]
MozReview Request: mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

It's nice to know if a commit is a backout commit. Let's add some
functionality for parsing backout info from commit messages.

Comment 22

[Gregory Szorc [:gps]]

Comment on attachment 8615429 [details]
MozReview Request: hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

Now that we can extract backout information from commit messages, let's
do something useful with it. We implement rendering of links to the
backed out changesets.

Comment 23

[Gregory Szorc [:gps]]

Comment on attachment 8615430 [details]
MozReview Request: hgmo: show that a commit was backed out (bug 1171321); r?smacleod

hgmo: show that a commit was backed out (bug 1171321); r?smacleod

When looking at commits, it would be nice if we could quickly see if the
commit was backed out. This commit implements support for that feature.

"Bad" commits are denoted two ways. First, a pilo of poo is rendered
next to the changeset node in the page header. Second, a "backed out by"
message is rendered at the top of the changeset metadata section. It's
hard to miss.

The styling isn't the greatest. But something is better than nothing.

Comment 24

[Gregory Szorc [:gps]]

Attachment: MozReview Request: ansible: create version-control-tools repo on hgweb machines (bug 1171321); r?bkero

ansible: create version-control-tools repo on hgweb machines (bug 1171321); r?bkero

In order to activate the hgmo extension on hgweb, we need a copy of the
version-control-tools repo to serve it from. Establish it.

Comment 25

[Gregory Szorc [:gps]]

Attachment: MozReview Request: ansible: activate hgmo extension on hgweb machines (bug 1171321); r?bkero

ansible: activate hgmo extension on hgweb machines (bug 1171321); r?bkero

Now that version-control-tools is present on these machines, we can
start running content from it.

Comment 26

[Ben Kero [:bkero]]

Comment on attachment 8615440 [details]
MozReview Request: ansible: create version-control-tools repo on hgweb machines (bug 1171321); r?bkero

https://reviewboard.mozilla.org/r/10253/#review8973

Ship It!

Comment 27

[Ben Kero [:bkero]]

Comment on attachment 8615441 [details]
MozReview Request: ansible: activate hgmo extension on hgweb machines (bug 1171321); r?bkero

https://reviewboard.mozilla.org/r/10255/#review8975

Ship It!

Comment 28

[Gregory Szorc [:gps]]

https://hg.mozilla.org/hgcustom/version-control-tools/rev/f5c483089feb

Comment 29

[Steven MacLeod [:smacleod]]

Comment on attachment 8615058 [details]
MozReview Request: pushlog: defer imports; r?smacleod

https://reviewboard.mozilla.org/r/10151/#review8977

Ship It!

Comment 30

[Steven MacLeod [:smacleod]]

Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10153/#review8979

::: hgext/pushlog/__init__.py:375
(Diff revision 4)
> +        Returns a tuple of (pushid, who, when, [nodes]) or None if there is

Returning a `namedtuple` could be nice for consumers.

Comment 31

[Steven MacLeod [:smacleod]]

Comment on attachment 8615093 [details]
MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10171/#review8983

Commit message typo: "This extensions relied on"

Comment 32

[Steven MacLeod [:smacleod]]

Comment on attachment 8615094 [details]
MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10173/#review8985

Ship It!

Comment 33

[Steven MacLeod [:smacleod]]

Comment on attachment 8615108 [details]
MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10175/#review8989

::: hgtemplates/gitweb_mozilla/map:315
(Diff revision 3)
> +reviewerlink = '<a href="{url|urlescape}log?rev={revset|urlescape}&revcount=50">{name|escape}</a>'

It doesn't look like the search (log) page has any pagination.... so this will always just show the last 50 they reviewed.

Is pagination something we can turn on for stuff like this? or infinite scrolling?

Comment 34

[Gregory Szorc [:gps]]

https://reviewboard.mozilla.org/r/10175/#review8993

> It doesn't look like the search (log) page has any pagination.... so this will always just show the last 50 they reviewed.
> 
> Is pagination something we can turn on for stuff like this? or infinite scrolling?

I'm not sure exactly what happens in the search page. From basic testing, it appears infinite scrolling works. But it only displays up to the revcount entries, so not very infinite. I suspect it is querying windows of N changesets per request until it finds revcount results.

I'm inclined to punt this to a follow-up. As I said in the commit message, something is better than nothing.

Comment 35

[Steven MacLeod [:smacleod]]

Comment on attachment 8615428 [details]
MozReview Request: mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10247/#review8991

::: pylib/mozautomation/mozautomation/commitparser.py:69
(Diff revision 2)
> +    Returns a 2-typle of (nodes, bugs) where each entry is an iterable of

"2-typle"

Comment 36

[Steven MacLeod [:smacleod]]

Comment on attachment 8615429 [details]
MozReview Request: hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10249/#review8995

Ship It!

Comment 37

[Steven MacLeod [:smacleod]]

Comment on attachment 8615430 [details]
MozReview Request: hgmo: show that a commit was backed out (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10251/#review8997

Commit message nit: "a pilo of poo"

::: hgext/hgmo/__init__.py:78
(Diff revision 2)
> +        if count >= 100:

Maybe this limit should be defined somewhere better or passed in as an argument.

Luckily most firefox commit messages are super short so this won't have to parse too much text :P heh.

::: hgtemplates/gitweb_mozilla/changeset.tmpl:34
(Diff revision 2)
> +{if(backedoutbynode, '<tr><td><strong>&#x1f4a9;&#x1f4a9; backed out by <a style="font-family: monospace" href="{url|urlescape}rev/{backedoutbynode|short}">{backedoutbynode|short}</a> &#x1f4a9; &#x1f4a9;</strong></td></tr>')}

I worry people will eventually assume this _not_ appearing means something was _never_ backed out, but it was actually backed out far enough in the future. I wonder how rare this actually is? I'm sure it wouldn't be too hard for you to find that out in a local clone (how often a backout is > 100 changesets after something lands).

Is there anything we could do to make this more obvious, maybe be more explicit about how the backout detection works when it appears? or do we even care?

I'm fine landing this as is, but I'd be happier if you took a couple minutes to think about if we can do better.

Comment 38

[Gregory Szorc [:gps]]

https://reviewboard.mozilla.org/r/10251/#review9005

> I worry people will eventually assume this _not_ appearing means something was _never_ backed out, but it was actually backed out far enough in the future. I wonder how rare this actually is? I'm sure it wouldn't be too hard for you to find that out in a local clone (how often a backout is > 100 changesets after something lands).
> 
> Is there anything we could do to make this more obvious, maybe be more explicit about how the backout detection works when it appears? or do we even care?
> 
> I'm fine landing this as is, but I'd be happier if you took a couple minutes to think about if we can do better.

I view the message as best effort. Now, how do we communicate that to people, I don't know. I say that because most commits aren't backed out. Thus the backout search won't yield anything. Therefore, backed out N>100 commits from now is undistinguishable from not backed out.

I we wanted to get fancy, we could have a cron do backout calculations and pull backout lookups from a cache. Follow-up fodder (if necessary).

> Maybe this limit should be defined somewhere better or passed in as an argument.
> 
> Luckily most firefox commit messages are super short so this won't have to parse too much text :P heh.

I'll make this into a config file lookup. Thank you for calling me out on use of magic numbers.

Comment 39

[Gregory Szorc [:gps]]

https://hg.mozilla.org/hgcustom/version-control-tools/rev/8e5585e70cf3
https://hg.mozilla.org/hgcustom/version-control-tools/rev/f39448d0b95b
https://hg.mozilla.org/hgcustom/version-control-tools/rev/57bda639f3fa
https://hg.mozilla.org/hgcustom/version-control-tools/rev/129ffe82c2d9
https://hg.mozilla.org/hgcustom/version-control-tools/rev/eb454545b4df
https://hg.mozilla.org/hgcustom/version-control-tools/rev/68752046218f
https://hg.mozilla.org/hgcustom/version-control-tools/rev/081f4808b593
https://hg.mozilla.org/hgcustom/version-control-tools/rev/87cabc8985cb

Comment 40

[Gregory Szorc [:gps]]

This is deployed. bbouvier found a bug in reviewer parsing, which I deployed after a pastebin review by smacleod. That bug could also be leading to MozReview not detecting reviewers properly.