Closed Bug 1171321 Opened 6 years ago Closed 6 years ago

Show rich metadata on hgweb

Categories

(Developer Services :: Mercurial: hg.mozilla.org, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gps, Assigned: gps)

References

(Blocks 2 open bugs)

Details

Attachments

(10 files)

40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
smacleod
: review+
Details
40 bytes, text/x-review-board-request
bkero
: review+
Details
40 bytes, text/x-review-board-request
bkero
: review+
Details
I added a feature to Mercurial 3.4 that makes it easier to inject extra content into the templates used to render web pages. We can leverage this for all kinds of awesome. Let's start by showing more pushlog info.
pushlog: defer imports

Mercurial uses an on-demand importer. However, "from X import Y"
short circuits it and causes immediate imports. This adds a slight
performance overhead to load the extension. Don't do that.
Attachment #8615058 - Flags: review?(smacleod)
pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.
Attachment #8615059 - Flags: review?(smacleod)
Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.
hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

Let's display more useful info on per-changeset pages on hgweb!

This commit introduces support for rendering the list of reviewers and
bugs.

The state of the hgweb extensions on hg.mozilla.org is kinda funky. Most
of the code today lives in the poorly named hgext/pushlog-legacy
directory (named this way because content came from the original
pushlog repository that was imported into v-c-t around the same time
work started on the new pushlog extension, which is under hgext/pushlog).
So, given that the code is in a weird state and I'm about to start work
to pour a lot of love into it, I took the opportunity to create an
"hgmo" extension. My hope is this extension will become the location for
all the hgweb customizations we do on hg.mozilla.org.

This extensions relied on a v-c-t checkout, which we currently don't
have on the hgweb machines. We'll need a follow-up commit with Ansible
changes to teach the hgweb machines how to run from v-c-t. Since
the hgrc for hg.mozilla.org is stored in this repo and we haven't
introduced this extension to it yet, this commit is safe to check in.
Attachment #8615093 - Flags: review?(smacleod)
hgmo: add milestone to changeset output (bug 1171321); r?smacleod

The config/milestone.txt file in Firefox repos identifies the Gecko/app
version of the changeset. By displaying the milestone, people can easily
get an answer for "what was the first version of Firefox this changeset
landed in."
Attachment #8615094 - Flags: review?(smacleod)
hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

What's better than listing reviewers in changeset output? Linking
to other work that reviewer did. This commit implements that.

We implement a "reviewer" revset that looks up changesets that were
reviewed by someone. We mark it as a safe symbol so it can be exposed
to hgweb's search feature. We finally hyperlink reviewers in
changeset output so you can easily find other changesets that they
reviewed.

Sadly, while developing this feature, I realized that the upstream
work I did to expose extra metadata to the templater did not
include search results. So, we're unable to display pushlog
and other metadata in the search results page yet. Boo.
Attachment #8615108 - Flags: review?(smacleod)
Let's expand the summary since I had some coding fun while drinking a beer tonight.

I'll try to code up the Ansible foo tomorrow so we can get this deployed.
Component: Mercurial: Pushlog → Mercurial: hg.mozilla.org
QA Contact: hwine
Summary: Show pushlog info in more places → Show rich metadata on hgweb
Attachment #8615058 - Attachment description: MozReview Request: pushlog: defer imports → MozReview Request: pushlog: defer imports; r?smacleod
Comment on attachment 8615058 [details]
MozReview Request: pushlog: defer imports; r?smacleod

pushlog: defer imports; r?smacleod

Mercurial uses an on-demand importer. However, "from X import Y"
short circuits it and causes immediate imports. This adds a slight
performance overhead to load the extension. Don't do that.
Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.
Comment on attachment 8615093 [details]
MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

Let's display more useful info on per-changeset pages on hgweb!

This commit introduces support for rendering the list of reviewers and
bugs.

The state of the hgweb extensions on hg.mozilla.org is kinda funky. Most
of the code today lives in the poorly named hgext/pushlog-legacy
directory (named this way because content came from the original
pushlog repository that was imported into v-c-t around the same time
work started on the new pushlog extension, which is under hgext/pushlog).
So, given that the code is in a weird state and I'm about to start work
to pour a lot of love into it, I took the opportunity to create an
"hgmo" extension. My hope is this extension will become the location for
all the hgweb customizations we do on hg.mozilla.org.

This extensions relied on a v-c-t checkout, which we currently don't
have on the hgweb machines. We'll need a follow-up commit with Ansible
changes to teach the hgweb machines how to run from v-c-t. Since
the hgrc for hg.mozilla.org is stored in this repo and we haven't
introduced this extension to it yet, this commit is safe to check in.
Comment on attachment 8615094 [details]
MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

hgmo: add milestone to changeset output (bug 1171321); r?smacleod

The config/milestone.txt file in Firefox repos identifies the Gecko/app
version of the changeset. By displaying the milestone, people can easily
get an answer for "what was the first version of Firefox this changeset
landed in."
Comment on attachment 8615108 [details]
MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

What's better than listing reviewers in changeset output? Linking
to other work that reviewer did. This commit implements that.

We implement a "reviewer" revset that looks up changesets that were
reviewed by someone. We mark it as a safe symbol so it can be exposed
to hgweb's search feature. We finally hyperlink reviewers in
changeset output so you can easily find other changesets that they
reviewed.

Sadly, while developing this feature, I realized that the upstream
work I did to expose extra metadata to the templater did not
include search results. So, we're unable to display pushlog
and other metadata in the search results page yet. Boo.
mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

It's nice to know if a commit is a backout commit. Let's add some
functionality for parsing backout info from commit messages.
Attachment #8615428 - Flags: review?(smacleod)
hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

Now that we can extract backout information from commit messages, let's
do something useful with it. We implement rendering of links to the
backed out changesets.
Attachment #8615429 - Flags: review?(smacleod)
hgmo: show that a commit was backed out (bug 1171321); r?smacleod

When looking at commits, it would be nice if we could quickly see if the
commit was backed out. This commit implements support for that feature.

"Bad" commits are denoted two ways. First, a pilo of poo is rendered
next to the changeset node in the page header. Second, a "backed out by"
message is rendered at the top of the changeset metadata section. It's
hard to miss.

The styling isn't the greatest. But something is better than nothing.
Attachment #8615430 - Flags: review?(smacleod)
Comment on attachment 8615058 [details]
MozReview Request: pushlog: defer imports; r?smacleod

pushlog: defer imports; r?smacleod

Mercurial uses an on-demand importer. However, "from X import Y"
short circuits it and causes immediate imports. This adds a slight
performance overhead to load the extension. Don't do that.
Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

I added a feature to Mercurial 3.4 that allows you to easily monkeypatch
the functions that provide per-changeset data to the templater for hgweb
pages. Now that we have Mercurial 3.4.1 deployed on hg.mozilla.org, we can
start using this feature for awesomeness.

This commit exposes pushlog metadata to the templater and renders it on
per-changeset pages and the changelog template. When looking at these
pages, you now don't have to click another link to find pushlog info.
Comment on attachment 8615093 [details]
MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

Let's display more useful info on per-changeset pages on hgweb!

This commit introduces support for rendering the list of reviewers and
bugs.

The state of the hgweb extensions on hg.mozilla.org is kinda funky. Most
of the code today lives in the poorly named hgext/pushlog-legacy
directory (named this way because content came from the original
pushlog repository that was imported into v-c-t around the same time
work started on the new pushlog extension, which is under hgext/pushlog).
So, given that the code is in a weird state and I'm about to start work
to pour a lot of love into it, I took the opportunity to create an
"hgmo" extension. My hope is this extension will become the location for
all the hgweb customizations we do on hg.mozilla.org.

This extensions relied on a v-c-t checkout, which we currently don't
have on the hgweb machines. We'll need a follow-up commit with Ansible
changes to teach the hgweb machines how to run from v-c-t. Since
the hgrc for hg.mozilla.org is stored in this repo and we haven't
introduced this extension to it yet, this commit is safe to check in.
Comment on attachment 8615094 [details]
MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

hgmo: add milestone to changeset output (bug 1171321); r?smacleod

The config/milestone.txt file in Firefox repos identifies the Gecko/app
version of the changeset. By displaying the milestone, people can easily
get an answer for "what was the first version of Firefox this changeset
landed in."
Comment on attachment 8615108 [details]
MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

What's better than listing reviewers in changeset output? Linking
to other work that reviewer did. This commit implements that.

We implement a "reviewer" revset that looks up changesets that were
reviewed by someone. We mark it as a safe symbol so it can be exposed
to hgweb's search feature. We finally hyperlink reviewers in
changeset output so you can easily find other changesets that they
reviewed.

Sadly, while developing this feature, I realized that the upstream
work I did to expose extra metadata to the templater did not
include search results. So, we're unable to display pushlog
and other metadata in the search results page yet. Boo.
Comment on attachment 8615428 [details]
MozReview Request: mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

It's nice to know if a commit is a backout commit. Let's add some
functionality for parsing backout info from commit messages.
Comment on attachment 8615429 [details]
MozReview Request: hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

Now that we can extract backout information from commit messages, let's
do something useful with it. We implement rendering of links to the
backed out changesets.
Comment on attachment 8615430 [details]
MozReview Request: hgmo: show that a commit was backed out (bug 1171321); r?smacleod

hgmo: show that a commit was backed out (bug 1171321); r?smacleod

When looking at commits, it would be nice if we could quickly see if the
commit was backed out. This commit implements support for that feature.

"Bad" commits are denoted two ways. First, a pilo of poo is rendered
next to the changeset node in the page header. Second, a "backed out by"
message is rendered at the top of the changeset metadata section. It's
hard to miss.

The styling isn't the greatest. But something is better than nothing.
ansible: create version-control-tools repo on hgweb machines (bug 1171321); r?bkero

In order to activate the hgmo extension on hgweb, we need a copy of the
version-control-tools repo to serve it from. Establish it.
Attachment #8615440 - Flags: review?(bkero)
ansible: activate hgmo extension on hgweb machines (bug 1171321); r?bkero

Now that version-control-tools is present on these machines, we can
start running content from it.
Attachment #8615441 - Flags: review?(bkero)
Comment on attachment 8615440 [details]
MozReview Request: ansible: create version-control-tools repo on hgweb machines (bug 1171321); r?bkero

https://reviewboard.mozilla.org/r/10253/#review8973

Ship It!
Attachment #8615440 - Flags: review?(bkero) → review+
Attachment #8615441 - Flags: review?(bkero) → review+
Comment on attachment 8615441 [details]
MozReview Request: ansible: activate hgmo extension on hgweb machines (bug 1171321); r?bkero

https://reviewboard.mozilla.org/r/10255/#review8975

Ship It!
Comment on attachment 8615058 [details]
MozReview Request: pushlog: defer imports; r?smacleod

https://reviewboard.mozilla.org/r/10151/#review8977

Ship It!
Attachment #8615058 - Flags: review?(smacleod) → review+
Comment on attachment 8615059 [details]
MozReview Request: pushlog: render pushlog info in more places in hgweb (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10153/#review8979

::: hgext/pushlog/__init__.py:375
(Diff revision 4)
> +        Returns a tuple of (pushid, who, when, [nodes]) or None if there is

Returning a `namedtuple` could be nice for consumers.
Attachment #8615059 - Flags: review?(smacleod) → review+
Comment on attachment 8615093 [details]
MozReview Request: hgmo: render reviewer and bug list in hgweb (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10171/#review8983

Commit message typo: "This extensions relied on"
Attachment #8615093 - Flags: review?(smacleod) → review+
Comment on attachment 8615094 [details]
MozReview Request: hgmo: add milestone to changeset output (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10173/#review8985

Ship It!
Attachment #8615094 - Flags: review?(smacleod) → review+
Attachment #8615108 - Flags: review?(smacleod) → review+
Comment on attachment 8615108 [details]
MozReview Request: hgmo: expose and hyperlink revset for reviewers (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10175/#review8989

::: hgtemplates/gitweb_mozilla/map:315
(Diff revision 3)
> +reviewerlink = '<a href="{url|urlescape}log?rev={revset|urlescape}&revcount=50">{name|escape}</a>'

It doesn't look like the search (log) page has any pagination.... so this will always just show the last 50 they reviewed.

Is pagination something we can turn on for stuff like this? or infinite scrolling?
https://reviewboard.mozilla.org/r/10175/#review8993

> It doesn't look like the search (log) page has any pagination.... so this will always just show the last 50 they reviewed.
> 
> Is pagination something we can turn on for stuff like this? or infinite scrolling?

I'm not sure exactly what happens in the search page. From basic testing, it appears infinite scrolling works. But it only displays up to the revcount entries, so not very infinite. I suspect it is querying windows of N changesets per request until it finds revcount results.

I'm inclined to punt this to a follow-up. As I said in the commit message, something is better than nothing.
Comment on attachment 8615428 [details]
MozReview Request: mozautomation: support for parsing backouts from commit messages (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10247/#review8991

::: pylib/mozautomation/mozautomation/commitparser.py:69
(Diff revision 2)
> +    Returns a 2-typle of (nodes, bugs) where each entry is an iterable of

"2-typle"
Attachment #8615428 - Flags: review?(smacleod) → review+
Comment on attachment 8615429 [details]
MozReview Request: hgmo: link to backed out changesets on per-changeset page (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10249/#review8995

Ship It!
Attachment #8615429 - Flags: review?(smacleod) → review+
Attachment #8615430 - Flags: review?(smacleod) → review+
Comment on attachment 8615430 [details]
MozReview Request: hgmo: show that a commit was backed out (bug 1171321); r?smacleod

https://reviewboard.mozilla.org/r/10251/#review8997

Commit message nit: "a pilo of poo"

::: hgext/hgmo/__init__.py:78
(Diff revision 2)
> +        if count >= 100:

Maybe this limit should be defined somewhere better or passed in as an argument.

Luckily most firefox commit messages are super short so this won't have to parse too much text :P heh.

::: hgtemplates/gitweb_mozilla/changeset.tmpl:34
(Diff revision 2)
> +{if(backedoutbynode, '<tr><td><strong>&#x1f4a9;&#x1f4a9; backed out by <a style="font-family: monospace" href="{url|urlescape}rev/{backedoutbynode|short}">{backedoutbynode|short}</a> &#x1f4a9; &#x1f4a9;</strong></td></tr>')}

I worry people will eventually assume this _not_ appearing means something was _never_ backed out, but it was actually backed out far enough in the future. I wonder how rare this actually is? I'm sure it wouldn't be too hard for you to find that out in a local clone (how often a backout is > 100 changesets after something lands).

Is there anything we could do to make this more obvious, maybe be more explicit about how the backout detection works when it appears? or do we even care?

I'm fine landing this as is, but I'd be happier if you took a couple minutes to think about if we can do better.
https://reviewboard.mozilla.org/r/10251/#review9005

> I worry people will eventually assume this _not_ appearing means something was _never_ backed out, but it was actually backed out far enough in the future. I wonder how rare this actually is? I'm sure it wouldn't be too hard for you to find that out in a local clone (how often a backout is > 100 changesets after something lands).
> 
> Is there anything we could do to make this more obvious, maybe be more explicit about how the backout detection works when it appears? or do we even care?
> 
> I'm fine landing this as is, but I'd be happier if you took a couple minutes to think about if we can do better.

I view the message as best effort. Now, how do we communicate that to people, I don't know. I say that because most commits aren't backed out. Thus the backout search won't yield anything. Therefore, backed out N>100 commits from now is undistinguishable from not backed out.

I we wanted to get fancy, we could have a cron do backout calculations and pull backout lookups from a cache. Follow-up fodder (if necessary).

> Maybe this limit should be defined somewhere better or passed in as an argument.
> 
> Luckily most firefox commit messages are super short so this won't have to parse too much text :P heh.

I'll make this into a config file lookup. Thank you for calling me out on use of magic numbers.
This is deployed. bbouvier found a bug in reviewer parsing, which I deployed after a pastebin review by smacleod. That bug could also be leading to MozReview not detecting reviewers properly.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Blocks: 1171673
Blocks: 1171719
Blocks: 1174554
Depends on: 1220669
Depends on: 1353195
You need to log in before you can comment on or make changes to this bug.