1174554 - hg shortlog loading slowly lately due to milestone lookups happening for each changeset

Status: VERIFIED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect, P1)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Filing this on behalf of the post in the URL field.

The shortlog for mozilla inbound is taking 14 seconds for me to load it. Similar load times for mozilla-central's shortlog.

Comm-central is only taking 1.2 seconds to load its shortlog.

The "last 10 days or so" lines up roughly when the mercurial upgrades were deployed, right?

Comment 1

[Gregory Szorc [:gps]]

I've noticed this too. I'll investigate.

Comment 2

[Gregory Szorc [:gps]]

This is a regression from bug 1171321. Something in the hgmo extension. I'm not sure what because `hg serve --profile` doesn't reveal anything useful for HTTP requests. But if I had to venture a guess, it would be the backout detection.

Comment 3

[Gregory Szorc [:gps]]

This is a result of obtaining the milestone for each changeset. Reading the milestone requires loading the manifest to resolve the filelog revision. Fixing this should be a matter of moving the milestone lookup to only happen in the changeset view, not the changelist view.

Comment 4

[Gregory Szorc [:gps]]

Attachment: MozReview Request: hgmo: don't resolve milestone when only cheap attributes are wanted (bug 1174554); r?smacleod

hgmo: don't resolve milestone when only cheap attributes are wanted (bug 1174554); r?smacleod

shortlog pages may request info on dozens of changesets. Resolving
manifests can take a few hundred milliseconds. This can result in page
load times of over 20 seconds.

Defer milestone resolution to the "onlycheap" section to avoid this
penalty.

Comment 5

[Steven MacLeod [:smacleod]]

Comment on attachment 8622558 [details]
MozReview Request: hgmo: don't resolve milestone when only cheap attributes are wanted (bug 1174554); r?smacleod

https://reviewboard.mozilla.org/r/11391/#review9759

::: hgext/hgmo/__init__.py:59
(Diff revision 1)
> -    # fast, so just do it.
> +    # fast, so just do it. However, we need to defer this to the "onlycheap"
> +    # section because resolving manifests is relatively slow and resolving

I find it strange we're calling the "onlycheap" section things we do when onlycheap = False. I'd say `moving *out* of the "onlycheap"` makes more sense here (and in the commit message).

Comment 6

[Gregory Szorc [:gps]]

https://hg.mozilla.org/hgcustom/version-control-tools/rev/24ab764db8f1

Comment 7

[Gregory Szorc [:gps]]

This is deployed but it won't take effect until the HTTP server processes refresh, which will take a few hours unless we force a restart. I'm inclined to wait.

FWIW, page load times should go from ~20s to ~0.250s.

Comment 8

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Seems to be working, shortlog loads much quicker.