1617835 - Multiple seconds hangs on treeherder when selecting a specific commit

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Frontend, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I can see slowness on Treeherder in various areas but here one example which caused a more than 20s hang when just selecting a specific changeset from mozilla-central.

Steps done:

1. Open https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=
2. Scroll a bit down and select one commit by clicking on its landing time

After step 2 I have seen that huge hang. Here the appropriate gecko profile from the last seconds of the hang, and when it was working again: https://perfht.ml/2w1ppD9

90% of the time (11 seconds) is spent in CopyDataPropertiesUnfiltered that is called from updateRange.

Comment 1

[Sarah Clements [:sclements]]

I'll be profiling and improving some of our APIs to help improve performance. I don't know if there are other UI improvements we need to make. Cam, is this a known issue?

Comment 2

[Cameron Dawson [:camd]]

This specific issue is not on my radar. But there is a semi-related Bug 1594565.

That being said, we do likely have slow areas that need addressing. Sometimes this can just be a result of network slowness. But if this is persistent (which it looks like it is) then we may have a missing index, or... well... we need to investigate. :)

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It's not happening all the time. Do we make use of some kind of caching?

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Problematic code identified. The object holding the mapping of job IDs to the jobs themselves got iteratively regenerated - the script walked over all jobs currently shown (with Array.prototype.reduce) and for each (sic!) job of the push which shall be shown, a new object got generated based on the old one + the new job id and data. Based on testing with mozilla-central and the default 10 pushes shown, this translates into ~90k jobs and slow page warnings (execution up to 46s measured). The new code takes one object and sets the id and data pairs iteratively in 100-200ms.

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/7222

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://github.com/mozilla/treeherder/commit/36967cc9b7c92fa47cda6294b225f84a7ac235e9