1149187 - [stage][regression] Signature reports are slow to load - sometimes timeout

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Matt Brandt [:mbrandt]]

Not sure what changed on stage - The reports for signatures are slow to load. I've also seen a few timeouts where socorro displays an error stating the information could not load.

Our automated tests flagged this as an issue, information in the reports is taking up to +180 seconds to load at times.

Steps to reproduce:
0. choose a signature from a top crashers report
1. navigate to the #tab-reports tab
2. navigate to the #tab-correlations tab

Actual:
Signature reports are slow to load

The reports tab sometimes times out and the error "Unable to load ..." is displayed.

The correlations tab consistently displays the error, "Unable to load correlations. Please try again later."

Comment 1

[Robert Helmer [:rhelmer]]

Looking over commits that landed since last push, this sticks out (bug 1143724):
https://github.com/mozilla/socorro/commit/1251574bc3960ae694b0c1bc1b9a37fba34b33ad

Adrian, this seems to have regressed perf quite badly on stage, can you take a look?

Comment 2

[Peter Bengtsson [:peterbe]]

I can see this repeatedly using the ES stage cluster (using my own middleware).

Comment 3

[Peter Bengtsson [:peterbe]]

Most definitely the problem is with this: 
https://gist.github.com/peterbe/b21ec2ce67ea959232d8

We're doing SuperSearch queries that are far far too large. 109Mb!!

It's only speculation but I bet this is related to the recent surge in these types of errors: https://errormill.mozilla.org/webtools/socorro-stage/group/395678/

What we need to do is figure out what data is needed from the SuperSearch API call and ask SuperSearch only for the fields we need and nothing more.

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

Bug 1144569, working on it.

Comment 5

[Justin Potts [:justinpotts]]

Fixed; verified through automation in test_selecting_one_version_doesnt_show_other_versions.