652880 - Long-running query/timeout on production, just by clicking the Filter Crash Reports button on https://crash-stats.mozilla.com/query

Status: VERIFIED FIXED

(Socorro :: Webapp, task)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

https://crash-stats.mozilla.com/query/query?product=Firefox&version=ALL%3AALL&range_value=1&range_unit=weeks&date=04%2F26%2F2011+11%3A21%3A17&query_search=signature&query_type=contains&query=&reason=&build_id=&process_type=any&hang_type=any&do_query=1 is timing out (even after 30 seconds).

It's making our automation fail: http://qa-selenium.mv.mozilla.com:8080/job/socorro.prod/34/console

STR:

1. Load https://crash-stats.mozilla.com/query
2. Click "Filter Crash Reports"

Comment 1

[[:jberkus] Josh Berkus]

The problem is that the Web UI is running this query:

SELECT COUNT(DISTINCT reports.signature) as count  FROM   reports WHERE  (reports.product = 'Firefox') AND reports.date_processed BETWEEN CAST('04/26/2011 11:14:26' AS TIMESTAMP WITHOUT TIME ZONE) - CAST('1 weeks' AS INTERVAL) AND CAST('04/26/2011 11:14:26' AS TIMESTAMP WITHOUT TIME ZONE)"

This is very wrong.  The WebUI should NEVER be running aggregates on the reports table, ever.  It should be querying top_crashes_by_signature instead.

Over to you, Brandon.

Comment 2

[Laura Thomson :laura]

Problem is, this is the workaround we use for running aggregates when the data we want isn't in TCBS.  We're going to have to fix that first.

Comment 3

[[:jberkus] Josh Berkus]

What do you mean "data isn't in TCBS"?

Comment 4

[Laura Thomson :laura]

TCBS doesn't let you query for a particular build, which is now important under the four channel model.  I'm trying to decide whether we should add that now or just wait until we replace it in early Q3.  I'm leaning towards the latter.

Comment 5

[Laura Thomson :laura]

Lars and I finally have a plan to fix this properly, will take the next couple of weeks though.  No way it can make 1.7.8, sorry WebQA.

Comment 6

[[:jberkus] Josh Berkus]

Seems like we need to have the channels in PostgreSQL.

Comment 7

[Laura Thomson :laura]

Will be part of the 2.2 focus.

Comment 8

[Laura Thomson :laura]

We now have all the pre-requisites in place to actually fix this.

Comment 12

[[:jberkus] Josh Berkus]

So, this long-running issue is one we can fix.  We've been putting it off for the last year because "Elastic search is coming".  Might it be worth taking a few other steps?

https://wiki.mozilla.org/Socorro/Speeding_Up_Postgres_Search

Comment 13

[Lonnen :lonnen]

Now that we're on elastic search this no longer appears to be an issue, please reopen if its still happening.

Comment 14

[Stephen Donner [:stephend] Not actively reading bugmail]

This use-case seems to work just fine for me on prod, now:

https://crash-stats.mozilla.com/query/?product=Firefox&version=ALL%3AALL&range_value=1&range_unit=weeks&date=08%2F23%2F2013+18%3A00%3A00&query_search=signature&query_type=contains&query=&reason=&release_channels=&build_id=&process_type=any&hang_type=any