1280214 - Unable to supersearch all crashes within a date range

Status: RESOLVED FIXED

(Socorro :: Webapp, task)

Description

[u279076]

I am unable to supersearch for *all* crashes within a date range anymore (not sure when this started but it was within the last couple of months). As an example, https://crash-stats.mozilla.com/search/?product=Firefox&date=%3E%3D2015-06-17 should return all Firefox crashes since June 17, 2015 but this returns an internal server error.

Comment 1

[u279076]

It looks like it's the date range, if I use 2015-12-17 data is returned (6 months of data) but if I use 2015-06-17 (12 months of data) it returns an Internal Server Error. In fact, it looks like the furthest I can go back from today is 2015-11-09 (219 days).

Comment 2

[[DEACTIVATED] Adrian Gaudebert]

Yes, we only keep 6 months of data in our databases as for our data retention policy. The bug here is that it raises an internal error instead of clearly explaining that you are querying things that do not exist anymore.

Comment 3

[Matt Brandt [:mbrandt]]

Attachment: Screenshot of error message

The error appears to clearly state why the data range isn't returned. Is this related to the API?

Comment 4

[u279076]

(In reply to Matt Brandt [:mbrandt] from comment #3)
> Created attachment 8763230 [details]
> Screenshot of error message
> 
> The error appears to clearly state why the data range isn't returned. Is
> this related to the API?

Hi Matt,

I'm reopening this because I suspect you clicked the URL in comment 0 instead of entering a date. Clicking this on any day after June 15 would show the "365 day" error because the date range *is* more than 365 days. You need to update the date in the query, eg. today the 365-day cutoff is 2015-06-25.

Here are revised steps:
1) Load https://crash-stats.mozilla.com/home/product/Firefox
2) Click the Super Search link
3) Click the New Line button
4) Choose field: date
5) Choose operator: >=
6) Value: Enter a date less than 365 days but greater than 220 days from *today*
7) Click the Search button

Result:
> An unexpected error occured :(
> 
> We have been automatically informed of that error, and are working on a solution.

Based on the API documentation a date range <=365 days should be valid. Debugging this further I found a date range <=219 days still works. This is either a bug in the API, a bug in the server, or a bug in the documentation.

Comment 5

[[DEACTIVATED] Adrian Gaudebert]

Hi, the problem was due to closed Elasticsearch indices that did not get deleted by our cleanup cron job. I have manually removed them, and Super Search should now behave as expected.

Comment 6

[Matt Brandt [:mbrandt]]

Per a conversation in IRC with :adrain, I've filed bug 1280214 to track the conversation around allowed date ranges.

Comment 7

[u279076]

(In reply to Matt Brandt [:mbrandt] from comment #6)
> Per a conversation in IRC with :adrain, I've filed bug 1280214 to track the
> conversation around allowed date ranges.

Wrong bug number. Can you please fix it?

Comment 8

[Matt Brandt [:mbrandt]]

Oops :) Let's try bug 1282474