721928 - We shouldn't let the user query /daily for dates past for which we don't have datasets

Status: VERIFIED WONTFIX

(Socorro :: Webapp, task)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

WebQA has a test that runs on staging, test_that_filtering_for_a_past_date_returns_results, which does, in a nutshell:

crash_per_user = csp.header.select_report('Crashes per User')
crash_per_user.type_start_date('1995-01-01')

The issue is, that when this runs in staging, it's not an effective test until we get a full set of data with which to work.

However, talking to lonnen on IRC, we shouldn't let the user search that far back--since, according to him--"we only store 1 year anyway".

Two things--and the latter can be split out, certainly--:

1. We should limit the UI to only allow for a reasonable query set
2. We should fix the test to be reasonable, and/or run it on prod, solely

I think the URL is https://crash-stats.allizom.org/daily?form_selection=by_version&p=Firefox&v[]=12.0a1&throttle[]=100.00&v[]=11.0a2&throttle[]=100.00&v[]=10.0b5&throttle[]=100.00&v[]=9.0.1&throttle[]=10.00&hang_type=any&os[]=Windows&os[]=Mac&os[]=Linux&date_start=1995-01-01&date_end=2012-01-26&submit=Generate

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

[1] test_that_filters_for_a_past_date_returns_results: https://github.com/mozilla/Socorro-Tests/blob/master/tests/test_crash_reports.py#L181

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

Here's the Sauce Labs video: https://saucelabs.com/jobs/6ef71a38a7b95448f9333fa0f5781fca#.

Comment 3

[Robert Kaiser]

(In reply to Stephen Donner [:stephend] from comment #0)
> However, talking to lonnen on IRC, we shouldn't let the user search that far
> back--since, according to him--"we only store 1 year anyway".

*That* far back might be true, but IIRC we said we'll store the aggregations (and that's what /daily uses) indefinitely (at least as long as we can't offload that data to metrics), we just purge underlying data.

Comment 4

[Laura Thomson :laura]

I'm not sure how we know we have data if we don't try the query anyway.

Comment 5

[Stephen Donner [:stephend] Not actively reading bugmail]

I think we should probably just get rid of this test, at least Selenium-side - I don't think we want to be in the business of asserting an expectation of data from 1995-01-01, one way or the other, nor that it's particularly useful to do so?

Comment 6

[Matt Brandt [:mbrandt]]

(In reply to Stephen Donner [:stephend] from comment #5)
> I think we should probably just get rid of this test, at least Selenium-side
> - I don't think we want to be in the business of asserting an expectation of
> data from 1995-01-01, one way or the other, nor that it's particularly
> useful to do so?

QA verified wontfix - thanks captain stephend!