1009016 - Ouija should only retain 6 months of data

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Dan Minor [:dminor]]

Ouija (http://www.github.com/dminor/ouija, running at http:54.215.155.53/index.html) is a tool for analyzing test failure data.

Currently we are retaining all data we collect, but we only really need 6 months of data. The updatedb script (https://github.com/dminor/ouija/blob/master/src/updatedb.py) is run by a cron job daily to collect fresh data.

The clearResults function currently deletes any data from the same range that we are updating to avoid duplicate results. It could delete old data at the same time.

Comment 1

[Juan C. Mojica Roa (:jcmojicar)]

Attachment: added a delete sql statement to delete test data older than 183 days

This will delete tests data older than 183 days. If more accuracy is needed we could use dateutil.relativedelta to work exactly with 6 months.

If you have any comments please let me know.

Comment 2

[Dan Minor [:dminor]]

Comment on attachment 8428205 [details] [diff] [review]
added a delete sql statement to delete test data older than 183 days

Review of attachment 8428205 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for looking at this! This looks good, but I would prefer to see the two deletes combined with an 'or' statement so we only have to execute one query instead of two - something like date < TOO_OLD or date > DELTA.

Comment 3

[Juan C. Mojica Roa (:jcmojicar)]

Attachment: 0001-join-the-deletes-of-delta-days-data-and-data-older-t.patch

I joined the deletes for the delta days and the data older than 183 days into one query as requested.

Please let me know if this works for you or if you think there is something else that needs to be changed.

Comment 4

[Dan Minor [:dminor]]

Comment on attachment 8428951 [details] [diff] [review]
0001-join-the-deletes-of-delta-days-data-and-data-older-t.patch

Looks great, thank you for contributing!

Comment 5

[Dan Minor [:dminor]]

Fixed in: https://github.com/dminor/ouija/commit/1ede78443039a61259a070e0bd08c35866efb473