716645 - Recalculate me too votes this past week for top 100 (or more) threads

Status: VERIFIED FIXED

(support.mozilla.org :: Questions, task, P1)

Description

[[:Cww]]

https://bugzilla.mozilla.org/show_bug.cgi?id=664456 made it so votes this past week are updated only on new votes.  However if a thread suddenly stops getting votes, the number of votes in the past week can be very wrong (and very high) and our list of most voted on threads is misleading.

Since croning a recalculation for all the threads was taking down celery, I think a good compromise is to just do a recalculation for the top 100 threads (or 1000 if we can handle that) every day.  That way the list of "most requested" will always be fairly up to date until page 5 (or 50).

Comment 1

[Kadir Topal [:atopal]]

Maybe for correctness sake, and if it's reasonable we can recalculate this for every thread that got any action within the last two weeks? The number should be more reasonable than all threads and still cover all threads that dropped from one week to the other.

Comment 2

[[:Cww]]

A better reformulation of what we "should" do is:

Calculate all threads for which "posts in the last week" is nonzero.

That number is 48K right now.

If we didn't have the bug, it'd be 17K. So if we think we can handle ~ 20K thread recalculations a day, let's do this (and do a one-time move).  If we can't, we should think about cutting it back.

Comment 3

[Ricky Rosario [:rrosario, :r1cky]]

Going with Comment 2

Comment 4

[Ricky Rosario [:rrosario, :r1cky]]

Looking at the crontab, it looks like we are *should* be already updating weekly votes daily. It is commented out under twice per week though.

```
# Once per day.
0 16 * * * {{ cron }} reload_wiki_traffic_stats
40 1 * * * {{ cron }} update_weekly_votes
42 0 * * * {{ cron }} update_top_contributors
0 21 * * * {{ cron }} cache_most_unhelpful_kb_articles
47 2 * * * {{ cron }} remove_expired_registration_profiles

# Twice per week.
#05 01 * * 1,4 {{ cron }} update_weekly_votes
```

Comment 5

[Kadir Topal [:atopal]]

Ricky, what does update_weekly_votes do exactly? does it update the vote count for *all* questions? The issue seemed to be that only questions with new votes were recalculated, so assuming that question 'foo' got 100 votes 30 days ago and none since, it would still be displayed with 'foo got 100 votes last week' even today. 

Cheng, do you have an example for this behavior?

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

My bad, I should I have looked closer at the query filter. We only update questions created in the past two weeks :-). So we just need to change this to what Cheng suggested.

Comment 7

[[:Cww]]

ugh, comment #2 should read "votes in the past week is nonzero"

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

This was really a 1 pointer. I didn't realize that the tasks were still there and we just had to tweak the query of what questions to update.

https://github.com/mozilla/kitsune/commit/98ef667e56b798447d0c97678fa44ad462883053

I have no idea how to verify this on stage so marking as [qa-]. Cww can verify it after it lands on prod.

Comment 9

[Rebecca Billings [:rbillings]]

Closed [qa-]