Recalculate me too votes this past week for top 100 (or more) threads

VERIFIED FIXED in 2012-01-31

6 years ago
6 years ago


(Reporter: Cww, Assigned: rrosario)



Firefox Tracking Flags

(Not tracked)


(Whiteboard: u=sumo-team c=questions s=2012.2 p=2 [qa-])



6 years ago made it so votes this past week are updated only on new votes.  However if a thread suddenly stops getting votes, the number of votes in the past week can be very wrong (and very high) and our list of most voted on threads is misleading.

Since croning a recalculation for all the threads was taking down celery, I think a good compromise is to just do a recalculation for the top 100 threads (or 1000 if we can handle that) every day.  That way the list of "most requested" will always be fairly up to date until page 5 (or 50).
Maybe for correctness sake, and if it's reasonable we can recalculate this for every thread that got any action within the last two weeks? The number should be more reasonable than all threads and still cover all threads that dropped from one week to the other.
Whiteboard: u=sumo-team c=questions s=2012.2 p=

Comment 2

6 years ago
A better reformulation of what we "should" do is:

Calculate all threads for which "posts in the last week" is nonzero.

That number is 48K right now.

If we didn't have the bug, it'd be 17K. So if we think we can handle ~ 20K thread recalculations a day, let's do this (and do a one-time move).  If we can't, we should think about cutting it back.


6 years ago
OS: Mac OS X → All
Hardware: x86 → All

Comment 3

6 years ago
Going with Comment 2
Whiteboard: u=sumo-team c=questions s=2012.2 p= → u=sumo-team c=questions s=2012.2 p=2

Comment 4

6 years ago
Looking at the crontab, it looks like we are *should* be already updating weekly votes daily. It is commented out under twice per week though.

# Once per day.
0 16 * * * {{ cron }} reload_wiki_traffic_stats
40 1 * * * {{ cron }} update_weekly_votes
42 0 * * * {{ cron }} update_top_contributors
0 21 * * * {{ cron }} cache_most_unhelpful_kb_articles
47 2 * * * {{ cron }} remove_expired_registration_profiles

# Twice per week.
#05 01 * * 1,4 {{ cron }} update_weekly_votes
Ricky, what does update_weekly_votes do exactly? does it update the vote count for *all* questions? The issue seemed to be that only questions with new votes were recalculated, so assuming that question 'foo' got 100 votes 30 days ago and none since, it would still be displayed with 'foo got 100 votes last week' even today. 

Cheng, do you have an example for this behavior?

Comment 6

6 years ago
My bad, I should I have looked closer at the query filter. We only update questions created in the past two weeks :-). So we just need to change this to what Cheng suggested.


6 years ago
Priority: -- → P1


6 years ago
Assignee: nobody → rrosario

Comment 7

6 years ago
ugh, comment #2 should read "votes in the past week is nonzero"

Comment 8

6 years ago
This was really a 1 pointer. I didn't realize that the tasks were still there and we just had to tweak the query of what questions to update.

I have no idea how to verify this on stage so marking as [qa-]. Cww can verify it after it lands on prod.
Last Resolved: 6 years ago
Resolution: --- → FIXED
Whiteboard: u=sumo-team c=questions s=2012.2 p=2 → u=sumo-team c=questions s=2012.2 p=2 [qa-]
Target Milestone: --- → 2012-01-31
Closed [qa-]
You need to log in before you can comment on or make changes to this bug.