Closed Bug 1002420 Opened 11 years ago Closed 11 years ago

Optimize the celery task that calculates the longest streaks of the users.

Categories

(Mozilla Reps Graveyard :: reps.mozilla.org, task)

Product:
Mozilla Reps Graveyard
Component:
reps.mozilla.org
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: tasos, Assigned: tasos)

Details

(Whiteboard: [kb=1347875])

Right now, the celery task that calculates the longest streak of the users, takes about 60 seconds to complete. This will become worse as new reports are added to the system. A permanent solution must be found so that this calculation can run in a reasonable time frame.
Whiteboard: [kb=1347875]
And from logs: [2014-04-25 14:32:57,009: INFO/Beat] Scheduler: Sending due task remo.reports.tasks.calculate_longest_streaks [2014-04-25 14:32:57,015: INFO/MainProcess] Got task from broker: remo.reports.tasks.calculate_longest_streaks[222726dd-3cb8-436c-a3ab-fd4fdf403a04] [2014-04-25 14:34:05,268: INFO/MainProcess] Task remo.reports.tasks.calculate_longest_streaks[222726dd-3cb8-436c-a3ab-fd4fdf403a04] succeeded in 68.203179121s: None
It makes sense to calculate streaks only for the reps that added a report during the last 24 hours. Only these reps may have changed their longest streak. This will drastically limit the execution time, down to a couple of seconds.
Status: NEW → ASSIGNED
OS: Linux → All
Hardware: x86_64 → All
Commits pushed to master at https://github.com/mozilla/remo https://github.com/mozilla/remo/commit/3580527b57eb56a6287704cdbb714d92ca92d1bf [fix bug 1002420] Optimize longest streak calculation. * Include only the users with a report in the last 24 hours, in the calculation. https://github.com/mozilla/remo/commit/8339fdfcf0850339e0c4308392a1a3d2ef58a5ed Merge pull request #727 from akatsoulas/1002420 [fix bug 1002420] Optimize longest streak task.
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Version: unspecified → next
Status: RESOLVED → VERIFIED
Whiteboard: [kb=1347875] → [kb=1347875] [qa-]
Whiteboard: [kb=1347875] [qa-] → [kb=1347875]
Version: next → 431
Task execution time down to 2 secs \o/ [2014-05-08 14:32:57,014: INFO/MainProcess] Got task from broker: remo.reports.tasks.calculate_longest_streaks[986b457b-db04-4f18-a180-510364efaa4b] [2014-05-08 14:32:59,308: INFO/MainProcess] Task remo.reports.tasks.calculate_longest_streaks[986b457b-db04-4f18-a180-510364efaa4b] succeeded in 2.28303003311s: None
Product: Mozilla Reps → Mozilla Reps Graveyard
You need to log in before you can comment on or make changes to this bug.