Move all of the subscription machinery to the worker



4 years ago
4 years ago


(Reporter: pmac, Assigned: mkelly)



(Whiteboard: [kb=1461762] )

Stop sending the token in the response to "subscribe" unless the "sync=Y" is sent along with a valid API key. Then when "sync" is not requested, do all of the subscription work in the task. This will greatly speed up subscriptions and allow us to retry more subscriptions if ET returns an error or times out.

While we're at it, let's make sure that a timeout error (likely socket.timeout) results in a task being retried.

NICE TO HAVE: We currently just retry every X minutes for X times. Would be better if we used an exponential time scale for retries. That is:

1st retry in 1min.
2nd retry in 2min.
3rd retry in 4min.
4th retry in 8min.
5th retry in 16min.
6th retry in 32min.

Basically every subsequent retry happens in 2x the previous interval. I believe there are celery examples of how to accomplish this. But using this we could retry in this way for a max of 8 times and get a really long retry time w/o bogging down the workers with failed tasks.
Blocks: 852965
Whiteboard: [kb=1461762]


4 years ago
Component: Newsletters → Basket
Product: → Other Applications
Version: Production → unspecified


4 years ago
Assignee: nobody → sancus


4 years ago
Summary: [basket] Move all of the subscription machinery to the worker → Move all of the subscription machinery to the worker
Product: Other Applications → Websites
Assignee: sancus → mkelly

Comment 1

4 years ago
The retry logic is covered by bug 1073657 now.

Comment 2

4 years ago
Commits pushed to master at
Fix bug 1047002: Do not generate token in subscribe view.

Includes rewritten tests for the view and utility function.
Merge pull request #107 from Osmose/async-subscribe

Fix bug 1047002: Async subscribe


4 years ago
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.