Move all of the subscription machinery to the worker

RESOLVED FIXED

Status

Websites
Basket
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: pmac, Assigned: mkelly)

Tracking

Details

(Whiteboard: [kb=1461762] )

Stop sending the token in the response to "subscribe" unless the "sync=Y" is sent along with a valid API key. Then when "sync" is not requested, do all of the subscription work in the task. This will greatly speed up subscriptions and allow us to retry more subscriptions if ET returns an error or times out.

While we're at it, let's make sure that a timeout error (likely socket.timeout) results in a task being retried.

NICE TO HAVE: We currently just retry every X minutes for X times. Would be better if we used an exponential time scale for retries. That is:

1st retry in 1min.
2nd retry in 2min.
3rd retry in 4min.
4th retry in 8min.
5th retry in 16min.
6th retry in 32min.
etc.

Basically every subsequent retry happens in 2x the previous interval. I believe there are celery examples of how to accomplish this. But using this we could retry in this way for a max of 8 times and get a really long retry time w/o bogging down the workers with failed tasks.
Blocks: 852965
Whiteboard: [kb=1461762]

Updated

4 years ago
Component: Newsletters → Basket
Product: www.mozilla.org → Other Applications
Version: Production → unspecified

Updated

4 years ago
Assignee: nobody → sancus

Updated

4 years ago
Summary: [basket] Move all of the subscription machinery to the worker → Move all of the subscription machinery to the worker
Product: Other Applications → Websites
Assignee: sancus → mkelly
(Assignee)

Comment 1

4 years ago
The retry logic is covered by bug 1073657 now.

Comment 2

4 years ago
Commits pushed to master at https://github.com/mozilla/basket

https://github.com/mozilla/basket/commit/ec2d2ebadd603a515cf1ac1a6854a90a96e99877
Fix bug 1047002: Do not generate token in subscribe view.

Includes rewritten tests for the view and utility function.

https://github.com/mozilla/basket/commit/4ed06808b728bb72e70036f8be65b8361114dfb2
Merge pull request #107 from Osmose/async-subscribe

Fix bug 1047002: Async subscribe

Updated

4 years ago
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.