1214387 - [sync] Configure celery to avoid syncing the same project simultaneously

Status: RESOLVED FIXED

(Webtools Graveyard :: Pontoon, defect, P2)

Description

[Osmose [:osmose, :mkelly]]

If a sync job takes a really long time, it is theoretically possible that the project will get scheduled for another sync before the first sync is finished, which will cause all sorts of problems as two sync jobs syncing the same project will share the same filesystem and overwrite their changes and so on. 

We should investigate to see if celery is able to prevent a task signature identical to a currently-running signature from being scheduled or run at the same time. http://docs.celeryproject.org/en/latest/tutorials/task-cookbook.html#ensuring-a-task-is-only-executed-one-at-a-time looks like it might be a solution.

Comment 1

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/8e1a35c7d88a7dd882e4d33a47ff4326903e9c0a
Fix bug 1214387. Added decorator to run only one task at time.

Comment 2

[Osmose [:osmose, :mkelly]]

So we totally missed during review that the code as implemented made us unable to run more than one project sync task at a time, when the intent was that we should not be able to sync the same project simultaneously. We also missed that the decorator wasn't added to the repo sync task as well.

mathjazz is about to commit a fix that removes the decorator from the project sync task.

Comment 3

[Matjaz Horvat [:mathjazz]]

https://github.com/mozilla/pontoon/commit/51d056bab94f29a61bc349fecba690c7b2406808