1208536 - Run per-project sync jobs in parallel

Status: RESOLVED FIXED

(Webtools Graveyard :: Pontoon, defect)

Description

[Osmose [:osmose, :mkelly]]

Individual projects can be synced independently of eachother, so we should be able to trigger multiple parallel sync processes instead of a single sync process that runs through the projects in order. I expect that this will help reduce the sync duration even further.

The current plan is to set up Celery and have the sync_projects management command trigger a set of Celery tasks. Locally, we can just run the site with CELERY_ALWAYS_EAGER to allow people to work on the site without running celery. On Heroku, we can set up a few dynos as celery worker processes.

Comment 1

[[github robot]]

Commit pushed to master at https://github.com/mozilla/pontoon

https://github.com/mozilla/pontoon/commit/3dfda07623f3a7fe142eaede27ddce1bc7d7006f
Bug 1208536: Run sync_projects in parallel.

Also removes the deleted field from Translations and returns
to deleting Translations directly. Now that we track changes
via ChangedEntityLocale, it is no longer needed, and removing
it allows us to parallelize sync_projects without coordinating
to remove deleted translations once we're done.