1728210 - stop using celery

Status: RESOLVED FIXED

(Tecken :: General, task, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Tecken current uses celery, but only just barely. It uses it for the following things:

1. populates uploads_created model after processing an upload (bookkeeping)
2. task for invalidating symbolicate cache (cache invalidation)
3. task for handling errors when storing missing symbol information (bookkeeping)

Items 1 and 2 are spawned after a successful upload. I think we should move the first one to part of the upload handling code. Seems weird we're spending the time to spawn a celery task and then create a db record in background processing.

Item 2 could go away now. Sure, symbolication might be slightly wrong for a small bit of time if someone re-uploads a file, but that doesn't happen often and Socorro doesn't invalidate its cache, so I think it's ok for symbolication to not update its cache especially if it lets us ditch this celery task.

Item 3 only happens in a celery task if the system is having problems such that the download view couldn't record the missing symbol. I do think it's important that the download API not fail if it can't record the missing symbol, but if Tecken misses some missing symbols today, that's ok. It'll see them tomorrow. This isn't a critical part of the service that has to be up 99% of the time.

Given all that, I vote we remove these tasks and get rid of celery.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this block the GCP migration and grabbing it to do. It looks pretty straight-forward, but it'll require some ops work so we'll have to do it in a few steps and finish it after the codefreeze is lifted.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 2400: bug 1728210: remove celery tasks

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #2400: "bug 1728210: remove celery tasks" in b27abde.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

The changes to remove celery tasks went to production in bug #1733473.

Next step is to dismantle the celery infrastructure.

After that, we can remove celery from the requirements and docker-compose setup.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I looked at the Grafana data in a Jupyter notebook:

https://github.com/willkg/socorro-jupyter/blob/main/notebooks/bug_1728210_upload_timings.ipynb

There's definitely an increase in the times (as expected), but it doesn't feel like it's a big enough deal to roll back.

I wrote up https://mozilla-hub.atlassian.net/browse/DSRE-277 to dismantle the celery bits of the infrastructure. We can close this out now.