Closed Bug 735884 Opened 12 years ago Closed 12 years ago

Pin celery tasks to master when writing

Categories

(addons.mozilla.org Graveyard :: Developer Pages, defect, P1)

x86
macOS
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kumar, Assigned: kumar)

References

Details

If a celery task gets spawned soon after a DB write then it needs to query the master DB.

Now that task exceptions show up in email, this out of sync scenario seems to happen occasionally.


Here is an example:

Task devhub.tasks.validator with id ba147472-2034-446f-bcdd-98dfa5b3c651 raised exception:
"ObjectDoesNotExist('FileUpload matching query does not exist.',)"


Task was called with args: ('a9491722be514f4c9c6791c7b1d9921f',) kwargs: {}.

The contents of the full traceback was:

Traceback (most recent call last):
 File "/data/www/addons-dev.allizom.org/zamboni/vendor/lib/python/celery/execute/trace.py", line 34, in trace
   return cls(states.SUCCESS, retval=fun(*args, **kwargs))
 File "/data/www/addons-dev.allizom.org/zamboni/vendor/lib/python/celery/task/base.py", line 241, in __call__
   return self.run(*args, **kwargs)
 File "/data/www/addons-dev.allizom.org/zamboni/vendor/lib/python/celery/app/__init__.py", line 141, in run
   return fun(*args, **kwargs)
 File "/data/www/addons-dev.allizom.org/zamboni/vendor/src/nuggets/celeryutils.py", line 35, in wrapped
   return fun(*args, **kw)
 File "/data/www/addons-dev.allizom.org/zamboni/apps/amo/decorators.py", line 112, in wrapper
   return f(*args, **kw)
 File "/data/www/addons-dev.allizom.org/zamboni/apps/amo/decorators.py", line 104, in wrapper
   return f(*args, **kw)
 File "/data/www/addons-dev.allizom.org/zamboni/apps/devhub/tasks.py", line 43, in validator
   upload = FileUpload.objects.get(pk=upload_id)
 File "/data/www/addons-dev.allizom.org/zamboni/vendor/src/django/django/db/models/manager.py", line 132, in get
   return self.get_query_set().get(*args, **kwargs)
 File "/data/www/addons-dev.allizom.org/zamboni/vendor/src/django/django/db/models/query.py", line 349, in get
   % self.model._meta.object_name)
DoesNotExist: FileUpload matching query does not exist.
Assignee: nobody → kumar.mcmillan
Priority: -- → P3
Target Milestone: --- → 6.4.7
We actually get several of these a day, bumping up priority
Priority: P3 → P1
I decided this wasn't worth the effort. Instead, we can use the existing @write decorator to pin tasks like this to the master DB.

Fixed: https://github.com/mozilla/zamboni/commit/3edf647
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.