Closed
Bug 1136809
Opened 10 years ago
Closed 10 years ago
[heartbeat] MultipleObjectsReturned: get() returned more than one Answer -- it returned 2!
Categories
(Input Graveyard :: Backend, defect, P1)
Input Graveyard
Backend
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: willkg, Assigned: willkg)
Details
(Whiteboard: u=gregg c=heartbeat p=2 s=input.2015q1)
Saw one traceback where it looks like we do a .get() for a flow_id that returns two records.
Needs investigation.
Traceback:
Internal Server Error: /api/v2/hb/
Traceback (most recent call last):
File "/data/www/input.mozilla.org/input/vendor/src/django/django/core/handlers/base.py", line 111, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/data/www/input.mozilla.org/venv/lib/python2.7/site-packages/newrelic-2.38.0.31/newrelic/hooks/framework_django.py", line 497, in wrapper
return wrapped(*args, **kwargs)
File "/data/www/input.mozilla.org/input/fjord/base/utils.py", line 457, in decorated_func
response = f(request, *args, **kwargs)
File "/data/www/input.mozilla.org/input/vendor/src/django/django/views/decorators/csrf.py", line 57, in wrapped_view
return view_func(*args, **kwargs)
File "/data/www/input.mozilla.org/input/vendor/src/django/django/views/generic/base.py", line 69, in view
return self.dispatch(request, *args, **kwargs)
File "/data/www/input.mozilla.org/venv/lib/python2.7/site-packages/newrelic-2.38.0.31/newrelic/hooks/component_djangorestframework.py", line 27, in _nr_wrapper_APIView_dispatch_
return wrapped(*args, **kwargs)
File "/data/www/input.mozilla.org/input/vendor/src/django-rest-framework/rest_framework/views.py", line 403, in dispatch
response = self.handle_exception(exc)
File "/data/www/input.mozilla.org/input/vendor/src/django-rest-framework/rest_framework/views.py", line 400, in dispatch
response = handler(request, *args, **kwargs)
File "/data/www/input.mozilla.org/input/fjord/heartbeat/views.py", line 58, in post
flow_id=valid_data.flow_id
File "/data/www/input.mozilla.org/input/vendor/src/django/django/db/models/manager.py", line 92, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/data/www/input.mozilla.org/input/vendor/src/django/django/db/models/query.py", line 361, in get
num if num <= MAX_GET_RESULTS else 'more than %s' % MAX_GET_RESULTS
MultipleObjectsReturned: get() returned more than one Answer -- it returned 2!
Also interesting is that the error email has this line in it:
POST:<could not parse>,
I suspect that's because it's not POST data but rather JSON data. We might need to tweak Django error handling so that it shows better errors for API-related requests.
Assignee | ||
Comment 1•10 years ago
|
||
Grabbing this to look into now.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Assignee | ||
Comment 2•10 years ago
|
||
I spun the POST:<could not parse> bit into bug #1136840.
The bit of code is question is this:
ans = Answer.objects.get(
person_id=valid_data.person_id,
survey_id=valid_data.survey_id,
flow_id=valid_data.flow_id
)
i.e. there are two hb answers with the same (person_id, survey_id, flow_id) combination.
There's no db integrity for making that tuple unique. It's entirely possible there's a race condition where two posts come in at the "same time" and both create answers in the db.
I think what we want to do is this:
1. write a data migration to delete any duplicates
2. add a unique_together thing to the model
3. change the post code so that if it hits an integrity error of this kind, it tries an update or something
Assignee | ||
Comment 3•10 years ago
|
||
In a PR: https://github.com/mozilla/fjord/pull/500
I skipped step 3 for now. I want to see what errors we get if we get any before I go fix them.
Assignee | ||
Comment 4•10 years ago
|
||
Landed in master: https://github.com/mozilla/fjord/commit/2d3d05ae430946adda30fe09782df970fbc01bde
Will push out in the next deploy.
Assignee | ||
Updated•10 years ago
|
Whiteboard: u=gregg c=heartbeat p= s=input.2015q1 → u=gregg c=heartbeat p=2 s=input.2015q1
Assignee | ||
Comment 5•10 years ago
|
||
Pushed this to prod just now.
Pretty sure this should be fixed now. If there are more instances, I'll see them in the error email and figure out whether to create a new bug or reopen this one thing.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Updated•8 years ago
|
Product: Input → Input Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•