1193137 - Error during submission of new add-on causing stranded add-on listing

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Developer Pages, defect, P1)

Description

[Kris Maglione [:kmag]]

This traceback is on stage, but I've seen it on production too.

This seems to happen when a developer tries to submit a new add-on, and includes a source code upload. I'm not sure what's happening, but it causes an addons table entry to be created without any owners, which prevents the developer from trying to upload the same add-on again, so it's a big deal.

Sentry URL:

http://sentry.dmz.phx1.mozilla.com/addons/addons-stage/group/28727/


Aggregate Details:

* First Seen: just now
* Last Seen: just now
* Number of tracebacks: undefined


Traceback:

Stacktrace (most recent call last):

  File "django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "newrelic/packages/wrapt/wrappers.py", line 454, in __call__
    args, kwargs)
  File "newrelic/hooks/framework_django.py", line 492, in wrapper
    return wrapped(*args, **kwargs)
  File "amo/decorators.py", line 167, in wrapper
    return f(*args, **kw)
  File "amo/decorators.py", line 154, in __call__
    return self.f(*args, **kw)
  File "amo/decorators.py", line 32, in wrapper
    return func(request, *args, **kw)
  File "devhub/views.py", line 1334, in wrapper
    return f(request, *args, **kw)
  File "devhub/views.py", line 1380, in submit_addon
    is_listed=is_listed)
  File "addons/models.py", line 537, in from_upload
    Version.from_upload(upload, addon, platforms, source=source)
  File "versions/models.py", line 158, in from_upload
    is_beta=is_beta)
  File "files/models.py", line 147, in from_upload
    file_.filename = file_.generate_filename(extension=ext or '.xpi')
  File "files/models.py", line 240, in generate_filename
    if self.version.compatible_apps:
  File "amo/__init__.py", line 74, in __get__
    value = self.func(obj)
  File "versions/models.py", line 259, in compatible_apps
    return self._compat_map(avs)
  File "versions/models.py", line 410, in _compat_map
    for av in avs:
  File "django/db/models/query.py", line 96, in __iter__
    self._fetch_all()
  File "django/db/models/query.py", line 857, in _fetch_all
    self._result_cache = list(self.iterator())
  File "caching/base.py", line 112, in __iter__
    obj = iterator.next()
  File "caching/base.py", line 180, in fetch_by_id
    cache.set_many(new)
  File "django/core/cache/backends/memcached.py", line 143, in set_many
    self._cache.set_multi(safe_data, self._get_memcache_timeout(timeout))
  File "newrelic/packages/wrapt/wrappers.py", line 490, in __call__
    args, kwargs)
  File "newrelic/api/memcache_trace.py", line 43, in _nr_wrapper_memcache_trace_
    return wrapped(*args, **kwargs)
  File "site-packages/memcache.py", line 725, in set_multi
    min_compress_len)
  File "site-packages/memcache.py", line 782, in _val_to_store_info
    pickler.dump(val)

Comment 1

[Kris Maglione [:kmag]]

Actually, it appears this is the same error that's causing bug 1193056, just with slightly different outcomes.

Comment 2

[Kris Maglione [:kmag]]

https://github.com/mozilla/olympia/pull/653

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla/olympia

https://github.com/mozilla/olympia/commit/5998137545e64550448f17fa0458529b9b8443e1
Bug 1193137: Remove `commit_manually` decorator from app version update task.

https://github.com/mozilla/olympia/commit/b1064675dddc2065a6049340ea343c9f7d2351ee
Merge pull request #653 from kmaglione/update-appsupport

Bug 1193137: Remove `commit_manually` decorator from app version update task.

Comment 5

[Kris Maglione [:kmag]]

There's a lot of weird stuff going on here.

For one, the property that's causing this caching error does:

    self.apps.select_related('versions', 'license')

But there are no such related fields.

For another, this view updates and then reads back data, but doesn't use the `@write` decorator, or pin its reads to master, which means it may well be reading and writing from databases with different states.

Going to poke around some more, but it's going to be hard to pin down without a consistent way to reproduce.

Comment 6

[Andreas Wagner [:TheOne] [use NI]]

I don't know whether Kris is still working on this, probably not. Andy, can we see if we can get it fixed some time?

Thanks.

Comment 7

[Andy McKay]

Adding to the AWS grab bag of fun queue.

Comment 8

[Jorge Villalobos [:jorgev] (he/him)]

This could have been fixed in bug 1119331.

Comment 9

[Andy McKay]

We believe this is fixed with the changes to using database transactions for everything in the developer hub.