1193056 - Uploading source package during the submission process results in SyntaxError

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Developer Pages, defect)

Description

[Andreas Wagner [:TheOne] [use NI]]

A developer of an unlisted add-on uploaded a new version and wanted to attach a sources package during the submission process. However, that failed with the following error in the error console:

common-min.js#1:4939
SyntaxError: JSON.parse: unexpected character at line 1 column 1

As this is an unlisted add-on I can't attach the code here, but I have the files and can share it to authorized persons who are investigating this bug.

Comment 2

[Kris Maglione [:kmag]]

Quoting myself from bug 1193137:

(In reply to Kris Maglione [:kmag] from bug 1193137 comment #0)
> This traceback is on stage, but I've seen it on production too.
> 
> This seems to happen when a developer tries to submit a new add-on, and
> includes a source code upload. I'm not sure what's happening, but it causes
> an addons table entry to be created without any owners, which prevents the
> developer from trying to upload the same add-on again, so it's a big deal.
> 
> Sentry URL:
> 
> http://sentry.dmz.phx1.mozilla.com/addons/addons-stage/group/28727/
> 
> 
> Aggregate Details:
> 
> * First Seen: just now
> * Last Seen: just now
> * Number of tracebacks: undefined
> 
> 
> Traceback:
> 
> Stacktrace (most recent call last):
> 
>   File "django/core/handlers/base.py", line 112, in get_response
>     response = wrapped_callback(request, *callback_args, **callback_kwargs)
>   File "newrelic/packages/wrapt/wrappers.py", line 454, in __call__
>     args, kwargs)
>   File "newrelic/hooks/framework_django.py", line 492, in wrapper
>     return wrapped(*args, **kwargs)
>   File "amo/decorators.py", line 167, in wrapper
>     return f(*args, **kw)
>   File "amo/decorators.py", line 154, in __call__
>     return self.f(*args, **kw)
>   File "amo/decorators.py", line 32, in wrapper
>     return func(request, *args, **kw)
>   File "devhub/views.py", line 1334, in wrapper
>     return f(request, *args, **kw)
>   File "devhub/views.py", line 1380, in submit_addon
>     is_listed=is_listed)
>   File "addons/models.py", line 537, in from_upload
>     Version.from_upload(upload, addon, platforms, source=source)
>   File "versions/models.py", line 158, in from_upload
>     is_beta=is_beta)
>   File "files/models.py", line 147, in from_upload
>     file_.filename = file_.generate_filename(extension=ext or '.xpi')
>   File "files/models.py", line 240, in generate_filename
>     if self.version.compatible_apps:
>   File "amo/__init__.py", line 74, in __get__
>     value = self.func(obj)
>   File "versions/models.py", line 259, in compatible_apps
>     return self._compat_map(avs)
>   File "versions/models.py", line 410, in _compat_map
>     for av in avs:
>   File "django/db/models/query.py", line 96, in __iter__
>     self._fetch_all()
>   File "django/db/models/query.py", line 857, in _fetch_all
>     self._result_cache = list(self.iterator())
>   File "caching/base.py", line 112, in __iter__
>     obj = iterator.next()
>   File "caching/base.py", line 180, in fetch_by_id
>     cache.set_many(new)
>   File "django/core/cache/backends/memcached.py", line 143, in set_many
>     self._cache.set_multi(safe_data, self._get_memcache_timeout(timeout))
>   File "newrelic/packages/wrapt/wrappers.py", line 490, in __call__
>     args, kwargs)
>   File "newrelic/api/memcache_trace.py", line 43, in
> _nr_wrapper_memcache_trace_
>     return wrapped(*args, **kwargs)
>   File "site-packages/memcache.py", line 725, in set_multi
>     min_compress_len)
>   File "site-packages/memcache.py", line 782, in _val_to_store_info
>     pickler.dump(val)

Comment 3

[Kris Maglione [:kmag]]

It looks like there are multiple problems here. The add-on I tested was broken because of the issue in comment 2, but there's also a problem with generating names for source package files:

http://sentry.mktmon.services.phx1.mozilla.com/mkt/addonsmozillaorg/group/15990/

The problem seems to be that the add-on's slug contains non-ascii characters.

So I'm going to undup bug 1193137 and leave this to address the above error.

Comment 4

[Mathieu Agopian [:magopian]]

PR: https://github.com/mozilla/olympia/pull/667

Comment 5

[Mathieu Agopian [:magopian]]

STR: Make sure that you can properly submit an add-on with some non-ascii characters, even if you choose to upload a source file (any file will do, whatever its name, it's the add-on slug that needs to have non-ascii characters).

Comment 6

[[github robot]]

Commits pushed to master at https://github.com/mozilla/olympia

https://github.com/mozilla/olympia/commit/155ed50f8fed56f608b947634d33a4eca2dceb9c
Properly deal with unicode filenames for source uploads (bug 1193056)

https://github.com/mozilla/olympia/commit/4d9095b5e321855a84389d9ca0452302cc6fdca9
Merge pull request #667 from magopian/1193056-unicode-source-filename

Properly deal with unicode filenames for source uploads (bug 1193056)

Comment 7

[Mathieu Agopian [:magopian]]

While QAing for this, please make sure that the files uploaded can be properly downloaded in different places (the devhub pages, the reviewer pages, the public pages...) Thanks!