1233540 - [commbadge] Traceback 'ascii' codec can't encode character u'\xf3' in position 87: ordinal not in range(128)

Status: RESOLVED FIXED

(Marketplace Graveyard :: Developer Pages, defect, P2)

Description

[Andrew Williamson [:eviljeff]]

https://sentry.prod.mozaws.net/operations/zamboni-2/group/149590/ and other similar reports - its firing for me the thread; the app*; and /comm itself.

*https://marketplace.firefox.com/app/smokingted/

Comment 1

[Mathieu Pillard [:mat]]

The issue is that we are passing an unicode object for a filename to imghdr(), which uses low-level APIs so it expect (byte)strings.

We should call .encode('utf-8') before using the path or use django storage API instead of open the file and pass a File instance to imghdr().

Comment 2

[Mathieu Pillard [:mat]]

Stacktrace (most recent call last):

  File "rest_framework/views.py", line 400, in dispatch
    response = handler(request, *args, **kwargs)
  File "rest_framework/mixins.py", line 98, in list
    return Response(serializer.data)
  File "rest_framework/serializers.py", line 576, in data
    self._data = self.to_native(obj)
  File "rest_framework/serializers.py", line 355, in to_native
    value = field.field_to_native(obj, field_name)
  File "rest_framework/serializers.py", line 414, in field_to_native
    return [self.to_native(item) for item in value.all()]
  File "rest_framework/serializers.py", line 355, in to_native
    value = field.field_to_native(obj, field_name)
  File "rest_framework/serializers.py", line 414, in field_to_native
    return [self.to_native(item) for item in value.all()]
  File "rest_framework/serializers.py", line 355, in to_native
    value = field.field_to_native(obj, field_name)
  File "rest_framework/fields.py", line 337, in field_to_native
    return super(WritableField, self).field_to_native(obj, field_name)
  File "rest_framework/fields.py", line 208, in field_to_native
    value = get_component(value, component)
  File "rest_framework/fields.py", line 62, in get_component
    return val()
  File "mkt/comm/models.py", line 338, in is_image
    return imghdr.what(self.full_path()) is not None
  File "python2.7/imghdr.py", line 12, in what
    f = open(file, 'rb')

Comment 3

[Allen Short [:ashort]]

https://github.com/mozilla/zamboni/commit/3fe8db62a9b017f2d5825ce4eee261bad71984a1

Comment 5

[Madalin Cotetiu]

Can you please give us some steps to reproduce or mark this bug as [qa-] if no qa is needed.

Comment 6

[Andrew Williamson [:eviljeff]]

Try to attach a file with a non-ascii filename to a commbadge thread.  If you can upload and then access the thread its fixed.

Comment 7

[Madalin Cotetiu]

I managed to upload a file with unicode filename and the thread can still be opened. But when trying to download the attachment the file can't be found.
Please view this screencast:  http://screencast.com/t/taQxNm9Uor

Reopening this bug.

Comment 8

[Allen Short [:ashort]]

This seems to only be a problem for existing uploads not new ones, so no code fix is necessary. Reopen if there are old files we need to make accessible in production.

Comment 9

[Madalin Cotetiu]

(In reply to Allen Short [:ashort] from comment #8)
> This seems to only be a problem for existing uploads not new ones, so no
> code fix is necessary. Reopen if there are old files we need to make
> accessible in production.

We can't verify this in production. If someone can help with this and confirm it is ok in production I think we can close it.