921243 - [stage] Uploading specific profile image causes a 500 error

Status: VERIFIED FIXED

(Participation Infrastructure :: Phonebook, defect)

Description

[Matt Brandt [:mbrandt]]

Attachment: smokey-meme.jpg

Steps to reproduce:
0. goto stage, /user/edit
1. change profile image to the attached jpg


Traceback (most recent call last):

  File "/data/www/mozillians.allizom.org/mozillians/vendor/lib/python/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.10.2.38/newrelic/api/object_wrapper.py", line 220, in __call__
    self._nr_instance, args, kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.10.2.38/newrelic/hooks/framework_django.py", line 475, in wrapper
    return wrapped(*args, **kwargs)

  File "/data/www/mozillians.allizom.org/mozillians/vendor/lib/python/django/views/decorators/cache.py", line 89, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)

  File "/data/www/mozillians.allizom.org/mozillians/mozillians/phonebook/views.py", line 73, in home
    return render(request, 'phonebook/home.html')

  File "/data/www/mozillians.allizom.org/mozillians/vendor/lib/python/django/shortcuts/__init__.py", line 44, in render
    return HttpResponse(loader.render_to_string(*args, **kwargs),

  File "/data/www/mozillians.allizom.org/mozillians/vendor/lib/python/django/template/loader.py", line 176, in render_to_string
    return t.render(context_instance)

  File "/data/www/mozillians.allizom.org/mozillians/vendor/src/jingo/jingo/__init__.py", line 189, in render
    return super(Template, self).render(context_dict)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.10.2.38/newrelic/api/object_wrapper.py", line 220, in __call__
    self._nr_instance, args, kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.10.2.38/newrelic/api/function_trace.py", line 82, in dynamic_wrapper
    return wrapped(*args, **kwargs)

  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)

  File "/data/www/mozillians.allizom.org/mozillians/mozillians/phonebook/templates/phonebook/home.html", line 1, in top-level template code
    {% extends "base.html" %}

  File "/data/www/mozillians.allizom.org/mozillians/templates/base.html", line 156, in top-level template code
    {% block content_wrapper %}

  File "/data/www/mozillians.allizom.org/mozillians/templates/base.html", line 159, in block "content_wrapper"
    {% block content %}{% endblock %}

  File "/data/www/mozillians.allizom.org/mozillians/mozillians/phonebook/templates/phonebook/home.html", line 108, in block "content"
    <img src="{{ user.userprofile.get_photo_url('150x150') }}" alt="{{ user.username }}">

  File "/data/www/mozillians.allizom.org/mozillians/mozillians/users/models.py", line 393, in get_photo_url
    return self.get_photo_thumbnail(geometry, **kwargs).url

  File "/data/www/mozillians.allizom.org/mozillians/mozillians/users/models.py", line 380, in get_photo_thumbnail
    return get_thumbnail(self.photo, geometry, **kwargs)

  File "/data/www/mozillians.allizom.org/mozillians/vendor-local/lib/python/sorl/thumbnail/shortcuts.py", line 8, in get_thumbnail
    return default.backend.get_thumbnail(file_, geometry_string, **options)

  File "/data/www/mozillians.allizom.org/mozillians/vendor-local/lib/python/sorl/thumbnail/base.py", line 61, in get_thumbnail
    thumbnail)

  File "/data/www/mozillians.allizom.org/mozillians/vendor-local/lib/python/sorl/thumbnail/base.py", line 86, in _create_thumbnail
    image = default.engine.create(source_image, geometry, options)

  File "/data/www/mozillians.allizom.org/mozillians/vendor-local/lib/python/sorl/thumbnail/engines/base.py", line 16, in create
    image = self.colorspace(image, geometry, options)

  File "/data/www/mozillians.allizom.org/mozillians/vendor-local/lib/python/sorl/thumbnail/engines/base.py", line 34, in colorspace
    return self._colorspace(image, colorspace)

  File "/data/www/mozillians.allizom.org/mozillians/vendor-local/lib/python/sorl/thumbnail/engines/pil_engine.py", line 56, in _colorspace
    return image.convert('RGB')

  File "/usr/lib64/python2.6/site-packages/PIL/Image.py", line 679, in convert
    self.load()

  File "/usr/lib64/python2.6/site-packages/PIL/ImageFile.py", line 201, in load
    raise IOError("image file is truncated (%d bytes not processed)" % len(b))

IOError: image file is truncated (75 bytes not processed)

Comment 1

[Giorgos Logiotatidis [:giorgos]]

I can reproduce this for this image, but not for all images. 

I'll investigate further.

Comment 2

[Giorgos Logiotatidis [:giorgos]]

Matt is this a new test case?

Comment 3

[Giorgos Logiotatidis [:giorgos]]

I checkout c39b5ec which is currently on prod and I can still reproduce this locally, while I can't reproduce this on prod. I suspect that maybe a different version of PIL can cause this error.

Comment 4

[Matt Brandt [:mbrandt]]

(In reply to Giorgos Logiotatidis [:giorgos] from comment #2)
> Matt is this a new test case?

No it's not a new testcase - each release I test this code path, although I have several different images that I randomly select from when testing. This time I happened to use Fire Marshal Matt :)

Comment 5

[Giorgos Logiotatidis [:giorgos]]

I can only reproduce this with the Fire Marshal Matt. This is the cause of the fire.

Comment 6

[Justin Crawford [:hoosteeno] [:jcrawford]]

We should fix this, but it's not blocking our summit improvements.

Comment 7

[Andrei Hajdukewycz [:sancus]]

This issue in the exif cleaning code needs to be resolved before that can go live.

Comment 8

[Giorgos Logiotatidis [:giorgos]]

Probably this is because Fire Marshal Matt image doesn't contain any EXIF data.

Comment 9

[Giorgos Logiotatidis [:giorgos]]

After some deeper investigation this was caused due to image having no EXIF data, therefore raising an AttributeError in the new "image cleaning" section, which was added to prevent broken EXIF data.

The cleaning section, re-saves the image to cleanup EXIF data, which sometimes (i.e. image format convertion) can result to larger images being generated. In this case we didn't update the size of the `photo` variable and due to this PIL was fed incomplete data, which resulted in the IOError.

This is fixed here, along with a new test case.

https://github.com/mozilla/mozillians/pull/651

Comment 10

[[github robot]]

Commits pushed to master at https://github.com/mozilla/mozillians

https://github.com/mozilla/mozillians/commit/bf201f41d47f428522194d18cc72b04f0e62763a
[fix bug 921243] Reset the image size after image clean.

https://github.com/mozilla/mozillians/commit/797d1c38cb687975e46efe750b47c49418708ead
Merge pull request #651 from glogiotatidis/921243

[fix bug 921243] Reset the image size after image clean.

Comment 11

[Matt Brandt [:mbrandt]]

Reopening:

Williamr was able to generate this traceback when accessing /user/edit on dev

Traceback (most recent call last):

  File "/data/www/mozillians-dev.allizom.org/mozillians/vendor/lib/python/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.10.2.38/newrelic/api/object_wrapper.py", line 220, in __call__
    self._nr_instance, args, kwargs)

  File "/usr/lib64/python2.6/site-packages/newrelic-1.10.2.38/newrelic/hooks/framework_django.py", line 475, in wrapper
    return wrapped(*args, **kwargs)

  File "/data/www/mozillians-dev.allizom.org/mozillians/vendor/lib/python/django/views/decorators/cache.py", line 89, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)

  File "/data/www/mozillians-dev.allizom.org/mozillians/mozillians/phonebook/views.py", line 158, in edit_profile
    if (user_form.is_valid() and profile_form.is_valid() and accounts_formset.is_valid() and

  File "/data/www/mozillians-dev.allizom.org/mozillians/vendor/lib/python/django/forms/forms.py", line 124, in is_valid
    return self.is_bound and not bool(self.errors)

  File "/data/www/mozillians-dev.allizom.org/mozillians/vendor/lib/python/django/forms/forms.py", line 115, in _get_errors
    self.full_clean()

  File "/data/www/mozillians-dev.allizom.org/mozillians/vendor/lib/python/django/forms/forms.py", line 270, in full_clean
    self._clean_fields()

  File "/data/www/mozillians-dev.allizom.org/mozillians/vendor-local/src/happyforms/happyforms/__init__.py", line 32, in _clean_fields
    value = getattr(self, 'clean_%s' % name)()

  File "/data/www/mozillians-dev.allizom.org/mozillians/mozillians/phonebook/forms.py", line 132, in clean_photo
    photo.size = cleaned_photo.tell()

AttributeError: can't set attribute

Comment 12

[Giorgos Logiotatidis [:giorgos]]

I filed a bug about this before I saw that you re-opened this. I'll mark as fixed this one and work on the new bug that catches the problem more specifically. (bug 925256)

Thanks for keeping an eye on this!

Comment 13

[Matt Brandt [:mbrandt]]

Cool .. thanks Giorgos :) bumping back to verified. Sorry for the dupe