817765 - Sporadic IntegrityErrors from sorl.thumbnail when saving the hashed string

Status: RESOLVED FIXED

(Webtools Graveyard :: Air Mozilla, defect)

Description

[Peter Bengtsson [:peterbe]]

Sample traceback::


 Traceback (most recent call last):
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/data/www/air.mozilla.org/air/airmozilla/main/views.py", line 91, in home
    'tags': tags
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/shortcuts/__init__.py", line 44, in render
    return HttpResponse(loader.render_to_string(*args, **kwargs),
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/template/loader.py", line 176, in render_to_string
    return t.render(context_instance)
  File "/data/www/air.mozilla.org/air/vendor/src/jingo/jingo/__init__.py", line 189, in render
    return super(Template, self).render(context_dict)
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)
  File "/data/www/air.mozilla.org/air/airmozilla/main/templates/main/home.html", line 2, in top-level template code
    {% set front_page = (events.number == 1) %}
  File "/data/www/air.mozilla.org/air/airmozilla/main/templates/main/main_base.html", line 3, in top-level template code
    {% set nav_main = [
  File "/data/www/air.mozilla.org/air/airmozilla/base/templates/base.html", line 12, in top-level template code
    {% block content %}{% endblock %}
  File "/data/www/air.mozilla.org/air/airmozilla/main/templates/main/main_base.html", line 65, in block "content"
    {% block content_main %}{% endblock %}
  File "/data/www/air.mozilla.org/air/airmozilla/main/templates/main/home.html", line 36, in block "content_main"
    {% set thumb = thumbnail(live.placeholder_img, '160x160', crop='center') %}
  File "/data/www/air.mozilla.org/air/airmozilla/main/helpers.py", line 41, in thumbnail
    return get_thumbnail(filename, geometry, **options)
  File "/data/www/air.mozilla.org/air/vendor-local/lib/python/sorl/thumbnail/shortcuts.py", line 8, in get_thumbnail
    return default.backend.get_thumbnail(file_, geometry_string, **options)
  File "/data/www/air.mozilla.org/air/vendor-local/lib/python/sorl/thumbnail/base.py", line 66, in get_thumbnail
    default.kvstore.set(thumbnail, source)
  File "/data/www/air.mozilla.org/air/vendor-local/lib/python/sorl/thumbnail/kvstores/base.py", line 33, in set
    self._set(image_file.key, image_file)
  File "/data/www/air.mozilla.org/air/vendor-local/lib/python/sorl/thumbnail/kvstores/base.py", line 138, in _set
    self._set_raw(add_prefix(key, identity), s)
  File "/data/www/air.mozilla.org/air/vendor-local/lib/python/sorl/thumbnail/kvstores/cached_db_kvstore.py", line 36, in _set_raw
    kv = KVStoreModel.objects.get_or_create(key=key)[0]
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/manager.py", line 134, in get_or_create
    return self.get_query_set().get_or_create(**kwargs)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/query.py", line 452, in get_or_create
    obj.save(force_insert=True, using=self.db)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/base.py", line 463, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/base.py", line 551, in save_base
    result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/manager.py", line 203, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/query.py", line 1593, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/models/sql/compiler.py", line 910, in execute_sql
    cursor.execute(sql, params)
  File "/data/www/air.mozilla.org/air/vendor/lib/python/django/db/backends/mysql/base.py", line 114, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 173, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
 IntegrityError: (1062, "Duplicate entry 'sorl-thumbnail||image||c43fd42550f8ddabf2302f0767dc4f7b' for key 'PRIMARY'")


The short-term solution is to solve it with a try:except in the helper function.

Comment 1

[Peter Bengtsson [:peterbe]]

Note: I've never been able to reproduce this locally and even on production it's hard to catch because the really short term solution is to just refresh the page where it happened.

Comment 2

[[github robot]]

Commit pushed to master at https://github.com/mozilla/airmozilla

https://github.com/mozilla/airmozilla/commit/419e7bc470e150295a44e5fe5886ad973b3a071e
bug 817765 - integrityerrors on get_thumbnail sometimes