584353 - Run migrate_questions management command on support-stage

Status: VERIFIED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[James Socol [:jsocol, :james]]

On support-stage, in the kitsune virtualenv, please run the "migrate_questions" management command:

./manage.py migrate_questions

It will take between 15-30 minutes.

Comment 1

[Shyam Mani [:fox2mike]]

Ran into an issue :

Starting migration for forum "Firefox" (1)
Processing thread 699902...
Traceback (most recent call last):
  File "/data/www/support-stage-new.mozilla.com/kitsune/manage.py", line 36, in <module>
    execute_manager(settings)
  File "/data/virtualenvs/kitsune/src/django/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/data/virtualenvs/kitsune/src/django/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/data/virtualenvs/kitsune/src/django/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/data/virtualenvs/kitsune/src/django/django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/questions/management/commands/migrate_questions.py", line 190, in handle
    create_question_metadata(question)
  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/questions/management/commands/migrate_questions.py", line 115, in create_question_metadata
    for meta in metadata:
  File "/data/virtualenvs/kitsune/src/django/django/db/models/query.py", line 106, in _result_iter
    self._fill_cache()
  File "/data/virtualenvs/kitsune/src/django/django/db/models/query.py", line 760, in _fill_cache
    self._result_cache.append(self._iter.next())
  File "/data/virtualenvs/kitsune/src/django-cache-machine/caching/base.py", line 127, in __iter__
    obj = iterator.next()
  File "/data/virtualenvs/kitsune/src/django/django/db/models/query.py", line 269, in iterator
    for row in compiler.results_iter():
  File "/data/virtualenvs/kitsune/src/django/django/db/models/sql/compiler.py", line 672, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/data/virtualenvs/kitsune/src/django/django/db/models/sql/compiler.py", line 727, in execute_sql
    cursor.execute(sql, params)
  File "/data/virtualenvs/kitsune/src/django/django/db/backends/mysql/base.py", line 86, in execute
    return self.cursor.execute(query, args)
  File "/data/virtualenvs/kitsune/lib/python2.6/site-packages/MySQLdb/cursors.py", line 173, in execute
    self.errorhandler(self, exc, value)
  File "/data/virtualenvs/kitsune/lib/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
_mysql_exceptions.OperationalError: (1054, "Unknown column 'tiki_comments_metadata.id' in 'field list'")

Comment 3

[Paul Craciunoiu [:paulc]]

WRT comment 1 -- make sure |schematic migrations| has run beforehand. It should add that column.

Comment 4

[James Socol [:jsocol, :james]]

The migration that added the column was numbered incorrectly and so never ran. It's been fixed on development.

Comment 5

[Shyam Mani [:fox2mike]]

Hosed the first time, I guess. What do I need to remove from the DB?

[root@mrapp-stage04 ~]# /data/virtualenvs/kitsune/bin/python /data/www/support-stage-new.mozilla.com/kitsune/manage.py migrate_questions
Starting migration for forum "Firefox" (1)
Processing thread 699902...
Traceback (most recent call last):
  File "/data/www/support-stage-new.mozilla.com/kitsune/manage.py", line 36, in <module>
    execute_manager(settings)
  File "/data/virtualenvs/kitsune/src/django/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/data/virtualenvs/kitsune/src/django/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/data/virtualenvs/kitsune/src/django/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/data/virtualenvs/kitsune/src/django/django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/questions/management/commands/migrate_questions.py", line 190, in handle
    create_question_metadata(question)
  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/questions/management/commands/migrate_questions.py", line 112, in create_question_metadata
    _clean_question_content(question)
  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/questions/management/commands/migrate_questions.py", line 82, in _clean_question_content
    question.add_metadata(troubleshooting=troubleshooting)
  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/questions/models.py", line 105, in add_metadata
    value=value)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/manager.py", line 138, in create
    return self.get_query_set().create(**kwargs)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/query.py", line 352, in create
    obj.save(force_insert=True, using=self.db)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/base.py", line 435, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/base.py", line 528, in save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/manager.py", line 195, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/query.py", line 1479, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/sql/compiler.py", line 783, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "/data/virtualenvs/kitsune/src/django/django/db/models/sql/compiler.py", line 727, in execute_sql
    cursor.execute(sql, params)
  File "/data/virtualenvs/kitsune/src/django/django/db/backends/mysql/base.py", line 86, in execute
    return self.cursor.execute(query, args)
  File "/data/virtualenvs/kitsune/lib/python2.6/site-packages/MySQLdb/cursors.py", line 173, in execute
    self.errorhandler(self, exc, value)
  File "/data/virtualenvs/kitsune/lib/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.IntegrityError: (1062, "Duplicate entry '699902-troubleshooting' for key 2")

Comment 6

[Paul Craciunoiu [:paulc]]

What I usually do locally is wipe out the questions and all related objects by doing (from |python manage.py shell|):
from questions.models import Question
Question.objects.all().delete()

Alternatively you can only delete the ones with ID greater than, say, 600,000.

Comment 7

[James Socol [:jsocol, :james]]

(In reply to comment #6)
> What I usually do locally is wipe out the questions and all related objects by
> doing (from |python manage.py shell|):

If we leave the questions that won't collide, like all the ones number < 600,000, we won't break all of QA's test links, for now anyway.

Comment 8

[Shyam Mani [:fox2mike]]

I just ran :

mysql> delete from questions_questionmetadata where question_id=699902;
Query OK, 1 row affected (0.00 sec)

mysql> delete from questions_question where id = 699902;
Query OK, 1 row affected (0.02 sec)

And the stuff is running fine now.

Comment 9

[Shyam Mani [:fox2mike]]

Successfully migrated posts in forum "Firefox" (1)
Reached maximum number of threads to migrate (15000) and stopped.

Comment 10

[James Socol [:jsocol, :james]]

Great! I see thousands of threads, thanks!