1187070 - Certain documents can't be deleted

Status: RESOLVED FIXED

(support.mozilla.org :: Knowledge Base Software, task, P4)

Description

[Kadir Topal [:atopal]]

While removing broken template documents for the work in bug 1166090, I ran into a small number of documents that were impossible to find/delete from the admin:

The document IDs are:
* 5859
* 9870
* 11297
* 11946
* 13238


Then there are several documents that can be seen in the admin, but deleting them leads to server errors:

* 18335
* 18338
* 18820
* 18821

Comment 1

[Michael Cooper [:mythmon]]

For the ones that you can't find in the admin, how did you learn they existed? It sounds like they got deleted since you got your list of documents.

As for the ones that cause an error when you delete them, I reproduced those locally. Below is the traceback for the first one. I'm not really sure what to make of this. I feel like we should simply delete them directly instead of using the admin, though I would like to figure out what is going on here.


Traceback:
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/options.py" in wrapper
  583.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  105.                     response = view_func(request, *args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  52.         response = view_func(request, *args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/sites.py" in inner
  206.             return view(request, *args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapper
  29.             return bound_func(*args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  105.                     response = view_func(request, *args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/utils/decorators.py" in bound_func
  25.                 return func.__get__(self, type(self))(*args2, **kwargs2)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/db/transaction.py" in inner
  394.                 return func(*args, **kwargs)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/options.py" in delete_view
  1638.             [obj], opts, request.user, self.admin_site, using)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/utils.py" in get_deleted_objects
  153.     to_delete = collector.nested(format_callback)
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/utils.py" in nested
  211.             roots.extend(self._nested(root, seen, format_callback))
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/utils.py" in _nested
  194.             children.extend(self._nested(child, seen, format_callback))
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/utils.py" in _nested
  196.             ret = [format_callback(obj)]
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/contrib/admin/utils.py" in format_callback
  126.                                    force_text(obj))
File "/home/mythmon/src/kitsune/venv/lib/python2.7/site-packages/django/utils/encoding.py" in force_text
  95.             raise DjangoUnicodeDecodeError(s, *e.args)

Exception Type: DjangoUnicodeDecodeError at /admin/wiki/document/18335/delete/
Exception Value: 'ascii' codec can't decode byte 0xd0 in position 16: ordinal not in range(128). You passed in <DocumentLink: [Bad Unicode data]> (<class 'kitsune.wiki.models.DocumentLink'>)

Comment 2

[Michael Cooper [:mythmon]]

In trying to investigate this to figure out how to estimate this, I found a very easy change that makes this problem go away.

PR: https://github.com/mozilla/kitsune/pull/2609

Comment 3

[Michael Cooper [:mythmon]]

I deployed the fix for not being able to delete

Comment 4

[Michael Cooper [:mythmon]]

Oops, hint enter too quick.

I deployed the fix for not being able to delete certain documents through the admin. Still waiting for more information about the "missing" documents.

Comment 5

[Michael Cooper [:mythmon]]

I can't reproduce these missing documents. If you have more information, please re-open. Until then, I'm closing this.

Comment 6

[Kadir Topal [:atopal]]

Mike, I found them using this SQL query on the SUMO database: 

SELECT *
FROM `wiki_document`
WHERE `wiki_document`.`category` = 60
AND `wiki_document`.`title` NOT LIKE "Template:%"

So, there are 5 of these left, and I can't find a way to delete them. We are still seeing reports of issues with the document counts, maybe it's related to this.

The document IDs are:
* 5859
* 9870
* 11297
* 11946
* 13238

Any idea, how to delete them, Mike? I guess a solution of last resort would be to run a SQL command on the production database?

If you don't think think that these documents might be causing issues, we can just close this bug.

Comment 7

[Michael Cooper [:mythmon]]

Those documents aren't showing up in the admin because they aren't in the database. I just ran your query on the production master database, and no documents were found.

Which database specifically were you running these against? I did this from supportadm.private.pxh1:

[[e-mail]1 ~]$ mysql -h db-sumo-rw -u support -p
Enter password: 
mysql> use support_mozilla_com
Database changed
mysql> SELECT * FROM `wiki_document` WHERE `wiki_document`.`category` = 60 AND `wiki_document`.`title` NOT LIKE "Template:%";
Empty set (0.01 sec)

Comment 8

[Kadir Topal [:atopal]]

I ran the query against sumotools1.webapp.phx1.mozilla.com

My understanding was that sumotools was a slave of the production master database and should be an exact copy of that, but apparently not?

Comment 9

[Michael Cooper [:mythmon]]

That was my understanding as well.

Sheeri: I didn't see anything in Mana. Can you clarify the relationship between sumotools1.webapp.phx1.mozilla.com and the Sumo prod db?

Comment 10

[Sheeri Cabral [:sheeri]]

sumotools is in fact a slave of the sumo prod db (currently slaving support3.db.phx1.mozilla.com).

It's an exact copy, insofar as people can write to the sumotools db, and changes don't replicate back to production.

As far as we are concerned, the most recent checksum run came out clean:
mysql> select count(*),max(ts) from percona.checksums;
+----------+---------------------+
| count(*) | max(ts)             |
+----------+---------------------+
|      494 | 2015-08-31 06:17:47 |
+----------+---------------------+
1 row in set (0.00 sec)

mysql> select * from percona.checksums where this_crc!=master_crc;
Empty set (0.00 sec)


The tool isn't 100% flawless, but that's what we get paged based on.

Comment 11

[Michael Cooper [:mythmon]]

I can't reproduce this problem. I don't see any ghost articles, and further I can't imagine a way for them to show up in the database but not on the site. There isn't anything concrete to do here.

Also, this bug conflates two issues. In the future it would be much better to have two bugs. It is easy to mark bugs as duplicates, but we can't easily split bugs.