775625 - Restore TOCs that were lost due to section edit bug

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Wiki pages, defect)

Description

[John Karahalis [:openjck]]

In bug 772898, we discovered that section editing sometimes removes the table of contents and tags associated with a page. We should look into restoring this information before launch.

Not very many edits have been happening on developer-new (maybe a few hundred or so), and even fewer of them could have been section edits. We could potentially restore this manually if we can first identify all section edits that have been made.

Comment 1

[Les Orchard [:lorchard]]

Tried taking a quick shot at identifying documents without tags or with TOC disabled... turns out there are a whole lot of documents and revisions that fit the bill:

mysql> select count(*) from wiki_document left join wiki_revision on wiki_revision.id=wiki_document.current_revision_id where wiki_revision.show_toc=0 and wiki_document.is_template=0;
+----------+
| count(*) |
+----------+
|      214 |
+----------+
1 row in set (0.20 sec)

mysql> select count(*) from wiki_document left join wiki_taggeddocument on wiki_taggeddocument.content_object_id=wiki_document.id where wiki_taggeddocument.tag_id is null;
+----------+
| count(*) |
+----------+
|    31009 |
+----------+
1 row in set (0.24 sec)

So... lots of documents with no tags, and little over 200 docs with TOC disabled. Not sure what to do to restore them, since I think a non-trivial number of these have been recently edited and we don't want to clobber them with a simple re-migration

Comment 2

[Jean-Yves Perrier [:teoli]]

Considering that
1) the bug was removing both the TOC and the tags
2) No TOC was a new feature of KUMA
3) Plenty of pages had no tags before.

The 214 list is the list of problems. I, personally, have only one page where the TOC have been voluntarily edited. So if we force-set the TOC on all these pages, I'm ok. I'll reedit that page manually.

Then for the tags, I think for those of these 214 that still don't have tags (that is not manually fixed), we should reimport the tags only. Note if the amount of pages still to fix is low (read <50), we can do it manually.

Comment 3

[Les Orchard [:lorchard]]

Looks like 196 pages with TOC disabled and no tags: http://pastebin.mozilla.org/1717242

There's no such tool that just migrates tags from MindTouch. Need to think if there's an easy way to change the current migration tool, or if it would be easier to build a new one

Comment 4

[Jean-Yves Perrier [:teoli]]

Most of them are very similar (and will have the same list of tags), so I think it would be quicker to fix these pages manually.

Comment 5

[Les Orchard [:lorchard]]

Okay, going to call this wontfix for now. We can re-open if it turns out to be more of a pain to fix over time

Comment 6

[Eric Shepherd [:sheppy]]

How do we restore the TOCs then? That needs to be done still. The checkbox when you go into the editor doesn't indicate that the TOC is set any different than pages that do have the TOC, so this doesn't look like something we can fix in the editor.

Comment 7

[Eric Shepherd [:sheppy]]

Reopening to at least restore the TOCs; let's just reset this for all pages, and then we can go through and hide the TOC on the few pages on which we want that to be the case.

Comment 8

[Les Orchard [:lorchard]]

Just ran this on prod after a quick backup dump (just in case):

mysql> create temporary table notoc select current_revision_id, mindtouch_page_id, wiki_document.locale, wiki_document.slug from wiki_document 
    -> left join wiki_revision on wiki_revision.id=wiki_document.current_revision_id
    -> left join wiki_taggeddocument on wiki_taggeddocument.content_object_id=wiki_document.id 
    -> where 
    -> wiki_revision.show_toc=0 and 
    -> wiki_document.is_template=0 and 
    -> wiki_taggeddocument.tag_id is null and 
    -> wiki_document.mindtouch_page_id is not null;
Query OK, 128 rows affected (0.27 sec)
Records: 128  Duplicates: 0  Warnings: 0

mysql> update wiki_revision set show_toc=1 where id in (select current_revision_id from notoc);
Query OK, 128 rows affected (16.13 sec)
Rows matched: 128  Changed: 128  Warnings: 0

I think the smaller number (ie. 128) is because :teoli as been hand-repairing these over the past hour or so.

Comment 9

[Les Orchard [:lorchard]]

Oh, and this is a list of the docs I just tweaked: 
http://pastebin.mozilla.org/1717511

Comment 10

[Jean-Yves Perrier [:teoli]]

I confirm, the en-US TOC fixes start exactly where I was. I'm fixing the tags list too (only on en-US, due to the editor bug).

Comment 11

[Jean-Yves Perrier [:teoli]]

I fixed the en-US pages regarding tags. For the pages in other languages I need bug 773295 to be fixed first. (Leave it blocking this bug, so that I will be reminded of this bug once it get fixed).

Comment 12

[John Karahalis [:openjck]]

Were the tags restored as well? Reopening just so that I keep track of this.

Comment 13

[Jean-Yves Perrier [:teoli]]

No. I manually restored the tags for English articles. The tags for non-English articles has not been restored (we don't have an interface to put/modify tags on non-English articles)

Comment 14

[John Karahalis [:openjck]]

Thanks, Jean-Yves. Opened bug 779241 for that.