Closed Bug 586487 Opened 10 years ago Closed 9 years ago

Migrate KB articles to Kitsune

Categories

(support.mozilla.org :: Knowledge Base Software, task, P1)

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: jsocol, Assigned: paulc)

References

Details

Attachments

(2 files)

Migrate the existing KB articles and translations to Kitsune. We don't need the history.
Depends on: 597243
I'm adding locale context to the wiki hooks for internal links, videos, etc, in bug 600747. We hadn't hit this before, but having the same title in multiple locales throws around tracebacks right now. Bug 600747 fixes that, which is obviously required for the migration.
Depends on: 600747
Depends on: 602693
Depends on: 599006
Depends on: 605275
Noticed a minor issue on the Style Guide[1] (not sure how common the markup is).

The initial markup is

   __~~#006600:Good~~__

The resulting markup is

   '''~~#006600:Good~~'''

My thoughts are either a) leave it for manual fixing, or b) convert it to <span style="color: #foo">...</span>. If it's fixed manually, it could be done with the surrounding tags, eg: <strong style="color: #foo">.

[1] https://master.support.mozilla.com/en-US/kb/Style%20Guide
Can we migrate {mactoc} to __TOC__ and get rid of the automatic TOC on Kitsune? This way we can control whether or not there is a TOC and at what point in the article it shows up.
You might already know this, but there seem to be a number of obviously non-English articles coming over with having an en-US locale. For example:

select locale, title from wiki_document where title like '%--%';
select locale, title from wiki_document where title like '% zh-TW';
Some wiki_revisions have IDs that aren't in order of their creation. It would simplify a few spots in my l10n dashboard queries if this weren't the case. Is it a big deal to fix?
I requested that we change the Mobile showfor Fx versions from "Firefox Mobile 1.0" and "Firefox Mobile 1.1" to just "Firefox 4 for Mobile" in bug 609013. James asked me to comment here in case that changes anything happening as part of the article migration. FYI, none of the mobile articles use showfor at this time.
(In reply to comment #5)
> Some wiki_revisions have IDs that aren't in order of their creation. It would
> simplify a few spots in my l10n dashboard queries if this weren't the case. Is
> it a big deal to fix?
I'm currently ordering by page_id, which I assumed was synonymous. Hopefully ordering by creation date will do the trick, which would be an easy fix.

(In reply to comment #4)
> You might already know this, but there seem to be a number of obviously
> non-English articles coming over with having an en-US locale. For example:
> 
> select locale, title from wiki_document where title like '%--%';
> select locale, title from wiki_document where title like '% zh-TW';
Sadly, some tiki_page rows have invalid locales or locales set to NULL or empty. Can't do much about those.

(In reply to comment #3)
> Can we migrate {mactoc} to __TOC__ and get rid of the automatic TOC on Kitsune?
> This way we can control whether or not there is a TOC and at what point in the
> article it shows up.
This looks like a nice fix worth getting in before doing another migration run on stage.

(In reply to comment #6)
> I requested that we change the Mobile showfor Fx versions from "Firefox Mobile
> 1.0" and "Firefox Mobile 1.1" to just "Firefox 4 for Mobile" in bug 609013.
> James asked me to comment here in case that changes anything happening as part
> of the article migration. FYI, none of the mobile articles use showfor at this
> time.
I highly doubt it will change anything for the migration, but if it does, I'll make a note of it.
When we load the dump on production (or even staging?), let's not disable foreign key checks. I don't like even the possibility of dangling references; it wastes our time chasing nonexistent bugs, and computer time is cheap.

Just remove the things from the dump that look like /*!40000 ALTER TABLE `wiki_revision` DISABLE KEYS */;.
(One such bug was bug 609081.)
As long as ENABLE KEYS shows up again later, it speeds up the import significantly to disable keys while loading large datasets.
Sure, I'm familiar with (for example) dropping indexes before bulk-loading data, but, unlike with Postgres's DEFER CONSTRAINTS thing, MySQL doesn't seem to go through and check the loaded data when you reenable the constraints or commit the transaction. My DB is somehow full of dangling refs.
It should. Are you sure keys are enabled on those tables, and that creator_id isn't nullable for some reason?

We can't leave foreign key checks enabled, anyway: we have circular foreign key references between documents and revisions.
Ah yes, document <-> revision, as Paul aptly notes. creator_id is NOT NULL, but that's not the issue here. Somehow I got actual ints in that field that don't reference an auth_user. I'll try a toy case...
MySQL has nothing like Postgres's SET CONSTRAINTS DEFERRED; it always checks constraints immediately. Thus, it's impossible to load non-nullable circular refs into MySQL without shutting off constraints altogether first. ALTER TABLE `wiki_revision` DISABLE KEYS appears to do nothing; MySQL still checks constraints on that table. SET foreign_key_checks = 0 let you insert what you want, but even setting it back to 1 before committing doesn't trigger a check on the inserted data: you can still commit bad stuff.

Thus, to ensure we ultimately end up with good data, we'll need to do a query for each foreign key after the fact, like this one:

select wiki_revision.id, creator_id, auth_user.id from wiki_revision left join auth_user on creator_id=auth_user.id where auth_user.id is null;

It hurts my soul.
Some articles that should have approved revisions don't. For example en-US:"Deleting cookies", id 760 in wiki_media_11_01.sql. I compared it against wiki.sql from last week.

Running this query:

> select locale, title, current_revision_id, is_approved
> from wiki_document d join wiki_revision r on d.id = r.document_id
> where d.title = 'Deleting cookies';

against the old, I get:

+--------+------------------+---------------------+-------------+
| locale | title            | current_revision_id | is_approved |
+--------+------------------+---------------------+-------------+
| da     | Deleting cookies |                3766 |           1 |
| en-US  | Deleting cookies |                 768 |           1 |
+--------+------------------+---------------------+-------------+

And in the new:

+--------+------------------+---------------------+-------------+
| locale | title            | current_revision_id | is_approved |
+--------+------------------+---------------------+-------------+
| da     | Deleting cookies |                3814 |           1 |
| en-US  | Deleting cookies |                NULL |           0 |
+--------+------------------+---------------------+-------------+
https://master.support.mozilla.com/en-US/kb/Websites%20say%20cookies%20are%20blocked#w_clear-history-for-that-site

The image is gone, and Topal says: "the filename is strangely put under the images".

Milos.
What category are help articles migrated to? In Kitsune the categories I see are Troubleshooting, How to Contribute, Administration and Templates.
Just a quick note: when migrating articles, please don't migrate the tags. Since tags are used for topics in Kitsune we'd like to start fresh.
Thanks for attaching those James.

(In reply to comment #17)
> What category are help articles migrated to? In Kitsune the categories I see
> are Troubleshooting, How to Contribute, Administration and Templates.
Troubleshooting. Sounds good?
As an important note, if you have created new English articles that are NOT in the "Knowledge Base" category on production, and you want them migrated, please notify me or they will be skipped.

(In reply to comment #20)
> Just a quick note: when migrating articles, please don't migrate the tags.
> Since tags are used for topics in Kitsune we'd like to start fresh.
Noted. Tags are not migrated at the moment so I'll just keep it that way :)
(In reply to comment #21)
> (In reply to comment #17)
> > What category are help articles migrated to? In Kitsune the categories I see
> > are Troubleshooting, How to Contribute, Administration and Templates.
> Troubleshooting. Sounds good?

Paul, can we create a category called How-to and migrate all help articles to that instead (troubleshooting articles should still go to troubleshooting). Eventually we want to have different looks for these categories and this way we won't loose the small bit of categorization that's already been done.
The "Help with" option defaults to Windows and Firefox 4, even though I'm using Firefox 3.6 on Mac OS 10.6.  Also, the "showfor" feature itself is buggy. No matter what I select for the "Help with" option, Firefox only partially displays the content.  

See my post here for details and other comments:
https://support.mozilla.com/en-US/forums/contributors/704768
I also hear that all migrated articles should have the Firefox 4 version.
(In reply to comment #24)
> I also hear that all migrated articles should have the Firefox 4 version.

Yes please. 98% will be relevant to Firefox 4. Would be nice not to have to edit them all and check that box. :)
Also include the "Get help with Firefox 4 beta" article as per bug 608061 comment 5.
Need to merge categories of translated article with the ones of the english article, before migration. For instance, "Utiliser Firefox" french article (translated from "Browsing basics") is not in any category, as "Browsing basics" is in "help", "3.0", "3.5/3.6", "in-product" categories.

Need to convert:
* out-of-date markup: ~hs~ => non-breaking space, ~np~ item ~/np => comment?
* unicode characters: ~123~, ~125~, ....
* markup and special character inside ((|)): e.g. {DIV}, « »

Bad display of:
* array syntax : ||item1|item2||
* indented list in template: **, ##
* special character in article title: e.g. é
* link to localized article with the english article: [[Installing Firefox]] for instance does not refer to the localized article, e.g. "Installer Firefox" for french
* {note} markup: no different in display with plain text
(In reply to comment #27)
> Need to merge categories of translated article with the ones of the english
> article, before migration. For instance, "Utiliser Firefox" french article
> (translated from "Browsing basics") is not in any category, as "Browsing
> basics" is in "help", "3.0", "3.5/3.6", "in-product" categories.
Should the categories of the translated article just be EXACTLY the same as the English? Or do we want to inherit + use any additional ones that a translation has?

> 
> Need to convert:
> * out-of-date markup: ~hs~ => non-breaking space, ~np~ item ~/np => comment?
Did not know about ~hs~, so thank you.
But I'm pretty sure ~np~ => <nowiki>. I just tried it on support-stage for this:
Input: ~np~ some text <b>here</b> ((A link)) ~/np~
Output: some text <b>here</b> ((A link))
The output renders exactly like that.

> * unicode characters: ~123~, ~125~, ....
> * markup and special character inside ((|)): e.g. {DIV}, « »
Can you provide actual examples of Tiki syntax and what it should turn into here? I'm not sure what to look for.
> 
> Bad display of:
> * array syntax : ||item1|item2||
I think this is actually tables, and, unfortunately, table syntax is much too complicated to be converted with reasonable effort. So it will have to be done manually.

> * indented list in template: **, ##
> * special character in article title: e.g. é
Examples for these two please?

> * link to localized article with the english article: [[Installing Firefox]]
> for instance does not refer to the localized article, e.g. "Installer Firefox"
> for french
This is actually not a migration issue. It is fixed already but re-rendering doesn't seem to happen on master.support yet. We're on it, though.

> * {note} markup: no different in display with plain text
Good point. Again, not a migration issue, but filed bug 611057.

Thanks a lot for your feedback! I'm on it, right now :)
Fixed most of the stuff in comment 27. Made a note of nested lists in bug 611057.

I'm still unclear on these:
> * markup and special character inside ((|)): e.g. {DIV}, « »
> * special character in article title: e.g. é


I'm also making translations inherit the exact same categories, firefox versions and operating systems as the English parent at all times (if parent exists). Objections?
Reply to comment 28:
> Should the categories of the translated article just be EXACTLY the same as
> the English? Or do we want to inherit + use any additional ones that a
> translation has?
With existing categories, I don't see any reason to add a specific category to a translated article. If an existing translated article has an additional category, it is probably because it is no more maintained or impossible to remove category. So the english article must be the reference for categories.

> But I'm pretty sure ~np~ => <nowiki>
I think it is used to allow markup to be displayed as plain text. For instance, in https://support.mozilla.com/en-US/kb/Using+the+Adobe+Reader+plugin+with+Firefox, see [http://www.schubert-it.com/pluginpdf/|Schubert~np~|~/np~it PDF Browser Plugin]

> > * unicode characters: ~123~, ~125~, ....
> > * markup and special character inside ((|)): e.g. {DIV}, « »
> Can you provide actual examples of Tiki syntax and what it should turn into
> here? I'm not sure what to look for.
For unicode characters: I don't find an example but I know it exists.
For markup and special character inside ((|)): see https://support.mozilla.com/en-US/kb/Options+window+-+Content+panel

> I think this is actually tables, and, unfortunately, table syntax is much too
> complicated to be converted with reasonable effort. So it will have to be done
> manually.
It is a pity, especially for https://support.mozilla.com/en-US/kb/Keyboard+shortcuts and https://support.mozilla.com/en-US/kb/N900+keyboard+shortcut.
Is there a way that these articles are readable? May be by replacing | by TAB and removing || in table markup.

> > * indented list in template: **, ##
> > * special character in article title: e.g. é
> Examples for these two please?
* nested list in template: see https://master.support.mozilla.com/en-US/kb/Template:profileFolder
* special character in article title: see https://master.support.mozilla.com/fr/kb/Comment%20changer%20les%20pr%EF%BF%BDf%EF%BF%BDrences%20dans%20Firefox%20Mobile instead of https://support.mozilla.com/fr/kb/Comment+changer+les+pr%C3%A9f%C3%A9rences+dans+Firefox+pour+mobile


By the way, about non-breaking space, is it possible to replace each space:
* before: : ; ? ! »
* after: «
by a non-breaking space (typography for some european languages)?
(In reply to comment #30)
> Reply to comment 28:
> With existing categories, I don't see any reason to add a specific category to
> a translated article. If an existing translated article has an additional
> category, it is probably because it is no more maintained or impossible to
> remove category. So the english article must be the reference for categories.
Agreed. Done this.

> > But I'm pretty sure ~np~ => <nowiki>
> I think it is used to allow markup to be displayed as plain text. For instance,
> in https://support.mozilla.com/en-US/kb/Using+the+Adobe+Reader+plugin+with+Firefox,
> see [http://www.schubert-it.com/pluginpdf/|Schubert~np~|~/np~it PDF Browser
> Plugin]
What should this render as? It looks like this now: http://is.gd/gWB32

> For unicode characters: I don't find an example but I know it exists.
I made these work anyway, thanks :)

> For markup and special character inside ((|)): see
> https://support.mozilla.com/en-US/kb/Options+window+-+Content+panel
I still don't understand what to look for here.

These last ones I leave up to the SUMO team to decide:

> Is there a way that these articles are readable? May be by replacing | by TAB
> and removing || in table markup.
It's not hard to do, but these should just be MediaWiki-style tables instead. Doing this replacement makes it harder to identify they used to be tables in Tiki, IMO.

> By the way, about non-breaking space, is it possible to replace each space:
> * before: : ; ? ! »
> * after: «
> by a non-breaking space (typography for some european languages)?
I'm no language expert, so I'd like a second opinion. If the SUMO team agrees, this isn't hard to do.


As for this one, sadly you are referring to two distinct articles.
> * special character in article title: see
> https://master.support.mozilla.com/fr/kb/Comment%20changer%20les%20pr%EF%BF%BDf%EF%BF%BDrences%20dans%20Firefox%20Mobile
Equivalent in Tiki:
https://support.mozilla.com/fr/kb/Comment%20changer%20les%20pr%EF%BF%BDf%EF%BF%BDrences%20dans%20Firefox%20Mobile
> instead of
> https://support.mozilla.com/fr/kb/Comment+changer+les+pr%C3%A9f%C3%A9rences+dans+Firefox+pour+mobile
Equivalent in Kitsune:
https://master.support.mozilla.com/fr/kb/Comment%20changer%20les%20pr%C3%A9f%C3%A9rences%20dans%20Firefox%20pour%20mobile
Same difference ;)
(In reply to comment #31)
> > Is there a way that these articles are readable? May be by replacing | by TAB
> > and removing || in table markup.
> It's not hard to do, but these should just be MediaWiki-style tables instead.
> Doing this replacement makes it harder to identify they used to be tables in
> Tiki, IMO.
Talked to Cheng and yeah, these would be better off fixed as MediaWiki tables. It's a pity that they have to be done manually, yes.

> > By the way, about non-breaking space, is it possible to replace each space:
> > * before: : ; ? ! »
> > * after: «
> > by a non-breaking space (typography for some european languages)?
> I'm no language expert, so I'd like a second opinion. If the SUMO team agrees,
> this isn't hard to do.
Ah, to clarify here, I'm assuming you mean to do this replacement in the article source (as opposed to on parsing). In that case, there's a risk that : ; ? and ! are part of wiki syntax and it would be sad to break that.

However, I don't think there's any problem with » and «. I'll do these.

Also, as Cheng points out -- this doesn't have to be part of the migration. We can always fix articles with this problem in the future. We could have a separate bug and a list of articles to fix, after 2.3. Or a list of locales. Etc.
> What should this render as? It looks like this now: http://is.gd/gWB32
That is what it must look like.

>> For markup and special character inside ((|)): see
>> https://support.mozilla.com/en-US/kb/Options+window+-+Content+panel
> I still don't understand what to look for here.
((Options window|{DIV(class=win,type=span)}Options{DIV}{DIV(class=noWin,type=span)}Preferences{DIV} window)) must become 
[[Options window|{for win}Options{/for}{for mac,linux}Preferences{/for} window]]

> > Is there a way that these articles are readable? May be by replacing | by
> > TAB and removing || in table markup.
> It's not hard to do, but these should just be MediaWiki-style tables instead.
> Doing this replacement makes it harder to identify they used to be tables in
> Tiki, IMO.
I let you decide an another solution that keeps table history.

> Ah, to clarify here, I'm assuming you mean to do this replacement in the
> article source (as opposed to on parsing). In that case, there's a risk that :
> ; ? and ! are part of wiki syntax and it would be sad to break that.
My request is to replace existing space before or after some punctuations with non-breaking space. I don't want that you add space.
(In reply to comment #33)
> > ; ? and ! are part of wiki syntax and it would be sad to break that.
> My request is to replace existing space before or after some punctuations with
> non-breaking space. I don't want that you add space.
I understand. Do we know for certain that these are not preceded by space when wiki syntax is used?
> Do we know for certain that these are not preceded by space when wiki syntax is > used?
Here is a wiki syntax guide:
http://doc.tiki.org/Wiki-Syntax+Text
There is no problem for: ? « »
For: : ! ; no replacement must be done as it is used by wiki syntax
Depends on: 611584
Duplicate of this bug: 611584
No longer depends on: 611584
Here is a copy of bug 611584 comment 0:
{SHOWFOR} for browser version has been used to differentiate:
* FF 3.0 from FF 2.0 with {SHOWFOR(browser=firefox3)}
* FF 3.5/3.6 from FF 3.0 with {SHOWFOR(browser=firefox3.5)} or
{SHOWFOR(browser=firefox3.6)}
In order to prevent some blank sections for FF 4.0, the migration script must
replace:
* {SHOWFOR(browser=firefox3)} by {for fx3, fx4}
* {SHOWFOR(browser=firefox3.5)} by {for fx36, fx4}
* {SHOWFOR(browser=firefox3.6)} by {for fx36, fx4}
Note to self: non-English documents shouldn't explicitly have firefox versions or operating systems set if they have a parent.
Also see bug 611729 comment 5.
Prevent SUMO articles in staging area and never approved to become current in Kitsune. They must be in "Out-of-date translations" status in Kitsune.
For instance:
https://support.mozilla.com/fr/kb/La+fen%C3%AAtre+Firefox+est+trop+grande+ou+d%C3%A9passe+l%E2%80%99%C3%A9cran is in "to be approved for the first time" status.
https://master.support.mozilla.com/fr/kb/La%20fen%C3%AAtre%20Firefox%20est%20trop%20grande%20ou%20d%C3%A9passe%20l%E2%80%99%C3%A9cran is in "current" status.
SUMO articles in "Unreviewed changes" & "out-of-date translations" status must also keep their status in Kitsune.
Migrate localized images and videos that are used in article. Indeed, in master.support., localized media gallery is empty.
(In reply to comment #41)
> Migrate localized images and videos that are used in article. Indeed, in
> master.support., localized media gallery is empty.

Unfortunately, media doesn't have an associate locale in Tiki. We've implemented a fallback mechanism that checks the locale of the document, then English.
Keep list formatting. See bug 612238, bug 611737, and bug 611729.
In addition to comment 40 about article state:
If we want to start cleanly, every localized approved article that has an history date before english parent's one must be in an "Unreviewed changes", except if it is already in "out-of-date translations" because either it is in this state in tiki SUMO or for comment 40 reason.
Oups! In comment 44, replace "Unreviewed changes" by "Translations Needing Updates"
Remove orphan articles.
In reply to comment 31:
> It's not hard to do, but these should just be MediaWiki-style tables instead.
> Doing this replacement makes it harder to identify they used to be tables in
> Tiki, IMO.
About table replacement: Is it possible to use tables like the one in:
https://support-stage.mozilla.com/en-US/kb/kitsune-markup
In the migration we just imported to support-stage, it looks like at least some, if not all, parent/translation information was lost. For example, on master.support (still on wiki_media_11_01.sql) the French article "Gérer les cookies" is a translation of the English "Cookies" but on support-stage (wiki_media_11_09_thumbnails.sql) it isn't.

Eg:

https://support-stage.mozilla.com/fr/kb/Gérer les cookies
https://support-stage.mozilla.com/en-US/kb/Cookies

https://master.support.mozilla.com/en-US/kb/Cookies
https://master.support.mozilla.com/fr/kb/Gérer les cookies
(In reply to comment #46)
> Remove orphan articles.
Does this mean non-English articles with NO parents? IIRC, we discussed this with the SUMO team and decided to keep the ones in the Knowledge Base/Staging categories, for now. The rest are not migrated.

(In reply to comment #47)
> About table replacement: Is it possible to use tables like the one in:
> https://support-stage.mozilla.com/en-US/kb/kitsune-markup
I'll look into it. Probably not, because of the syntax differences. I make no promises. :)

(In reply to comment #49)
> In the migration we just imported to support-stage, it looks like at least
> some, if not all, parent/translation information was lost. For example, on
> master.support (still on wiki_media_11_01.sql) the French article "Gérer les
> cookies" is a translation of the English "Cookies" but on support-stage
> (wiki_media_11_09_thumbnails.sql) it isn't.
> 
> Eg:
> 
> https://support-stage.mozilla.com/fr/kb/Gérer les cookies
> https://support-stage.mozilla.com/en-US/kb/Cookies
> 
> https://master.support.mozilla.com/en-US/kb/Cookies
> https://master.support.mozilla.com/fr/kb/Gérer les cookies
This specific case will help me debug. Thanks.
(In reply to comment #48)
> Don't remove localized articles that have no reason to be removed. For
> instance,
> https://support.mozilla.com/fr/kb/Qu%E2%80%99est+ce+que+la+Navigation+priv%C3%A9e
> exists, as
> https://support-stage.mozilla.com/fr/kb/Qu%E2%80%99est%20ce%20que%20la%20Navigation%20priv%C3%A9e
> does not exits
Ah, good one! Turns out this happens because there is another article: Navigation privée, that is also French. Of course, it's only French on the edit page (in the tiki_pages table), but not in the translation table.
See here:
https://support.mozilla.com/tiki-editpage.php?locale=sq&page=Navigation privée
VS here
https://support.mozilla.com/tiki-edit_translation.php?locale=fr&page=Qu%E2%80%99est%20ce%20que%20la%20Navigation%20priv%C3%A9e

I've switched my script to use the translation table locale first, and on collision fall back to the tiki_pages locale. That fixes this article, but could, in theory, place others under invalid locales. Unfortunately only true AI could figure out which one is accurate.

I've done a bunch for mixes and I will run the migration tonight to see how things are going. Thanks again for all the feedback.
> I've done a bunch for mixes and I will run the migration tonight to see how
> things are going. Thanks again for all the feedback.
If you ran the migration script on 11/17/2010, then there are some important remaining issues:

1. Number of articles:
In support, there are:
* 281 "How to" & "Troubleshooting" category articles,
* about 30 "How to contribute" category  articles,
* about 17 "Templates" category articles,
* about 5 "Navigation" category articles,
* about 10 "Administration" category articles,
* about 20 orphan articles for French.
The overall is about 370 articles.
While in support-stage, there are 560 articles, so 200 "ghost" articles.

2. Article states:
In support, for French, there are:
* about 70 "Unreviewed" (for the first time) articles ("How to" & "Troubleshooting" categories),
* about 25 "Untranslated" articles ("How to" & "Troubleshooting" categories).
In support-stage, there are:
* 1 "Unreviewed" article,
* about 250 "Untranslated" articles.
So there is an inconsistency between these two KB.
We're converting both "content templates" and "dynamic content" in Tiki to articles, as well, which may account for much of the discrepancy.
Yes, those 200 "ghost" articles are templates.
Here is a summary of my comments that are not taken into account in support-stage-new and some new comments:

1. KB article state inconsistency:
* How about the automatic approbation of "To be approved for the first time" articles? (see comment 52) Some of these articles are in this state for more than 2 years, so they are "out-of-date". See comment 40.
* Some articles are in "Unreviewed" states in Kitsune and in "Translated" states in SUMO:
https://support.mozilla.com/fr/kb/Fen%C3%AAtre+des+Options+-+Panneau+Contenu
https://support-stage-new.mozilla.com/fr/kb/Fen%C3%AAtre%20des%20Options%20-%20Panneau%20Contenu

2. Valid approved articles are removed:
See comment 48, still applicable in support-stage-new.

3. Inconsistency in Kitsune article number between kb/all and localization dashboard:
In French, according to the localization dashboard, there are 567 articles. According to kb/all, there are 366 articles. There are 201 "ghost" articles (see comment 52).
In these 201 "ghost" articles, I see:
* some articles in Arabic, Chinese, Russian...

4. Orphan and FF 2.0 articles in approved state in Kitsune:
There are orphan (see comment 46) and FF 2.0 articles in approved state. For instance:
https://support-stage-new.mozilla.com/fr/kb/Inspecteur%20DOM in French,
https://support-stage-new.mozilla.com/en-US/kb/DOM%20Inspector in English.
The best way to prevent that is to removed these articles.

5. Localized images are lost
For instance:
https://support-stage-new.mozilla.com/fr/kb/Fenêtre des Options - Panneau Vie privée
See comment 41.

6. Some articles in "Unreviewed" state are not in the localization dashboard:
For instance:
https://support-stage-new.mozilla.com/fr/kb/Fen%C3%AAtre%20des%20Options%20-%20Panneau%20Contenu
https://support-stage-new.mozilla.com/fr/kb/Fenêtre des Options - Panneau Vie privée

7. Adding {for fx4} in some articles
See comment 37.

8. Clean start:
See comment 44 and comment 45.

9. Table conversion:
See comment 47.
Replies inline...

(In reply to comment #55)
> Here is a summary of my comments that are not taken into account in
> support-stage-new and some new comments:
> 
> 1. KB article state inconsistency:
> * How about the automatic approbation of "To be approved for the first time"
> articles? (see comment 52) Some of these articles are in this state for more
> than 2 years, so they are "out-of-date". See comment 40.
> * Some articles are in "Unreviewed" states in Kitsune and in "Translated"
> states in SUMO:
> https://support.mozilla.com/fr/kb/Fen%C3%AAtre+des+Options+-+Panneau+Contenu
> https://support-stage-new.mozilla.com/fr/kb/Fen%C3%AAtre%20des%20Options%20-%20Panneau%20Contenu
This *should* be fixed today, at least for the majority of them, hopefully. These two bullet items look the same to me.

> 
> 2. Valid approved articles are removed:
> See comment 48, still applicable in support-stage-new.
> 
I've checked -- this will be migrated with an approved revision now.

> 3. Inconsistency in Kitsune article number between kb/all and localization
> dashboard:
> In French, according to the localization dashboard, there are 567 articles.
> According to kb/all, there are 366 articles. There are 201 "ghost" articles
> (see comment 52).
> In these 201 "ghost" articles, I see:
> * some articles in Arabic, Chinese, Russian...
See comment 54. Additionally, all templates default to English if no (valid) locale is specified.
> 
> 4. Orphan and FF 2.0 articles in approved state in Kitsune:
> There are orphan (see comment 46) and FF 2.0 articles in approved state. For
> instance:
> https://support-stage-new.mozilla.com/fr/kb/Inspecteur%20DOM in French,
> https://support-stage-new.mozilla.com/en-US/kb/DOM%20Inspector in English.
> The best way to prevent that is to removed these articles.
> 
This is currently being migrated as a document with no approved revisions.

> 5. Localized images are lost
> For instance:
> https://support-stage-new.mozilla.com/fr/kb/Fenêtre des Options - Panneau Vie
> privée
> See comment 41.
All images and videos are migrated to the English locale. We don't currently allow changing locales in the gallery, but if you know some of those images are only used in French and want them to have a French locale, please file a separate bug.
> 
> 6. Some articles in "Unreviewed" state are not in the localization dashboard:
> For instance:
> https://support-stage-new.mozilla.com/fr/kb/Fen%C3%AAtre%20des%20Options%20-%20Panneau%20Contenu
> https://support-stage-new.mozilla.com/fr/kb/Fenêtre des Options - Panneau Vie
> privée
Please check again tomorrow as we are getting a new migration imported today. If this is still true, reply here and we will look into it.


I'm leaving items 7 and 8 to the SUMO team.
I just noticed that the migrated How to set the home page article is missing some sections:
Original: https://support.mozilla.com/en-US/kb/How+to+set+the+home+page
Migrated version:https://support-stage.mozilla.com/en-US/kb/How%20to%20set%20the%20home%20page
Verdi, You just noticed that some migrated articles are missing content?  I mentioned that in comment 23 and here:

https://support.mozilla.com/en-US/forums/contributors/704768
(In reply to comment 56)
> This *should* be fixed today, at least for the majority of them, hopefully.
> These two bullet items look the same to me.
It has not been already fixed, at least for the testcase I gave.

> See comment 54. Additionally, all templates default to English if no (valid)
> locale is specified.
OK. Here are articles that are wrongly classified as English:
https://support-stage-new.mozilla.com/en-US/kb/%E9%81%B8%E9%A0%85%E8%A6%96%E7%AA%97%20--%20%E4%B8%80%E8%88%AC
https://support-stage-new.mozilla.com/en-US/kb/%D8%A7%D9%86%D9%87%D9%8A%D8%A7%D8%B1%20%D9%85%D8%B4%D8%BA%D9%84%20%D8%A7%D9%84%D9%81%D9%84%D8%A7%D8%B4%20Adobe%20Flash
https://support-stage-new.mozilla.com/en-US/kb/%E6%AD%A3%E9%AB%94%E4%B8%AD%E6%96%87%20zh-TW
https://support-stage-new.mozilla.com/fr/kb/%D0%9E%D0%BD%D0%BE%D0%B2%D0%BB%D0%B5%D0%BD%D0%BD%D1%8F%20%D0%B4%D0%BE%20Firefox%2035
https://support-stage-new.mozilla.com/en-US/kb/new?redirectlocale=en-US&redirectslug=Fallimento+registrazione+chrome&title=Errore+nella+registrazione+chrome
https://support-stage-new.mozilla.com/en-US/kb/Plugin%20relat%C3%B3rio%20de%20falhas
https://support-stage-new.mozilla.com/en-US/kb/Ce%20certificat%20a%20un%20num%C3%A9ro%20de%20s%C3%A9rie%20doublon%20avec%20un%20autre%20certificat
https://support-stage-new.mozilla.com/en-US/kb/%22%20OBESIDADE%20%20E%20%20NEUROSE%20%22
https://support-stage-new.mozilla.com/en-US/kb/Firefox es lento
https://support-stage-new.mozilla.com/en-US/kb/Firefox%20a%20Shuite%C3%A1il%20ar%20Mhac
https://support-stage-new.mozilla.com/en-US/kb/codigo%20fonte
https://support-stage-new.mozilla.com/en-US/kb/%D0%A3%D0%BA%D1%80%D0%B0%D0%B8%D0%BD%D1%81%D0%BA%D0%B8%D0%B9

> All images and videos are migrated to the English locale.
Yes, but for every articles with localized images, I have the message: "The image XXX.jpg does not exist".
See, for instance:
https://support-stage-new.mozilla.com/fr/kb/Fen%C3%AAtre%20des%20Options%20-%20Panneau%20Contenu#os=win&browser=fx35

> Please check again tomorrow as we are getting a new migration imported today.
> If this is still true, reply here and we will look into it.
It is still true for the example I gave.

> I'm leaving items 7 and 8 to the SUMO team.
I am waiting the answer from the SUMO team.
Item 7:

(In reply to comment #37)
> Here is a copy of bug 611584 comment 0:
> {SHOWFOR} for browser version has been used to differentiate:
> * FF 3.0 from FF 2.0 with {SHOWFOR(browser=firefox3)}
> * FF 3.5/3.6 from FF 3.0 with {SHOWFOR(browser=firefox3.5)} or
> {SHOWFOR(browser=firefox3.6)}
> In order to prevent some blank sections for FF 4.0, the migration script must
> replace:
> * {SHOWFOR(browser=firefox3)} by {for fx3, fx4}
> * {SHOWFOR(browser=firefox3.5)} by {for fx36, fx4}
> * {SHOWFOR(browser=firefox3.6)} by {for fx36, fx4}

As far as I can tell, this wouldn't work properly. Those showfors are in most cases for specific differences between revisions, so we can't just stick Fx4 in there. We need to check that with every instance of showfor.

Item 8:

(In reply to comment #44)
> In addition to comment 40 about article state:
> If we want to start cleanly, every localized approved article that has an
> history date before english parent's one must be in an "Translations Needing
Updates",
> except if it is already in "out-of-date translations" because either it is in
> this state in tiki SUMO or for comment 40 reason.

Not sure about that, why would we want to mark an article as "out of date" if we only changed a typo in the English version on Tiki? IMHO only articles that were marked as out of date on Tiki should be marked as out of date on Kitsune as well.
(In reply to comment #59)
> It has not been already fixed, at least for the testcase I gave.
I'm not sure what the behavior should be here...
> 
> OK. Here are articles that are wrongly classified as English:
These are all English in Tiki, as far as I can tell. Will need to be fixed manually, I guess.

> Yes, but for every articles with localized images, I have the message: "The
> image XXX.jpg does not exist".
> See, for instance:
> https://support-stage-new.mozilla.com/fr/kb/Fen%C3%AAtre%20des%20Options%20-%20Panneau%20Contenu#os=win&browser=fx35
I didn't migrate this image... I have an outdated list. We'll do an up-to-date list when we get closer to the push.

You can try uploading it yourself and make it have an English locale. It should show up in French documents then.
> I'm not sure what the behavior should be here...
Here is my answer according to the bullets of comment 55 item 1:
* I checked that "to be approved for the first time" articles are now in Kitsune in "Review" status, in "Unreviewed changes" in localization dashboard, with a message "This article doesn't have approved content yet. ". For me, it is correct. But if you try to edit article, current content is cleared.
* If the last revision date/time of an approved article (Article title) is after the last revision date/time of the staging copy of this article (*Article title), then "Article title" is translated.
This is done and on stage:
http://support-stage.mozilla.com/

If any blocking issues arise, feel free to reopen or file another bug.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Rebecca found the following issue:
A select few articles actually have %2A in their title (have = start with). These are not viewable (visiting the URL shows a 404 -- bug 615722) and would need to be manually fixed (from the admin) or deleted. Here is the list:

mysql> select slug, locale from wiki_document where title like '%%%2A%';
+-------------------------------------------------+--------+
| slug                                            | locale |
+-------------------------------------------------+--------+
| %2ACannot%20clear%20Location%20bar%20history    | en-US  |
| %2APDF-Dateien%20in%20Firefox%20%C3%B6ffnen     | en-US  |
| %2AHow%20to%20set%20the%20home%20page           | en-US  |
| %2AProfiles                                     | en-US  |
| %2APrivate%20Browsing                           | fa     |
| %2AFirefox%20ne%20d%C3%A9marre%20pas            | fr     |
| %2AHow%20to%20customize%20the%20toolbar         | id     |
| %2AThe%20Adobe%20Flash%20plugin%20has%20crashed | ru     |
+-------------------------------------------------+--------+
8 rows in set (0.14 sec)
Verified articles display, localized articles are there also. Verified buttons, images, and videos display correctly. Includes and links are also displayed correctly. Please re-open if other issues are found.

A separate bug was filed regarding the pre-fix issues with '%2A'
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.