555003 - Push desktop and mobile support to production on 4/6

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[James Socol [:jsocol, :james]]

This is basically an SVN UP + a few configuration changes (for us) + setting up the sphinx server with the new indexer. There may be SQL but I don't think so (I'll look it up). There's no tag for mobile, so just svn up.

Comment 1

[James Socol [:jsocol, :james]]

Stuff to do:

IT: 
* cp webroot/js/wikiplugin_showfor.js.dist-fennec webroot/js/wikiplugin_showfor.js
* run the SQL in https://bugzilla.mozilla.org/attachment.cgi?id=425557
* run the SQL in https://bugzilla.mozilla.org/attachment.cgi?id=432913

SUMO team:
* create the page in bug 535509
* enable forum friendly URLs in tiki-admin.php?page=forums
* enable support forum features for forum 5

Comment 2

[matthew zeier [:mrz]]

Bug was filed too late to get into tonight's window.  

Can this wait until next Tuesday?

Comment 3

[James Socol [:jsocol, :james]]

We can wait till Tuesday but in that case both mobile and the next desktop release should be ready to go. Changed the summary to reflect that.

We don't have a tag for desktop yet, but I'll post it when we do. That one will involve setting up Gearman workers; I'll talk to Shyam about those tonight.

Comment 4

[James Socol [:jsocol, :james]]

Also need to run webroot/htaccess.sh to pick up some rewrite changes.

Comment 5

[matthew zeier [:mrz]]

We have a release tomorrow night - is this push a lot of work?  Something that can wait till Thursday?

Comment 6

[James Socol [:jsocol, :james]]

(In reply to comment #5)
> We have a release tomorrow night - is this push a lot of work?  Something that
> can wait till Thursday?

It shouldn't be too much work, except for getting the gearman workers set up, which I'm working through with Shyam right now on stage.

We were aiming to get this out this quarter but if it's going to take too many resources away from the release then we can wait.

Comment 7

[James Socol [:jsocol, :james]]

We ran into a snag. Looks like we'll need to hold off until Thursday, anyway.

And adding to the list of tasks:

Update sphinx configs from SVN.

Comment 8

[matthew zeier [:mrz]]

If the only reason to push is to hit a Q1 goal, I'd consider it a hit in the spirit of the goal.  

Let's shelve this for tonight and do a Wednesday night push instead.  Any objections?

Comment 9

[James Socol [:jsocol, :james]]

(In reply to comment #8)
> If the only reason to push is to hit a Q1 goal, I'd consider it a hit in the
> spirit of the goal.  

Agreed.

> Let's shelve this for tonight and do a Wednesday night push instead.  Any
> objections?

If QA agrees, then it's fine with me.

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

Sure, we can do Wednesday night.

Comment 11

[Shyam Mani [:fox2mike]]

Whoever is pushing this out, if Bug 551513 is fixed, the docs for the gearman side of things are at https://intranet.mozilla.org/SysAdmin/index.php/Gearman

Feel free to call me over my phone (the one ending with 74) in case of any issues or if the doc isn't clear enough.

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

mrz: are we pushing this out tonight?  Thanks!

Comment 13

[matthew zeier [:mrz]]

sorry no - was going by the bug summary for 04/01 and must have typo'd the whiteboard.  updated.

Comment 14

[James Socol [:jsocol, :james]]

(In reply to comment #13)
> sorry no - was going by the bug summary for 04/01 and must have typo'd the
> whiteboard.  updated.

We were going by what you said in comment 8. Again, if 4/1 works for QA, it works for me.

Comment 15

[Stephen Donner [:stephend] Not actively reading bugmail]

4/1 is fine for Vishal and I.

Comment 16

[James Socol [:jsocol, :james]]

Tag for *desktop* is:
https://svn.mozilla.org/projects/sumo/tags/1.5/1.5.3_r65201_20100401

*Mobile* has no tag and is just an SVN UP.

We backed out the changes in bug 532498, so there is no gearman dependency to set up now. (Thanks for the help getting everything, Shyam. We'll start using it from Kitsune sooner than later.)

The tasks in comment 1 and comment 4 still need to happen.

Comment 17

[matthew zeier [:mrz]]

Pushing off, 3.6.3 taking precedence.

Comment 18

[Shyam Mani [:fox2mike]]

James, 

What needs to be done on the sphinx side of things?

Comment 19

[James Socol [:jsocol, :james]]

(In reply to comment #18)
> What needs to be done on the sphinx side of things?

Update the configs from SVN and reindex. The localsettings.py's should already have AGE_DIVISOR=86400

Comment 20

[Shyam Mani [:fox2mike]]

Desktop :

[root@mradm02 support.mozilla.com]# svn switch https://svn.mozilla.org/projects/sumo/tags/1.5/1.5.3_r65201_20100401
U    webroot/search.php
U    webroot/db/local.php.dist
U    webroot/lib/admin/adminlib.php
U    webroot/lib/userslib.php
U    webroot/styles/livechat_migration.css
U    webroot/templates/styles/mozkb/wiki-feedback.tpl
U    webroot/templates/styles/mozkb/mail/user_watch_wiki_page_changed.tpl
U    webroot/templates/styles/mozkb/tiki-livechat_migration.tpl
U    webroot/templates/styles/mozms2/searchbar.tpl
U    webroot/templates/styles/mozgn/tiki.tpl
U    webroot/templates/styles/mozmswin7/searchbar.tpl
U    webroot/htaccess.dist
U    scripts/sphinx/localsettings.py-dist
U    scripts/sphinx/sphinx.conf
U    scripts/minify/minify.conf.php.dist
Updated to revision 65393.

Mobile :

[root@mradm02 mobile.support.mozilla.com]# svn up
U    webroot/js/wikiplugin_showfor.js.dist-fennec
A    webroot/js/mozsadvanced.js
U    webroot/comments.php
U    webroot/tiki-newsearch.php
U    webroot/tiki-view_forum_thread.php
A    webroot/search.php
U    webroot/tiki-admin_include_forums.php
A    webroot/img/silk/original.png
A    webroot/img/silk/unsolve.png
A    webroot/images/search
A    webroot/images/search/active-tab.png
A    webroot/images/search/submit-button-bg.png
A    webroot/images/search/wait.gif
U    webroot/lib/search/sphinxlib.php
A    webroot/lib/search/suggest.php
A    webroot/lib/search/dictionaries
A    webroot/lib/search/dictionaries/add_to_dict.php
A    webroot/lib/search/dictionaries/sumo-en.pws
U    webroot/lib/setup/language.php
U    webroot/lib/tikilib.php
U    webroot/lib/admin/adminlib.php
U    webroot/lib/commentslib.php
U    webroot/lib/tiki-js.js
U    webroot/lib/notifications/notificationemaillib.php
U    webroot/styles/mozfr.css
U    webroot/styles/mozms.css
A    webroot/styles/jquery-ui-1.7.1.custom.css
U    webroot/styles/mozfr/ie7.css
U    webroot/styles/mozcommon.css
U    webroot/styles/mozms/ie.css
U    webroot/styles/mozms/ie7.css
A    webroot/styles/mozsadvanced.css
U    webroot/styles/mozcommon/ie.css
U    webroot/styles/mozcommon/ie7.css
U    webroot/tiki-view_forum.php
A    webroot/tiki-admin_search.php
A    webroot/templates/search.tpl
U    webroot/templates/styles/mozkb/comments.tpl
U    webroot/templates/styles/mozkb/wiki-feedback.tpl
U    webroot/templates/styles/mozkb/tiki-edit_help_tool.tpl
U    webroot/templates/styles/mozkb/comment-footer.tpl
U    webroot/templates/styles/mozkb/tiki-view_forum.tpl
U    webroot/templates/styles/mozkb/forum_advanced_post.tpl
U    webroot/templates/styles/mozkb/mail/forum_post_notification.tpl
U    webroot/templates/styles/mozkb/comment-header.tpl
U    webroot/templates/styles/mozkb/comment-body.tpl
A    webroot/templates/styles/mozfh/comment-footer.tpl
A    webroot/templates/styles/mozlc/comment-footer.tpl
A    webroot/templates/styles/mozms/comment-footer.tpl
A    webroot/templates/styles/mozih/comment-footer.tpl
U    webroot/templates/styles/mozms2/searchbar.tpl
A    webroot/templates/styles/mozms2/comment-footer.tpl
A    webroot/templates/styles/mozad/comment-footer.tpl
A    webroot/templates/styles/mozgn/comment-footer.tpl
A    webroot/templates/styles/mozgn2/comment-footer.tpl
U    webroot/templates/styles/mozfr/comment-icon.tpl
U    webroot/templates/styles/mozcommon/searchbar.tpl
G    webroot/templates/styles/mozcommon/footer.tpl
A    webroot/templates/styles/mozip/comment-footer.tpl
U    webroot/templates/tiki-admin-include-forums.tpl
A    webroot/templates/tiki-admin_search.tpl
U    webroot/htaccess.dist
U    scripts/showfor/config.php.dist-fennec
D    scripts/sphinx/indexer.php
D    scripts/sphinx/indexer.sh
D    scripts/sphinx/indexer-forums.php
A    scripts/sphinx/wordforms.txt
U    scripts/sphinx/search.conf.php.dist
A    scripts/sphinx/localsettings.py-dist
U    scripts/sphinx/stopwords.txt.dist
A    scripts/sphinx/sphinx.conf
U    scripts/minify/minify.conf.php.dist
Updated to revision 65393.

DB Updates for mobile :

mysql> INSERT INTO tiki_urlt_regex_out (regexid, regex_left, regex_right, regex_flags, continue_flag, comment_text) VALUES
    -> (80, '^\\/?tiki-view_forum_thread.php\\?(.*)&?locale=([\\-a-zA-Z]+)(&|&)forumId=([0-9]+)(&|&)comments_parentId=([0-9]+)(&.*)?$', '/$2/forum/$4/$6$1$7', '', '0', ''),
    -> (90, '^\\/?tiki-view_forum_thread\\.php\\?comments_parentId=([0-9]+)&(?:amp;)?forumId=([0-9]+)(?:(?:&(?:amp;)?)(.*))?$', '/en-US/forum/$2/$1?$3', '', '0', ''),
    -> (95, '^\\/?tiki-view_forum.php\\?locale=([\\-a-zA-Z]+)&(?:amp;)?forumId=([0-9]+)(?:&(?:amp;)?(.*))?$', '/$1/forum/$2?$3', NULL, NULL, NULL),
    -> (100, '^\\/?tiki-view_forum.php\\?(.*)&?locale=([\\-a-zA-Z]+)(&|&)forumId=([0-9]+)(&.*)?$', '/$2/forum/$4', NULL, NULL, NULL),
    -> (110, '^\\/?tiki-view_forum.php\\?forumId=([0-9]+)$', '/en-US/forum/$1', NULL, NULL, NULL),
    -> (120, '^\\/?tiki-view_forum.php\\?forumId=([0-9]+)(&|&)(.*)$', '/en-US/forum/$1?$3', NULL, NULL, NULL);
Query OK, 6 rows affected (0.01 sec)
Records: 6  Duplicates: 0  Warnings: 0

mysql> LOCK TABLES `tiki_polls` WRITE;
Query OK, 0 rows affected (0.00 sec)

mysql> /*!40000 ALTER TABLE `tiki_polls` DISABLE KEYS */;
Query OK, 0 rows affected (0.01 sec)

mysql> INSERT INTO `tiki_polls` VALUES (6508,'Firefox Support Forum',0,'o',1222300800,60,3,1);
Query OK, 1 row affected (0.01 sec)

mysql> /*!40000 ALTER TABLE `tiki_polls` ENABLE KEYS */;
Query OK, 0 rows affected (0.00 sec)

mysql> UNLOCK TABLES;
Query OK, 0 rows affected (0.00 sec)

mysql> LOCK TABLES `tiki_poll_options` WRITE;
Query OK, 0 rows affected (0.00 sec)

mysql> /*!40000 ALTER TABLE `tiki_poll_options` DISABLE KEYS */;
Query OK, 0 rows affected (0.02 sec)

mysql> INSERT INTO `tiki_poll_options` VALUES (6508,19847,'1',1,0),(6508,19848,'2',2,0),(6508,19849,'3',3,0),(6508,19850,'4',4,0),(6508,19851,'5',5,0);
Query OK, 5 rows affected (0.00 sec)
Records: 5  Duplicates: 0  Warnings: 0

mysql> /*!40000 ALTER TABLE `tiki_poll_options` ENABLE KEYS */;
Query OK, 0 rows affected (0.02 sec)

mysql> UNLOCK TABLES;
Query OK, 0 rows affected (0.00 sec)

Comment 21

[Shyam Mani [:fox2mike]]

And caches flushed.

Working on sphinx configs

Comment 22

[Shyam Mani [:fox2mike]]

Sphinx for SUMO and Mobile updated, mobile has been moved to pm-app-sphinx* like the rest of the sphinx stuff.

Should be all done!

Comment 23

[James Socol [:jsocol, :james]]

Since we're still seeing very bizarre search behavior, and the sphinx config changes are pretty well isolated, let's roll back the sphinx configs to 1.5.2:

http://svn.mozilla.org/projects/sumo/tags/1.5/1.5.2_r64063_20100311/scripts/sphinx/sphinx.conf

Comment 24

[Shyam Mani [:fox2mike]]

Sphinx config reverted and re-indexed.

Comment 25

[James Socol [:jsocol, :james]]

Now that we've figured out bug 557859, we can push the sphinx config changes.

Comment 26

[Shyam Mani [:fox2mike]]

Done.

Comment 27

[Vishal Kamdar [:vish_moz]]

Verified, FIXED.