551738 - Push SUMO 1.5.2 to production Thursday night 3/11/2010

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Laura Thomson :laura]

The tag is https://svn.mozilla.org/projects/sumo/tags/1.5/1.5.2_r64063_20100311/

Comment 1

[Laura Thomson :laura]

ToDo items:

- Run scripts/clean_up_users.php (bug 464087)
- Maybe run scripts/livechat/migration-script.php from bug 534351, blocked by bug 551751 (this shouldn't block the push though)

Comment 2

[James Socol [:jsocol, :james]]

- Add the SQL Listen directives to sphinx's localsettings.py, for bug 546757.

Comment 3

[Laura Thomson :laura]

OK, so apparently migration-script.php *was* run on stage, so that is not blocked.
Also
- Restart sphinxd

Comment 4

[Matthew Middleton (:zzxc)]

The process for activating Live Chat login migration is:

1. Shut down Openfire
2. Install the new RPM and update openfire.xml, following all the steps in https://bugzilla.mozilla.org/show_bug.cgi?id=541983#c13
4. Start Openfire
5. Do steps 1-4 in https://bugzilla.mozilla.org/show_bug.cgi?id=541983#c9 (I can do these steps myself, if preferred)
6. Shut down Openfire again
7. Update local.php on SUMO, adding the lines from https://bugzilla.mozilla.org/show_bug.cgi?id=541982#c7
8. Execute the SQL in https://bugzilla.mozilla.org/show_bug.cgi?id=541982#c12
9. Run the migration script on SUMO.  (bug 541982, just execute the script with no arguments)
10. Execute the following line of SQL on the SUMO database to insert the migrated accounts into the "Live Chat helpers" group:

INSERT INTO users_usergroups (userId, groupName) (SELECT userId, "Live Chat helpers" FROM users_users WHERE livechat_id IS NOT NULL);

11. Restart Openfire, and verify that migrated and non-migrated accounts work as expected.
12. Migration is complete, all tests at https://wiki.mozilla.org/Support/Live_Chat/SUMO_login/Testing should pass.

Comment 5

[Matthew Middleton (:zzxc)]

Also, in step 5 above, setting "sumologin.jdbc.connectionString" is no longer required due to bug 545947.

Comment 6

[Shyam Mani [:fox2mike]]

Openfire upgraded, sumo has been svn'd up.

[root@mradm02 support.mozilla.com]# svn switch https://svn.mozilla.org/projects/sumo/tags/1.5/1.5.2_r64063_20100311/
D    tests
A    webroot/lang/bg
A    webroot/lang/bg/language.php
A    webroot/lang/bg/index.php
U    webroot/lang/langmapping.php
U    webroot/css/admin.css
U    webroot/search.php
A    webroot/tiki-register_livechat.php
U    webroot/lib/pagelist/pagelistlib.php
U    webroot/lib/setup/language.php
U    webroot/lib/userslib.php
A    webroot/styles/mozcommon/searchmodule.gif
A    webroot/styles/livechat_migration.css
A    webroot/styles/mozms/sidebar-mobile.png
U    webroot/styles/mozms2.css
A    webroot/js/livechat_migration.js
U    webroot/js/wikiplugin_eventsignup.js
U    webroot/js/forumquestion.js
U    webroot/tiki-adminusers.php
A    webroot/tiki-livechat_migration.php
U    webroot/tiki-register.php
U    webroot/templates/styles/mozkb/tiki-user_preferences.tpl
U    webroot/templates/styles/mozkb/comment-footer.tpl
U    webroot/templates/styles/mozkb/tiki-view_forum.tpl
U    webroot/templates/styles/mozkb/forum_advanced_post.tpl
A    webroot/templates/styles/mozkb/tiki-register_livechat.tpl
A    webroot/templates/styles/mozkb/tiki-livechat_migration.tpl
A    webroot/templates/styles/mozkb/tiki-adminusers.tpl
A    webroot/templates/styles/mozfh/comment-footer.tpl
A    webroot/templates/styles/mozlc/comment-footer.tpl
A    webroot/templates/styles/mozms/comment-footer.tpl
A    webroot/templates/styles/mozih/comment-footer.tpl
A    webroot/templates/styles/mozms2/comment-footer.tpl
A    webroot/templates/styles/mozad/comment-footer.tpl
A    webroot/templates/styles/mozgn/comment-footer.tpl
A    webroot/templates/styles/mozmswin7/sidebar.tpl
 U   webroot/templates/styles/mozmswin7/searchbar.tpl
 U   webroot/templates/styles/mozmswin7/index.php
 U   webroot/templates/styles/mozmswin7/tiki.tpl
A    webroot/templates/styles/mozgn2/comment-footer.tpl
U    webroot/templates/styles/mozcommon/footer.tpl
A    webroot/templates/styles/mozip/comment-footer.tpl
A    scripts/livechat
A    scripts/livechat/migration-script.php
A    scripts/livechat/migration.conf.php
D    scripts/metrics/j_update_facts.kjb
D    scripts/metrics/j_sumo_etl.kjb
D    scripts/metrics/j_sumo_intialize.kjb
D    scripts/metrics/dwschema
D    scripts/metrics/t_update_dimensions.ktr
A    scripts/metrics/t_sync_sa.ktr
A    scripts/metrics/j_sumo_initialize.kjb
A    scripts/metrics/t_update_f_periodiccounts.ktr
A    scripts/metrics/j_update_f_events.kjb
U    scripts/metrics/generated_date.ktr
A    scripts/metrics/t_setvariables_date.ktr
A    scripts/metrics/t_update_d_user.ktr
U    scripts/metrics/t_setvariables.ktr
A    scripts/metrics/j_update_sa_chatlogs.kjb
A    scripts/metrics/t_update_d_status.ktr
A    scripts/metrics/t_update_d_article.ktr
A    scripts/metrics/t_get_past_periods.ktr
A    scripts/metrics/j_update_f_periodiccounts.kjb
A    scripts/metrics/j_update_sa_kbvotes.kjb
U    scripts/metrics/t_update_events_ft.ktr
A    scripts/metrics/j_update_sa.kjb
U    scripts/metrics/kettle.properties.dist
U    scripts/metrics/j_update_dimensions.kjb
A    scripts/metrics/j_sumo_generate_dates.kjb
A    scripts/metrics/t_update_d_thread.ktr
A    scripts/metrics/generate_d_date.kjb
A    scripts/metrics/j_sincronize_day_period.kjb
A    scripts/metrics/j_update_d_user.kjb
A    scripts/metrics/t_setvariables_voteDate.ktr
A    scripts/metrics/t_setvariables_startTime.ktr
A    scripts/metrics/t_isFirstChat.ktr
A    scripts/metrics/j_populate_sumodw.kjb
A    scripts/metrics/t_sync_sa_chatlogs.ktr
A    scripts/metrics/t_update_d_chatuser.ktr
A    scripts/metrics/t_update_d_chat.ktr
A    scripts/metrics/t_update_d_lang.ktr
A    scripts/metrics/t_set_today.ktr
A    scripts/metrics/t_sync_sa_kbvotes.ktr
U    scripts/sphinx/localsettings.py-dist
U    scripts/sphinx/sphinx.conf
U    scripts/minify/minify.conf.php.dist
A    scripts/clean_up_users.php
Updated to revision 64096.

Comment 7

[James Socol [:jsocol, :james]]

Step 7.5 for comment 4: Run the SQL in https://bug534340.bugzilla.mozilla.org/attachment.cgi?id=417205

Comment 8

[Shyam Mani [:fox2mike]]

That's taken care of : 

mysql> ALTER TABLE users_users ADD livechat_id VARCHAR( 255 ) NULL DEFAULT NULL ,
    -> # Add unique key
    -> ADD UNIQUE (livechat_id);
Query OK, 163237 rows affected (16.92 sec)
Records: 163237  Duplicates: 0  Warnings: 0

Comment 9

[Shyam Mani [:fox2mike]]

mysql> UPDATE users_users SET livechat_id=NULL;
Query OK, 0 rows affected (0.45 sec)
Rows matched: 163237  Changed: 0  Warnings: 0

And that.

Comment 10

[Shyam Mani [:fox2mike]]

[root@pm-app-generic05 livechat]# php -f migration-script.php 
1a. Updating Tiki database for same-email users... 345 distinct emails... Done.
1b. Emailing the above Openfire users... Done.
1c. Removing them from the Openfire database... Done.
2.  Emailing rest of OpenFire accounts (with email set)... [e-mail]@nathan.tuggycomputer.com... Invalid route address
1276 distinct emails... Done
Could not email the following: (format user;email)
    tuggyne;[e-mail]@nathan.tuggycomputer.com

3.  Creating livechat_id for non-conflicting TikiWiki accounts... 162302 rows... PHP Fatal error:  Call to undefined function bcsub() in /data/www/support.mozilla.com/scripts/livechat/migration-script.php on line 234

Last error can be ignored according to james and laura.

Comment 11

[Shyam Mani [:fox2mike]]

mysql> INSERT INTO users_usergroups (userId, groupName) (SELECT userId, "Live Chat
    "> helpers" FROM users_users WHERE livechat_id IS NOT NULL);
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    146189209
Current database: support_mozilla_com

Query OK, 162047 rows affected (4.06 sec)
Records: 162047  Duplicates: 0  Warnings: 0

Comment 12

[Shyam Mani [:fox2mike]]

[root@pm-app-generic05 scripts]# php -f clean_up_users.php 
Getting list of users to remove... done in 1.05 seconds.
Ignoring users from tiki_actionlog... done in 0.82 seconds.
Ignoring users from tiki_user_postings... done in 0.65 seconds.
Ignoring users from tiki_comments... done in 17.87 seconds.
Deleting from users_usergroups... done in 336.79 seconds.
Deleting from tiki_user_preferences... done in 1059.89 seconds.
Deleting from users_users... done in 405.52 seconds.
Deleted 106936 users and a total of 3606632 rows.

Script executed in 1822.6 seconds.

Sphinx configs updated as well

Comment 13

[Shyam Mani [:fox2mike]]

Openfire seems fine, calling this done.

Feel free to re-open if you see issues related to this push.

Comment 14

[James Socol [:jsocol, :james]]

The sphinx.conf file seems to have ended up in a strange state: It has the SQL listen directive, but not the keywords field in the wiki_pages index, which is odd because the keywords line was added first.

Comment 15

[Shyam Mani [:fox2mike]]

Fixed!

Comment 16

[Stephen Donner [:stephend] Not actively reading bugmail]

We tested a bunch of bugs again on production [1], and they were all re-verified.

Verified FIXED

[1] Bug 548439, bug 551224, bug 535604, bug 529730, as well as ad-hoc tested things like login/posting.

Comment 17

[Stephen Donner [:stephend] Not actively reading bugmail]

Looks like I spoke too soon; see bug 551911 (thankfully, I think it can be dealt with separately; no need to reopen this bug, I don't think).