Closed Bug 551738 Opened 14 years ago Closed 14 years ago

Push SUMO 1.5.2 to production Thursday night 3/11/2010

Categories

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Other
Platform:
All
Other
Type:
task
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: laura, Assigned: fox2mike)

Details

(Whiteboard: 2010-03-11 @7pm)

Assignee: server-ops → shyam
Flags: needs-downtime+
Whiteboard: 2010-03-11 @7pm
ToDo items:

- Run scripts/clean_up_users.php (bug 464087)
- Maybe run scripts/livechat/migration-script.php from bug 534351, blocked by bug 551751 (this shouldn't block the push though)
- Add the SQL Listen directives to sphinx's localsettings.py, for bug 546757.
OK, so apparently migration-script.php *was* run on stage, so that is not blocked.
Also
- Restart sphinxd
The process for activating Live Chat login migration is:

1. Shut down Openfire
2. Install the new RPM and update openfire.xml, following all the steps in https://bugzilla.mozilla.org/show_bug.cgi?id=541983#c13
4. Start Openfire
5. Do steps 1-4 in https://bugzilla.mozilla.org/show_bug.cgi?id=541983#c9 (I can do these steps myself, if preferred)
6. Shut down Openfire again
7. Update local.php on SUMO, adding the lines from https://bugzilla.mozilla.org/show_bug.cgi?id=541982#c7
8. Execute the SQL in https://bugzilla.mozilla.org/show_bug.cgi?id=541982#c12
9. Run the migration script on SUMO.  (bug 541982, just execute the script with no arguments)
10. Execute the following line of SQL on the SUMO database to insert the migrated accounts into the "Live Chat helpers" group:

INSERT INTO users_usergroups (userId, groupName) (SELECT userId, "Live Chat helpers" FROM users_users WHERE livechat_id IS NOT NULL);

11. Restart Openfire, and verify that migrated and non-migrated accounts work as expected.
12. Migration is complete, all tests at https://wiki.mozilla.org/Support/Live_Chat/SUMO_login/Testing should pass.
Also, in step 5 above, setting "sumologin.jdbc.connectionString" is no longer required due to bug 545947.
Openfire upgraded, sumo has been svn'd up.

[root@mradm02 support.mozilla.com]# svn switch https://svn.mozilla.org/projects/sumo/tags/1.5/1.5.2_r64063_20100311/
D    tests
A    webroot/lang/bg
A    webroot/lang/bg/language.php
A    webroot/lang/bg/index.php
U    webroot/lang/langmapping.php
U    webroot/css/admin.css
U    webroot/search.php
A    webroot/tiki-register_livechat.php
U    webroot/lib/pagelist/pagelistlib.php
U    webroot/lib/setup/language.php
U    webroot/lib/userslib.php
A    webroot/styles/mozcommon/searchmodule.gif
A    webroot/styles/livechat_migration.css
A    webroot/styles/mozms/sidebar-mobile.png
U    webroot/styles/mozms2.css
A    webroot/js/livechat_migration.js
U    webroot/js/wikiplugin_eventsignup.js
U    webroot/js/forumquestion.js
U    webroot/tiki-adminusers.php
A    webroot/tiki-livechat_migration.php
U    webroot/tiki-register.php
U    webroot/templates/styles/mozkb/tiki-user_preferences.tpl
U    webroot/templates/styles/mozkb/comment-footer.tpl
U    webroot/templates/styles/mozkb/tiki-view_forum.tpl
U    webroot/templates/styles/mozkb/forum_advanced_post.tpl
A    webroot/templates/styles/mozkb/tiki-register_livechat.tpl
A    webroot/templates/styles/mozkb/tiki-livechat_migration.tpl
A    webroot/templates/styles/mozkb/tiki-adminusers.tpl
A    webroot/templates/styles/mozfh/comment-footer.tpl
A    webroot/templates/styles/mozlc/comment-footer.tpl
A    webroot/templates/styles/mozms/comment-footer.tpl
A    webroot/templates/styles/mozih/comment-footer.tpl
A    webroot/templates/styles/mozms2/comment-footer.tpl
A    webroot/templates/styles/mozad/comment-footer.tpl
A    webroot/templates/styles/mozgn/comment-footer.tpl
A    webroot/templates/styles/mozmswin7/sidebar.tpl
 U   webroot/templates/styles/mozmswin7/searchbar.tpl
 U   webroot/templates/styles/mozmswin7/index.php
 U   webroot/templates/styles/mozmswin7/tiki.tpl
A    webroot/templates/styles/mozgn2/comment-footer.tpl
U    webroot/templates/styles/mozcommon/footer.tpl
A    webroot/templates/styles/mozip/comment-footer.tpl
A    scripts/livechat
A    scripts/livechat/migration-script.php
A    scripts/livechat/migration.conf.php
D    scripts/metrics/j_update_facts.kjb
D    scripts/metrics/j_sumo_etl.kjb
D    scripts/metrics/j_sumo_intialize.kjb
D    scripts/metrics/dwschema
D    scripts/metrics/t_update_dimensions.ktr
A    scripts/metrics/t_sync_sa.ktr
A    scripts/metrics/j_sumo_initialize.kjb
A    scripts/metrics/t_update_f_periodiccounts.ktr
A    scripts/metrics/j_update_f_events.kjb
U    scripts/metrics/generated_date.ktr
A    scripts/metrics/t_setvariables_date.ktr
A    scripts/metrics/t_update_d_user.ktr
U    scripts/metrics/t_setvariables.ktr
A    scripts/metrics/j_update_sa_chatlogs.kjb
A    scripts/metrics/t_update_d_status.ktr
A    scripts/metrics/t_update_d_article.ktr
A    scripts/metrics/t_get_past_periods.ktr
A    scripts/metrics/j_update_f_periodiccounts.kjb
A    scripts/metrics/j_update_sa_kbvotes.kjb
U    scripts/metrics/t_update_events_ft.ktr
A    scripts/metrics/j_update_sa.kjb
U    scripts/metrics/kettle.properties.dist
U    scripts/metrics/j_update_dimensions.kjb
A    scripts/metrics/j_sumo_generate_dates.kjb
A    scripts/metrics/t_update_d_thread.ktr
A    scripts/metrics/generate_d_date.kjb
A    scripts/metrics/j_sincronize_day_period.kjb
A    scripts/metrics/j_update_d_user.kjb
A    scripts/metrics/t_setvariables_voteDate.ktr
A    scripts/metrics/t_setvariables_startTime.ktr
A    scripts/metrics/t_isFirstChat.ktr
A    scripts/metrics/j_populate_sumodw.kjb
A    scripts/metrics/t_sync_sa_chatlogs.ktr
A    scripts/metrics/t_update_d_chatuser.ktr
A    scripts/metrics/t_update_d_chat.ktr
A    scripts/metrics/t_update_d_lang.ktr
A    scripts/metrics/t_set_today.ktr
A    scripts/metrics/t_sync_sa_kbvotes.ktr
U    scripts/sphinx/localsettings.py-dist
U    scripts/sphinx/sphinx.conf
U    scripts/minify/minify.conf.php.dist
A    scripts/clean_up_users.php
Updated to revision 64096.
That's taken care of : 

mysql> ALTER TABLE users_users ADD livechat_id VARCHAR( 255 ) NULL DEFAULT NULL ,
    -> # Add unique key
    -> ADD UNIQUE (livechat_id);
Query OK, 163237 rows affected (16.92 sec)
Records: 163237  Duplicates: 0  Warnings: 0
mysql> UPDATE users_users SET livechat_id=NULL;
Query OK, 0 rows affected (0.45 sec)
Rows matched: 163237  Changed: 0  Warnings: 0

And that.
[root@pm-app-generic05 livechat]# php -f migration-script.php 
1a. Updating Tiki database for same-email users... 345 distinct emails... Done.
1b. Emailing the above Openfire users... Done.
1c. Removing them from the Openfire database... Done.
2.  Emailing rest of OpenFire accounts (with email set)... bugzilla@nathan.tuggycomputer.combugzilla@nathan.tuggycomputer.com... Invalid route address
1276 distinct emails... Done
Could not email the following: (format user;email)
    tuggyne;bugzilla@nathan.tuggycomputer.combugzilla@nathan.tuggycomputer.com

3.  Creating livechat_id for non-conflicting TikiWiki accounts... 162302 rows... PHP Fatal error:  Call to undefined function bcsub() in /data/www/support.mozilla.com/scripts/livechat/migration-script.php on line 234

Last error can be ignored according to james and laura.
mysql> INSERT INTO users_usergroups (userId, groupName) (SELECT userId, "Live Chat
    "> helpers" FROM users_users WHERE livechat_id IS NOT NULL);
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    146189209
Current database: support_mozilla_com

Query OK, 162047 rows affected (4.06 sec)
Records: 162047  Duplicates: 0  Warnings: 0
[root@pm-app-generic05 scripts]# php -f clean_up_users.php 
Getting list of users to remove... done in 1.05 seconds.
Ignoring users from tiki_actionlog... done in 0.82 seconds.
Ignoring users from tiki_user_postings... done in 0.65 seconds.
Ignoring users from tiki_comments... done in 17.87 seconds.
Deleting from users_usergroups... done in 336.79 seconds.
Deleting from tiki_user_preferences... done in 1059.89 seconds.
Deleting from users_users... done in 405.52 seconds.
Deleted 106936 users and a total of 3606632 rows.

Script executed in 1822.6 seconds.

Sphinx configs updated as well
Openfire seems fine, calling this done.

Feel free to re-open if you see issues related to this push.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
The sphinx.conf file seems to have ended up in a strange state: It has the SQL listen directive, but not the keywords field in the wiki_pages index, which is odd because the keywords line was added first.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Fixed!
Status: REOPENED → RESOLVED
Closed: 14 years ago → 14 years ago
Resolution: --- → FIXED
We tested a bunch of bugs again on production [1], and they were all re-verified.

Verified FIXED

[1] Bug 548439, bug 551224, bug 535604, bug 529730, as well as ad-hoc tested things like login/posting.
Status: RESOLVED → VERIFIED
Looks like I spoke too soon; see bug 551911 (thankfully, I think it can be dealt with separately; no need to reopen this bug, I don't think).
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.