656444 - upgrade developer-stage9.mozilla.org to MindTouch 2010

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Luke Crouch [:groovecoder]]

We want to test the MindTouch 2010 upgrade (again) on the stage9 server.

Comment 1

[Luke Crouch [:groovecoder]]

http://etherpad.mozilla.org:9000/19COmhelOa

Comment 2

[Luke Crouch [:groovecoder]]

Jake, think we can do this today?

Comment 3

[Jake Maul [:jakem]]

Some problems with this:

1) http://developer.mindtouch.com/en/docs/MindTouch_Administration_Guide/Upgrading_MindTouch/Upgrade_MindTouch_on_CentOS%2f%2fRed_Hat

It wants a newer version of Mono on the servers. I should be able to upgrade devmostage01/02, but cannot make any statements as to how this might affect the existing dekiwiki deployments there (aka: things other than developer-stage9). It should work and I would expect full backwards compatibility, but this *will* affect things other than just -stage9... hopefully only in a positive way, but it's hard to say for certain. Later on we'd need to do this on prod as well.

2) * revert all the security patches
No idea what to do here. Do we have any records as to what or where these security patches are?

3) * svn switch to the mindtouch 2010 branch
https://svn.mindtouch.com/source/public/dekiwiki/
Which branch? 10.0 or 10.1 would be my choices. We are currently using 9.12.3.

4) * reapply all the patches
Would some/all of the patches have been accepted upstream? Need to know what those patches are/were to answer this.

Comment 4

[Jake Maul [:jakem]]

(In reply to comment #3)
> 3) * svn switch to the mindtouch 2010 branch
> https://svn.mindtouch.com/source/public/dekiwiki/
> Which branch? 10.0 or 10.1 would be my choices.
Asked and answered in chat... 10.1 appears to be pre-release... we will use 10.0.

Comment 5

[Luke Crouch [:groovecoder]]

(In reply to comment #3)

2. If we diff against the base 9.12.3 we might be able to see what all we've changed. Hopefully it's just the few official MindTouch security patches.

4. I've made it a habit to only use MindTouch patches so I assume they're the same patches. Of course this is MindTouch we're talking about - we sometimes got 2 or 3 different patches.

Comment 6

[Jake Maul [:jakem]]

Completed following this procedure. Note that step 5 took almost 2 hours on stage, but the site was (mostly?) functional during this step.


1) backup the dekiwiki document root on mradm02
2) backup the database(s) being upgraded
3) switch to the 10.0.9 SVN branch: svn switch
https://svn.mindtouch.com/source/public/dekiwiki/10.0.9/web
4) Deploy to frontends
5) run database upgrade script:  cd dekiwiki/maintenance; php update-db.php
6) issue-multi-command devmostage service dekiwiki9 restart

7) upgrade Mono by loosely following these instructions, but via puppet instead of directly:
http://developer.mindtouch.com/en/docs/mindtouch_setup/010Installation/060Installing_on_CentOS/Installing%2f%2fUpgrade_Mono_on_CentOS

8) issue-multi-command devmostage service dekiwiki9 restart


Issues found:

1) We had a custom robots.txt in place... this broke svn switch. Just move it out of the way *before* switching (after might work too but I don't know how to get a clean switch that way).

2) Very long update-db.php job. Almost 2 hours. Don't know what to do other than wait it out.

3) Puppet / Mono upgrade issues - should be easy to do prod by tweaking what's there now.

4) License file shenanigans- might be avoided during the prod upgrade simply due to the fact that prod isn't already expired. Also, the location of the license.xml seems to have moved. Just confirm with M-T before starting if the current one should still work and if we'll have to move it manually.

Comment 7

[Luke Crouch [:groovecoder]]

2) Let's make an announcement and/or a site-wide notification that the site may have intermittent issues while the upgrade goes on. We might want to avoid heavy revisions during that time.