582552 - Update bm-l10n-dashboard01 VM from ubuntu 9.04 to 10.04

Status: VERIFIED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Axel Hecht [:Pike]]

Could we get the ubuntu vm bm-l10n-dashboard01 updated from 9.04 to 10.04?

It's the current LTS, and it should make some setup tasks easier, like bug 578703.

If so, how much downtime do you need for this? We should probably get this through in an announced downtime of sorts.

Comment 1

[Shyam Mani [:fox2mike]]

Sure. The only users of the services on that box are l10 folks, so when you guys can afford a 4-5 hour downtime window would be best for us.

I'm giving that much of a window because we've not done this in the past, I'd probably have to push the box up to 9.10 first and then again to 10.04. Should be fairly straightforward and I'm sure it won't take 4 hours...but better have a bigger window and finish earlier than vice-versa.

I could even do this as early as tomorrow, but I'll wait for you to confirm downtime.

Comment 2

[Axel Hecht [:Pike]]

Sweet, works for us, too.

Can you give me a time window in UTC so that I can post an announcement? Or would you do that?

Also, I'd like to shut down the bots manually before we do the actual downtime, can we sync on irc for that?

Comment 3

[Shyam Mani [:fox2mike]]

(In reply to comment #2)
> Can you give me a time window in UTC so that I can post an announcement? Or
> would you do that?

How does 0800 to 1300 UTC on 29/07/2010 sound? 
 
> Also, I'd like to shut down the bots manually before we do the actual downtime,
> can we sync on irc for that?

Sure. The only downtime will be for the reboots to pick up the changes.

Comment 4

[Shyam Mani [:fox2mike]]

(In reply to comment #2)
> Can you give me a time window in UTC so that I can post an announcement? Or
> would you do that?

And I'd be happy to let you post the announcement; I wouldn't have the least idea of where to post :)

Comment 5

[Axel Hecht [:Pike]]

Sounds good, I'll announce with http://www.timeanddate.com/worldclock/fixedtime.html?day=29&month=7&year=2010&hour=8&min=0&sec=0&p1=0.

Comment 6

[Shyam Mani [:fox2mike]]

I've kicked off the upgrade process. We're going to do 9.04 to 9.10 first and then a jump from 9.10 to 10.04 as this is the safest upgrade route recommended.

I'll keep the bug posted with the details/issues if any.

Comment 7

[Shyam Mani [:fox2mike]]

Yay, so 9.04 -> 9.10 finished about 30 mins back, but we've run into some filesystem consistency issues. This is most likely because of the fact that we tailored (read fiddled) with this filesystem for your use.

I'm looking into the issue and will hopefully be able to repair it and bring the filesystem back online.

Will keep you guys updated here and on IRC.

Comment 8

[Shyam Mani [:fox2mike]]

Alright, so we're back up now. Starting the upgrade to 10.04

Comment 9

[Shyam Mani [:fox2mike]]

All done.

[shyam@boris ~]$ ssh bm-l10n-dashboard01
Linux bm-l10n-dashboard01 2.6.32-24-generic #38-Ubuntu SMP Mon Jul 5 09:22:14 UTC 2010 i686 GNU/Linux
Ubuntu 10.04.1 LTS

Party on!

Comment 10

[Axel Hecht [:Pike]]

Bots came up allright, too.

Looking at a df -i, I see a

/var/lib/ureadahead/debugfs

which looks like a mount of /, is that left over from trying to fix the fsck issue?

Comment 11

[Axel Hecht [:Pike]]

Oops, and regular df says

/dev/sda1             12928276  11345232    899080  93% /

Does this need a few package cache cleanups or something?