Closed Bug 512750 Opened 16 years ago Closed 15 years ago

Colo-hosted VMs to run the warning-tracking system

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: benjamin, Assigned: mrz)

Details

I'm currently hosting the warning-tracking system on my local machine, and IT suggested that it would be better to host that at the colo on VMs. It has the following pieces: * a build slave... can use the standard Linux build VM, but the build slave needs about 2.5G of RAM at peak compile time * a buildbot master * a database. Currently I'm using sqlite, but that's really inefficient and pgsql or mysql would be better * a public-facing webserver. The code is running on cherrypy (see http://office.smedbergs.us:8000/new) These could be on one or multiple VMs. Other than the database they really don't need backups.
I recommend using the c01 mysql db cluster. Shyam, can you do this? Update with ETA please.
Assignee: server-ops → shyam
Benjamin, 1) I could probably set you up with a VM with about 4GB of RAM. Would you be comfortable with handling the rest of the setup? (The buildbot master etc). If you'd like me to help with that, I'd need some instructions :) 2) Let me know what you need on the DB side. Ideally, we'd just take queries and set them up, but I guess one of us can chip in and create them for you if needed. 3) Location on svn or someplace similar for the cherrypy stuff, along with instructions on what's needed for it to run.
I can set up the machine if it comes from the build VM image, sure. I'm not sure what you mean about the DB. Are you talking about setting up the initial DB schema? Would that be MySQL or PGSQL? I can probably provide the necessary queries (originally I did it by hand, but it's not complicated). The code for the cherrypy app currently lives in the warning-ui subdirectory of http://hg.mozilla.org/users/bsmedberg_mozilla.com/static-analysis-buildbot/ The system running it will need a python DBAPI adapter for the database as well as cherrypy itself.
(In reply to comment #3) > I'm not sure what you mean about the DB. Are you talking about setting up the > initial DB schema? Would that be MySQL or PGSQL? I can probably provide the > necessary queries (originally I did it by hand, but it's not complicated). Yes, and it would be MySQL.
This should be in MPT on the pm-vmware DRS cluster, dmz is fine. It's taken awhile on this one - can we shoot for end of week?
dm-warning01.mozilla.org is now live on the dmz vlan. It's got 2 processors and 4 GB of RAM. Also, bsmedberg's keys are on the box, login as root. Apart from this, I'll be glad to setup the db side of things if you'd give me the schema and other related stuff. I'll also either add on pass on the auth details your app should use to be able to access the database.
Is this machine going to serve the webapp as well as be the build machine? If this machine, it will need a public-facing DNS name like build-warnings.mozilla.org or somesuch. Which network/VPN do I need to be on to access it? I've gotten a MySQL schema up to replace the old sqlite one: it is available here: http://hg.mozilla.org/users/bsmedberg_mozilla.com/static-analysis-buildbot/file/default/warning-ui/dbschema.sql I'd like my DB user to have permission to create/drop views and stored procedures, since those are likely to be in flux still: the table schema isn't likely to change.
(In reply to comment #7) > Is this machine going to serve the webapp as well as be the build machine? If > this machine, it will need a public-facing DNS name like > build-warnings.mozilla.org or somesuch. Which network/VPN do I need to be on to > access it? Either mrz or me can setup the public side once the app is upp and running. You should be able to access this from the mpt-vpm. > I've gotten a MySQL schema up to replace the old sqlite one: it is available > here: > http://hg.mozilla.org/users/bsmedberg_mozilla.com/static-analysis-buildbot/file/default/warning-ui/dbschema.sql > > I'd like my DB user to have permission to create/drop views and stored > procedures, since those are likely to be in flux still: the table schema isn't > likely to change. Will set this up and update the bug.
What SMTP server should I use to send mail to tinderbox?
smtp.mozilla.org should work.
Mailed DB info to Benjamin. Let me know when you have it setup and working and we'll see about the external URL (I'm all for warnings.build.mozilla.org, which keeps it in tune with what it's going to be doing). You can use dm-warning01.mozilla.org in the meantime for your testing etc.
Did this box really come from the build refplatform? It doesn't have a /tools directory or buildbot. Or is that not part of the VM image?
No, it did not come from a build reference VM, it was built from scratch. You could ask someone from build to copy over what you need or if you specify what's needed, I could copy them over.
I really need it to be the standard VM that releng uses for Linux build slaves (with extra memory if appropriate) I don't know how to recreate that.
That would need approval from Build...
i'll ack - just need to clone the scrubbed VM.
Ben, Can you please help scrub dm-warning01.mozilla.org before I hand it over to Benjamin? Thanks!
i'll scrub this vm shortly
Assignee: shyam → bhearsum
ok, I've scrubbed this machine. I'll send the passwords to bsmedberg and shyam privately.
Assignee: bhearsum → shyam
Benjamin, machine is all yours, I've changed the root password to the infra one, but your keys are on it, so you have access to the machine as root. There is a cltbld user which I would remove if you don't need the same. Do let me know.
The VM is in place. I think when you're done you can open a new bug to setup the external facing VIP and DNS.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Ben, doesn't look like you've started using this VM yet. Is this something you still need?
I'm still running the functions of my local machine, which I was told isn't a good idea, but I don't have a couple days to migrate it, so I don't know when I'll get the chance.
Wasn't this bug spawned in lieu of a UPS? I thought there was a sense of urgency to get it moved. Is there some way IT can help you move content over?
Let's get this migrated in the next couple of weeks - already been sitting for over 2 months with no action.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Been a month, wassup here?
Ben, how can I get this resolved? This was urgent months ago and we're largely waiting on you to help get it moved over.
Assignee: shyam → mrz
I gather this isn't a priority for anyone anymore? We're okay running this production service out of your house? Can you let me know one way or the either so I can kill the VM off?
As far as I can tell it was never a priority. I wiped the local machine this was running on a few weeks ago so I could use it for record/replay, and I haven't had the time to set this up locally or remotely. Eventually I think we should find someone to set it up again, but I don't know who or when.
Okay, i'll mark it as such. IIRC, it was urgent to get it running on a UPS so we had this bug to track moving it our of your house.
Status: REOPENED → RESOLVED
Closed: 16 years ago → 15 years ago
Resolution: --- → INVALID
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.