488878 - MySQL data base for staging buildbot masters

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Chris AtLee [:catlee]]

I need a mysql database set up which is accessible from:

staging-master.build.mozilla.org
qm-buildbot01.mozilla.org
sm-staging-try-master.mozilla.org

Also, two mysql user accounts on the database: one will full privileges (to create/drop tables, add indexes, etc.), and one with read-only privs.

Comment 1

[Dave Miller [:justdave]]

how much data is it going to be pushing?  Is it just coordinating buildbot data or are people going to be running reports off it, or?  Lots of number crunching, or just quick access storage?

Comment 2

[Chris AtLee [:catlee]]

(In reply to comment #1)
> how much data is it going to be pushing?

Maybe a few dozen INSERT / UPDATES every minute?  That's a bit of a wild guess right now.  The size of the data is pretty small - just start/end times, and properties for builds.

> Is it just coordinating buildbot data
> or are people going to be running reports off it, or?  Lots of number
> crunching, or just quick access storage?

Mostly quick access storage in terms of reporting.  I'm hoping to be able to do a few queries to generate reports, but anything complicated would be offloaded to the report generator.

Comment 3

[Dave Miller [:justdave]]

ok, B01 cluster sounds like a good place to put it. It's critical if it breaks because we can't build, but it's not user-facing, so it's not the end of the world. :)  We also don't want it on the C cluster because stuff there dies every time someone spiders SFX. ;)

Probably be over the weekend for me to set it up because I'm about to walk out the door and I'll be offline until early Saturday.  I can probably have it for you tomorrow mid-day though.

Comment 4

[Chris AtLee [:catlee]]

Awesome thanks.

Comment 5

[Reed Loden [:reed]]

Why not the _staging_ cluster since this is for _staging_ buildbot masters? None of this is for production stuff, right?

Comment 6

[Dave Miller [:justdave]]

(In reply to comment #5)
> Why not the _staging_ cluster since this is for _staging_ buildbot masters?
> None of this is for production stuff, right?

When I saw "staging" I thought of the process of pushing builds to the ftp staging server and figured it just meant these servers were pushing there.  If that's not the case, then this is a good point...

Comment 7

[Chris AtLee [:catlee]]

Yes, this is just for our development aka staging machines for now.

Comment 8

[Dave Miller [:justdave]]

OK, this is going on tm-stage01-master01 (and you can feel free to use tm-stage01-slave01 for any read-only queries).  If you move this software into a produciton capacity, we should set up a similar database on one of the B-level database clusters for it to use.

Username/password info has been mailed.

dmoore: need firewall poked:

Access to: 10.2.70.130:3306 and 10.2.70.131:3306
     from: 10.2.71.208, 10.2.73.14, and 10.2.76.35

Comment 9

[Derek Moore [:dmoore]]

Firewall changes made as requested.

Comment 10

[Chris AtLee [:catlee]]

(In reply to comment #8)

> dmoore: need firewall poked:
> 
> Access to: 10.2.70.130:3306 and 10.2.70.131:3306
>      from: 10.2.71.208, 10.2.73.14, and 10.2.76.35

Can you also allow access from 10.2.73.185 (talos-staging-master)?

Comment 11

[Derek Moore [:dmoore]]

(In reply to comment #10)

> Can you also allow access from 10.2.73.185 (talos-staging-master)?

Changes made as requested.