599147 - mysql access for new buildbot masters

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: NetOps, task)

Description

[Chris AtLee [:catlee]]

I need access for 'buildbot_setup' to the scheduler_staging database on tm-stage01-master01.mozilla.org from 10.12.48.120, 10.12.48.122. (buildbot-master{1,2}.scl1.build.mozilla.com)

Comment 1

[Justin Dow [:jabba]]

also 10.12.48.113 and 10.12.48.115 please, since the masters are going to change their IPs soon.

Comment 3

[Dave Miller [:justdave]]

from bug 599461, looks like 'buildbot' also needs access to 'buildbot_schedulers' from 10.12.48.111.  Any others as well?

dmoore or ravi: will need firewall holes poked:

destinations: 10.2.70.105:3306
              10.2.70.106:3306
sources: 10.12.48.111
         10.12.48.113
         10.12.48.115
         10.12.48.120
         10.12.48.122

Comment 4

[Dave Miller [:justdave]]

dmoore or ravi: ignore comment 3.  Here's the correct ACL list that needs opened on the firewall:

destinations: 10.2.70.105:3306
              10.2.70.106:3306
sources: 10.12.48.111

destinations: 10.2.70.130:3306
              10.2.70.131:3306
sources: 10.12.48.113
         10.12.48.115
         10.12.48.120
         10.12.48.122

Though I'm sure there'll be more for 105/106 before long.

Comment 5

[Dave Miller [:justdave]]

ACLs granted on the databases, over to netops for the firewall portion.

Comment 6

[Dave Miller [:justdave]]

it's middle of the night west coast US right now, given it was 14 hours ago when this was requested, it's highly likely the people waiting for it have given up and gone home, so I'm not going to page netops unless someone replies saying you're still desperate. :)  They should get to it in the morning otherwise.

Comment 7

[[e-mail] (:bhearsum)]

(In reply to comment #6)
> it's middle of the night west coast US right now, given it was 14 hours ago
> when this was requested, it's highly likely the people waiting for it have
> given up and gone home, so I'm not going to page netops unless someone replies
> saying you're still desperate. :)  They should get to it in the morning
> otherwise.

Most of RelEng is travelling over the weekend, so I'm going to lower this further -- it can wait until Monday.

Comment 8

[Derek Moore [:dmoore]]

Firewall rules opened as requested in Comment #4

Comment 9

[[e-mail] (:bhearsum)]

I think we're missing something here; when I run this I just get a hang after entering the password:
[cltbld@buildbot-master2 master]$ mysql -u buildbot -h tm-b01-master01.mozilla.org -p
Enter password:

Comment 10

[Shyam Mani [:fox2mike]]

Added access for 113 as well.

Comment 11

[[e-mail] (:bhearsum)]

Looks like the firewall needs adjusting, too. I still can't even telnet to 3306.

Comment 12

[Shyam Mani [:fox2mike]]

Derek, can you make sure 113 has access? Thanks!

Comment 13

[[e-mail] (:bhearsum)]

Can you see if there's any expiring of idle connections happening, too? I'm seeing an idle master reconnect to the database every few minutes -- which isn't normal.

Comment 14

[Derek Moore [:dmoore]]

I'm confused. Please take a look at Comment #4 and verify those are the required permissions. From Comment #9, it sounds like you want 10.12.48.113 to have access to 10.2.70.10[56], as well?

Comment 15

[[e-mail] (:bhearsum)]

(In reply to comment #14)
> I'm confused. Please take a look at Comment #4 and verify those are the
> required permissions. From Comment #9, it sounds like you want 10.12.48.113 to
> have access to 10.2.70.10[56], as well?

Correct, as far as I can tell. I think things got confused when Jabba changed the IPs of the masters after this bug was filed. They use to be .120/.122 but changed to .111/.113.

Comment 16

[Derek Moore [:dmoore]]

OK, fixed. Sorry for the mixup.

Comment 17

[[e-mail] (:bhearsum)]

No worries, thanks!

[cltbld@buildbot-master2 master]$ mysql -u buildbot -h tm-b01-master01.mozilla.org -p buildbot_schedulers
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 458187
Server version: 5.0.77-log Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> select * from builds limit 1;
+----+--------+------+------------+-------------+
| id | number | brid | start_time | finish_time |
+----+--------+------+------------+-------------+
|  1 |      0 |   11 | 1274974279 |  1274976052 | 
+----+--------+------+------------+-------------+
1 row in set (0.00 sec)