770475 - please allow mysql access from metrics servers

Status: VERIFIED FIXED

(Data & BI Services Team :: DB: MySQL, task)

Description

[Mark Reid [:mreid]]

As per bug 770295, metrics log processing code needs mysql access to the addons_mozilla_org database on the tp-amo01-ro-zeus.phx.mozilla.com server, connecting from etl1.metrics.scl3.mozilla.com and etl2.metrics.scl3.mozilla.com

Server connectivity seems to be working now, but mysql is not allowing connections with the message:
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0

Comment 1

[Sheeri Cabral [:sheeri]]

Hrm, I can confirm that the port is accessible:

[[e-mail]3 ~]$ nc -v tp-amo01-ro-zeus.phx.mozilla.com 3306
Connection to tp-amo01-ro-zeus.phx.mozilla.com 3306 port [tcp/mysql] succeeded!

But also I can confirm this problem:

[[e-mail]3 ~]$ mysql -h tp-amo01-ro-zeus.phx.mozilla.com 3306
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0

But there is a user allowed from that server, username metrics, so I'm not sure what's going on. I'll investigate.

Comment 2

[Sheeri Cabral [:sheeri]]

Just tried adding metrics@10.8.70.21_ (metrics@10.8.70.20_ was already defined).

Comment 3

[Sheeri Cabral [:sheeri]]

Adding some network guys.....I'm sure this is a MySQL error, but can you let me know what the pass-through IP is? 

There are users defined for 10.8.70.20_ and 10.8.70.21_. I even tried making a user at %, which is all hosts, and still got this response:

[[e-mail]3 ~]$ telnet  tp-amo01-ro-zeus.phx.mozilla.com 3306
Trying 10.8.70.19...
Connected to tp-amo01-ro-zeus.phx.mozilla.com.
Escape character is '^]'.
Connection closed by foreign host.

So I'm really not sure what's going on, could it be something because an scl3 server is trying to reach a phx VIP and box?

Comment 4

[Mark Reid [:mreid]]

The same problem is also happening when connecting to tp-amo01-rw-zeus.phx.mozilla.com - please add this one too when it gets sorted out.

Comment 5

[casey ransom [:casey]]

it wasn't a network problem per se, but the load balancer also specifies who gets to connect to that amo-rw vip. I added in etl1/etl2:
nc -v[[e-mail]3 ~]$ nc -v tp-amo01-ro-zeus.phx.mozilla.com 3306
Connection to tp-amo01-ro-zeus.phx.mozilla.com 3306 port [tcp/mysql] succeeded!
8
5.1.52-log
^?)BRT04Py?0"Xvi-BBeWeu