1442470 - Discussion about certs for fxa logs sent to rabbitmq using ssl connection on syslog-proxy1 in mdc1. (migrating from syslog-proxy1 in scl3)

Status: RESOLVED FIXED

(Firefox :: Firefox Accounts, enhancement)

Description

[Alicia Smith [:phrozyn] (use NEEDINFO)]

Hello!

I'm opening this bug as we are in the process of migrating our syslog-proxy1.dmz.scl3.mozilla.com event sources to a new host in mdc1 (syslog-proxy1.dmz.mdc1.mozilla.com (1439994)

The connection from fxa is over ssl to the current syslog-proxy1 host, and I'm not sure if you guys had to configure a client cert on your end to facilitate this connectivity, or if you are simply negotiating an ssl connection.

This bug is to start discovery around this and ensure that we  determine all requirements before starting work on this.

If this host is configured to send with a mozilla CA signed certificate, then we will need to  provision a new certificate (public signed digicert) 
If not, then no other work will be needed around that other than to point the fxa events to the new host once it's fully configured. 

Please let me know the configuration details on your side.

Thank you!

Comment 1

[Chris Kolosiwsky [:ckolos] (ckolos has left the building)]

302 -> :jbuck

Comment 2

[John Morrison [:jrgm]]

Sorry. What's the route on that? Or better, do you have an active connnection, and if so what is the IP:port of that?

Comment 3

[Alicia Smith [:phrozyn] (use NEEDINFO)]

54.187.93.80:39013 appears to be the the IP:port

You are connecting to syslog-proxy1.dmz.scl3.mozilla.com port 5671 to send these events.

Comment 4

[Jon Buckley [:jbuck]]

Ahh, this is the fxa-heka box i-9b4a0583 .

:whd, do you know how this box is configured to connect to syslog?

> The connection from fxa is over ssl to the current syslog-proxy1 host, and I'm not sure if you guys had to configure a client cert on your end to facilitate this connectivity, or if you are simply negotiating an ssl connection.

Comment 5

[Wesley Dawson [:whd]]

We're connecting over AMQPS to syslog-proxy1.scl3.mozilla.com:5671, without a  Mozilla-signed cert, so I agree with the assessment that the only work needed fxa-side will be to point the fxa events to the new host once it's fully configured.

Comment 6

[Alicia Smith [:phrozyn] (use NEEDINFO)]

When you have a chance can we point these events to syslog-proxy1.mdc1.mozilla.com and the port is 5671 (tls)

Comment 7

[Wesley Dawson [:whd]]

I can't resolve syslog-proxy1.mdc1.mozilla.com from AWS. This might have to wait until the DNS cutover that I believe is happening soon.

Comment 8

[Alicia Smith [:phrozyn] (use NEEDINFO)]

Agreed, I did ping johnb earlier about it - once he's told me it's good, I'll check and then update you.

Comment 9

[Alicia Smith [:phrozyn] (use NEEDINFO)]

These flows are open, and I see a connection to the fxa queue, let me know when you want to test switching from scl3 to mdc1 on syslog-proxy1

Comment 10

[Wesley Dawson [:whd]]

When testing I observed the following: 

Error making runner for MozDefOutput: Initialization failed for MozDefOutput': x509: certificate is valid for syslog-proxy1.dmz.mdc1.mozilla.com, not syslog-proxy1.mdc1.mozilla.com

syslog-proxy1.dmz.mdc1.mozilla.com does not resolve.

Comment 11

[Brian Hourigan [:digi]]

We need a SAN certificate to cover the public and private endpoints of this service, which has been requested in 1456879.

Comment 12

[Brian Hourigan [:digi]]

(In reply to Brian Hourigan [:digi] from comment #11)
> 1456879.

Whoops! Bug 1456897.

Comment 13

[Brian Hourigan [:digi]]

(In reply to Brian Hourigan [:digi] from comment #11)
> We need a SAN certificate to cover the public and private endpoints of this
> service, which has been requested in 1456897.

Updated certificate in 66ce1448f3fe4c4027fa1025a07f37394c1186b0, waiting for deploy. Will follow up with verification.

Comment 14

[Brian Hourigan [:digi]]

Having difficulty initiating a SSL connection to rabbitmq with openssl s_client. :whd, could you please retest?

Comment 15

[Wesley Dawson [:whd]]

Upon retesting it appears to have worked, so I left it pointed at the the new hostname and put in https://github.com/mozilla-services/puppet-config/pull/2727 to reflect this change.