Closed Bug 1231754 Opened 9 years ago Closed 5 years ago

Update irc.mozilla.org's TLS configuration

Categories

(Infrastructure & Operations Graveyard :: Infrastructure: IRC, task)

Product:
Infrastructure & Operations Graveyard
Component:
Infrastructure: IRC
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INACTIVE

People

(Reporter: April, Assigned: ashish)

References

(
URL
)

Details

irc.mozilla.org currently uses a lot of old protocols, ciphers, and handshake methods.  The following are areas to address:

- Add elliptic curve diffie-hellman (ECDHE) for handshakes
- Add support for AEADs, particularly AES-GCM for bulk encryption
- Disable SSLv3
- Disable RC4
- Update DHE group size from 1024 to 2048-bits, using a custom group

The intermediate configuration is probably what we should aim for, and addresses all of these concerns:

https://wiki.mozilla.org/Security/Server_Side_TLS#Intermediate_compatibility_.28default.29
http://gnutls.org/manual/gnutls.html#Priority-Strings need to figure out how to translate the wiki to that and see if it works probably.
(In reply to Ludovic Hirlimann [:Usul] from comment #1)
> http://gnutls.org/manual/gnutls.html#Priority-Strings need to figure out how
> to translate the wiki to that and see if it works probably.

looks like https://wiki.mozilla.org/Security/Server_Side_TLS#GnuTLS_ciphersuite is probably what needs tweaking.
April how did you check for the features ? (SSLv3 isn't supportedin gnutls at least with the settings currently in place)
Flags: needinfo?(april)
I used cipherscan:

 ❯ ./cipherscan irc.mozilla.org:6697                                                                                                                                      [15:46:14]
.................
Target: irc.mozilla.org:6697

prio  ciphersuite              protocols                    pfs
1     DHE-RSA-AES256-SHA256    TLSv1.2                      DH,1024bits  None
2     DHE-RSA-AES256-SHA       SSLv3,TLSv1,TLSv1.1,TLSv1.2  DH,1024bits  None
3     DHE-RSA-CAMELLIA256-SHA  TLSv1,TLSv1.1,TLSv1.2        DH,1024bits  None
4     AES256-SHA256            TLSv1.2                      None         None
5     AES256-SHA               SSLv3,TLSv1,TLSv1.1,TLSv1.2  None         None
6     CAMELLIA256-SHA          TLSv1,TLSv1.1,TLSv1.2        None         None
7     DHE-RSA-AES128-SHA256    TLSv1.2                      DH,1024bits  None
8     DHE-RSA-AES128-SHA       SSLv3,TLSv1,TLSv1.1,TLSv1.2  DH,1024bits  None
9     DHE-RSA-CAMELLIA128-SHA  TLSv1,TLSv1.1,TLSv1.2        DH,1024bits  None
10    AES128-SHA256            TLSv1.2                      None         None
11    AES128-SHA               SSLv3,TLSv1,TLSv1.1,TLSv1.2  None         None
12    CAMELLIA128-SHA          TLSv1,TLSv1.1,TLSv1.2        None         None
13    RC4-SHA                  SSLv3,TLSv1,TLSv1.1,TLSv1.2  None         None
14    RC4-MD5                  SSLv3,TLSv1,TLSv1.1,TLSv1.2  None         None
15    EDH-RSA-DES-CBC3-SHA     SSLv3,TLSv1,TLSv1.1,TLSv1.2  DH,1024bits  None
16    DES-CBC3-SHA             SSLv3,TLSv1,TLSv1.1,TLSv1.2  None         None

Certificate: trusted, 2048 bits, sha256WithRSAEncryption signature
TLS ticket lifetime hint: None
OCSP stapling: not supported
Cipher ordering: client
Curves ordering: none - fallback: no
TLS Tolerance: yes

But openssl works too:
❯ openssl s_client -ssl3 -connect irc.mozilla.org:6697                                                                                                                   [15:57:14]
CONNECTED(00000003)
...

SSL-Session:
    Protocol  : SSLv3
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: A4CE76380BDAB4E69F10B356ED794E49525A768C977273C68865CAE749BD43BD
    Session-ID-ctx:
    Master-Key: 7EC537676E3E3F198605FE7E67D97AA449E0119AA84ACFD3CBC2E6D127188F137FE04A1155F4309B4B7E6D2204581ABA
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1450130234
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
Flags: needinfo?(april)
current settinsg are :
priority="+VERS-TLS1.2:NORMAL"

Normal is Means all the known to be secure ciphersuites. The ciphers are sorted by security margin, although the 256-bit ciphers are included as a fallback only. The message authenticity security level is of 64 bits or more, and the certificate verification profile is set to GNUTLS_PROFILE_LOW (80-bits).

This priority string implicitly enables ECDHE and DHE. The ECDHE ciphersuites are placed first in the priority order, but due to compatibility issues with the DHE ciphersuites they are placed last in the priority order, after the plain RSA ciphersuites. 


(In reply to April King from comment #0)
> - Add elliptic curve diffie-hellman (ECDHE) for handshakes

Hum so according to docs that's in place

> - Add support for AEADs, particularly AES-GCM for bulk encryption

+AEAD needs to be added.

> - Disable SSLv3
> - Disable RC4
> - Update DHE group size from 1024 to 2048-bits, using a custom group

Can't easily find a way to configure that. cshields , ideas ?
Flags: needinfo?(cshields)
I'll get you all a gnutls string soon enough.  The dh parameters are usually specific to the software; if you don't have a way to use your own custom group, I would recommend removing the DHE ciphers from your options.  Let me know if the IRC software supports custom groups or not, and I'll get you my recommendations.
Looks like you can add dhbits to you <gnutls> tag to set it to a 2048-bit group, as so:

<gnutls dhbits="2048">

Note that the generation of 2048-bit DH groups can take a while on entropy-starved VMs, and may cause the server to take a while to come up.  In that can, you can generate a file like so:

$ openssl dhparams -o dh2048.pem 2048

And use that file with the dhfile parameter inside <gnutls>.

Unfortunately -- blame GnuTLS -- +VERS-TLS1.2:NORMAL is a bad setting.  VERS-TLS1.2 only adds it in if it's not there for some other reason, but it doesn't remove all the other protocols.  And NORMAL can be really dated, depending on which version of gnutls you use.  Here is what +VERS-TLS1.2:NORMAL allows:

$ gnutls-cli -l --priority '+VERS-TLS1.2:NORMAL'
Cipher suites for +VERS-TLS1.2:NORMAL
TLS_ECDHE_ECDSA_AES_128_GCM_SHA256                	0xc0, 0x2b	TLS1.2
TLS_ECDHE_ECDSA_AES_256_GCM_SHA384                	0xc0, 0x2c	TLS1.2
TLS_ECDHE_ECDSA_CAMELLIA_128_GCM_SHA256           	0xc0, 0x86	TLS1.2
TLS_ECDHE_ECDSA_CAMELLIA_256_GCM_SHA384           	0xc0, 0x87	TLS1.2
TLS_ECDHE_ECDSA_AES_128_CBC_SHA1                  	0xc0, 0x09	SSL3.0
TLS_ECDHE_ECDSA_AES_128_CBC_SHA256                	0xc0, 0x23	TLS1.0
TLS_ECDHE_ECDSA_AES_256_CBC_SHA1                  	0xc0, 0x0a	SSL3.0
TLS_ECDHE_ECDSA_AES_256_CBC_SHA384                	0xc0, 0x24	TLS1.0
TLS_ECDHE_ECDSA_CAMELLIA_128_CBC_SHA256           	0xc0, 0x72	TLS1.0
TLS_ECDHE_ECDSA_CAMELLIA_256_CBC_SHA384           	0xc0, 0x73	TLS1.0
TLS_ECDHE_ECDSA_3DES_EDE_CBC_SHA1                 	0xc0, 0x08	SSL3.0
TLS_ECDHE_ECDSA_ARCFOUR_128_SHA1                  	0xc0, 0x07	SSL3.0
TLS_ECDHE_RSA_AES_128_GCM_SHA256                  	0xc0, 0x2f	TLS1.2
TLS_ECDHE_RSA_AES_256_GCM_SHA384                  	0xc0, 0x30	TLS1.2
TLS_ECDHE_RSA_CAMELLIA_128_GCM_SHA256             	0xc0, 0x8a	TLS1.2
TLS_ECDHE_RSA_CAMELLIA_256_GCM_SHA384             	0xc0, 0x8b	TLS1.2
TLS_ECDHE_RSA_AES_128_CBC_SHA1                    	0xc0, 0x13	SSL3.0
TLS_ECDHE_RSA_AES_128_CBC_SHA256                  	0xc0, 0x27	TLS1.0
TLS_ECDHE_RSA_AES_256_CBC_SHA1                    	0xc0, 0x14	SSL3.0
TLS_ECDHE_RSA_AES_256_CBC_SHA384                  	0xc0, 0x28	TLS1.0
TLS_ECDHE_RSA_CAMELLIA_128_CBC_SHA256             	0xc0, 0x76	TLS1.0
TLS_ECDHE_RSA_CAMELLIA_256_CBC_SHA384             	0xc0, 0x77	TLS1.0
TLS_ECDHE_RSA_3DES_EDE_CBC_SHA1                   	0xc0, 0x12	SSL3.0
TLS_ECDHE_RSA_ARCFOUR_128_SHA1                    	0xc0, 0x11	SSL3.0
TLS_RSA_AES_128_GCM_SHA256                        	0x00, 0x9c	TLS1.2
TLS_RSA_AES_256_GCM_SHA384                        	0x00, 0x9d	TLS1.2
TLS_RSA_CAMELLIA_128_GCM_SHA256                   	0xc0, 0x7a	TLS1.2
TLS_RSA_CAMELLIA_256_GCM_SHA384                   	0xc0, 0x7b	TLS1.2
TLS_RSA_AES_128_CBC_SHA1                          	0x00, 0x2f	SSL3.0
TLS_RSA_AES_128_CBC_SHA256                        	0x00, 0x3c	TLS1.0
TLS_RSA_AES_256_CBC_SHA1                          	0x00, 0x35	SSL3.0
TLS_RSA_AES_256_CBC_SHA256                        	0x00, 0x3d	TLS1.0
TLS_RSA_CAMELLIA_128_CBC_SHA1                     	0x00, 0x41	SSL3.0
TLS_RSA_CAMELLIA_128_CBC_SHA256                   	0x00, 0xba	TLS1.0
TLS_RSA_CAMELLIA_256_CBC_SHA1                     	0x00, 0x84	SSL3.0
TLS_RSA_CAMELLIA_256_CBC_SHA256                   	0x00, 0xc0	TLS1.0
TLS_RSA_3DES_EDE_CBC_SHA1                         	0x00, 0x0a	SSL3.0
TLS_RSA_ARCFOUR_128_SHA1                          	0x00, 0x05	SSL3.0
TLS_RSA_ARCFOUR_128_MD5                           	0x00, 0x04	SSL3.0
TLS_DHE_RSA_AES_128_GCM_SHA256                    	0x00, 0x9e	TLS1.2
TLS_DHE_RSA_AES_256_GCM_SHA384                    	0x00, 0x9f	TLS1.2
TLS_DHE_RSA_CAMELLIA_128_GCM_SHA256               	0xc0, 0x7c	TLS1.2
TLS_DHE_RSA_CAMELLIA_256_GCM_SHA384               	0xc0, 0x7d	TLS1.2
TLS_DHE_RSA_AES_128_CBC_SHA1                      	0x00, 0x33	SSL3.0
TLS_DHE_RSA_AES_128_CBC_SHA256                    	0x00, 0x67	TLS1.0
TLS_DHE_RSA_AES_256_CBC_SHA1                      	0x00, 0x39	SSL3.0
TLS_DHE_RSA_AES_256_CBC_SHA256                    	0x00, 0x6b	TLS1.0
TLS_DHE_RSA_CAMELLIA_128_CBC_SHA1                 	0x00, 0x45	SSL3.0
TLS_DHE_RSA_CAMELLIA_128_CBC_SHA256               	0x00, 0xbe	TLS1.0
TLS_DHE_RSA_CAMELLIA_256_CBC_SHA1                 	0x00, 0x88	SSL3.0
TLS_DHE_RSA_CAMELLIA_256_CBC_SHA256               	0x00, 0xc4	TLS1.0
TLS_DHE_RSA_3DES_EDE_CBC_SHA1                     	0x00, 0x16	SSL3.0
TLS_DHE_DSS_AES_128_GCM_SHA256                    	0x00, 0xa2	TLS1.2
TLS_DHE_DSS_AES_256_GCM_SHA384                    	0x00, 0xa3	TLS1.2
TLS_DHE_DSS_CAMELLIA_128_GCM_SHA256               	0xc0, 0x80	TLS1.2
TLS_DHE_DSS_CAMELLIA_256_GCM_SHA384               	0xc0, 0x81	TLS1.2
TLS_DHE_DSS_AES_128_CBC_SHA1                      	0x00, 0x32	SSL3.0
TLS_DHE_DSS_AES_128_CBC_SHA256                    	0x00, 0x40	TLS1.0
TLS_DHE_DSS_AES_256_CBC_SHA1                      	0x00, 0x38	SSL3.0
TLS_DHE_DSS_AES_256_CBC_SHA256                    	0x00, 0x6a	TLS1.0
TLS_DHE_DSS_CAMELLIA_128_CBC_SHA1                 	0x00, 0x44	SSL3.0
TLS_DHE_DSS_CAMELLIA_128_CBC_SHA256               	0x00, 0xbd	TLS1.0
TLS_DHE_DSS_CAMELLIA_256_CBC_SHA1                 	0x00, 0x87	SSL3.0
TLS_DHE_DSS_CAMELLIA_256_CBC_SHA256               	0x00, 0xc3	TLS1.0
TLS_DHE_DSS_3DES_EDE_CBC_SHA1                     	0x00, 0x13	SSL3.0
TLS_DHE_DSS_ARCFOUR_128_SHA1                      	0x00, 0x66	SSL3.0

Protocols: VERS-TLS1.2, VERS-TLS1.1, VERS-TLS1.0, VERS-SSL3.0, VERS-DTLS1.2, VERS-DTLS1.0

This pretty closely tracks with the bad results that I was seeing before.

Here is what I would recommend:

$ gnutls-cli -l --priority 'NORMAL:-VERS-SSL3.0:-CIPHER-ALL:-SHA1:-MD5:+SHA1:-RSA:+RSA:+AES-256-GCM:+AES-128-GCM:+AES-256-CBC:+CAMELLIA-256-GCM:-CAMELLIA-256-CBC:+AES-128-CBC:-DHE-DSS:%SERVER_PRECEDENCE'
Cipher suites for NORMAL:-VERS-SSL3.0:-CIPHER-ALL:-SHA1:-MD5:+SHA1:-RSA:+RSA:+AES-256-GCM:+AES-128-GCM:+AES-256-CBC:+CAMELLIA-256-GCM:-CAMELLIA-256-CBC:+AES-128-CBC:-DHE-DSS:%SERVER_PRECEDENCE
TLS_ECDHE_ECDSA_AES_256_GCM_SHA384                	0xc0, 0x2c	TLS1.2
TLS_ECDHE_ECDSA_AES_128_GCM_SHA256                	0xc0, 0x2b	TLS1.2
TLS_ECDHE_ECDSA_AES_256_CBC_SHA384                	0xc0, 0x24	TLS1.0
TLS_ECDHE_ECDSA_AES_256_CBC_SHA1                  	0xc0, 0x0a	SSL3.0
TLS_ECDHE_ECDSA_CAMELLIA_256_GCM_SHA384           	0xc0, 0x87	TLS1.2
TLS_ECDHE_ECDSA_AES_128_CBC_SHA256                	0xc0, 0x23	TLS1.0
TLS_ECDHE_ECDSA_AES_128_CBC_SHA1                  	0xc0, 0x09	SSL3.0
TLS_ECDHE_RSA_AES_256_GCM_SHA384                  	0xc0, 0x30	TLS1.2
TLS_ECDHE_RSA_AES_128_GCM_SHA256                  	0xc0, 0x2f	TLS1.2
TLS_ECDHE_RSA_AES_256_CBC_SHA384                  	0xc0, 0x28	TLS1.0
TLS_ECDHE_RSA_AES_256_CBC_SHA1                    	0xc0, 0x14	SSL3.0
TLS_ECDHE_RSA_CAMELLIA_256_GCM_SHA384             	0xc0, 0x8b	TLS1.2
TLS_ECDHE_RSA_AES_128_CBC_SHA256                  	0xc0, 0x27	TLS1.0
TLS_ECDHE_RSA_AES_128_CBC_SHA1                    	0xc0, 0x13	SSL3.0
TLS_DHE_RSA_AES_256_GCM_SHA384                    	0x00, 0x9f	TLS1.2
TLS_DHE_RSA_AES_128_GCM_SHA256                    	0x00, 0x9e	TLS1.2
TLS_DHE_RSA_AES_256_CBC_SHA256                    	0x00, 0x6b	TLS1.0
TLS_DHE_RSA_AES_256_CBC_SHA1                      	0x00, 0x39	SSL3.0
TLS_DHE_RSA_CAMELLIA_256_GCM_SHA384               	0xc0, 0x7d	TLS1.2
TLS_DHE_RSA_AES_128_CBC_SHA256                    	0x00, 0x67	TLS1.0
TLS_DHE_RSA_AES_128_CBC_SHA1                      	0x00, 0x33	SSL3.0
TLS_RSA_AES_256_GCM_SHA384                        	0x00, 0x9d	TLS1.2
TLS_RSA_AES_128_GCM_SHA256                        	0x00, 0x9c	TLS1.2
TLS_RSA_AES_256_CBC_SHA256                        	0x00, 0x3d	TLS1.0
TLS_RSA_AES_256_CBC_SHA1                          	0x00, 0x35	SSL3.0
TLS_RSA_CAMELLIA_256_GCM_SHA384                   	0xc0, 0x7b	TLS1.2
TLS_RSA_AES_128_CBC_SHA256                        	0x00, 0x3c	TLS1.0
TLS_RSA_AES_128_CBC_SHA1                          	0x00, 0x2f	SSL3.0

Protocols: VERS-TLS1.2, VERS-TLS1.1, VERS-TLS1.0, VERS-DTLS1.2, VERS-DTLS1.0

And yes, I agree that GnuTLS cipher priorities are incantations not unlike witchcraft.  If that gnutls-cli command runs successfully on the IRC server and outputs what I have above, then I think that's probably the way to go.  Julien, can I get an r+ on that cipher suite choice?
Flags: needinfo?(jvehent)
Okay, I talked to jvehent and he suggests that we add 3DES-CBC for backwards compatibility with really, really old systems.  In that case, the recommendation would be:

gnutls-cli -l --priority 'NORMAL:-VERS-SSL3.0:-CIPHER-ALL:-SHA1:-MD5:+SHA1:-RSA:+RSA:+AES-256-GCM:+AES-128-GCM:+AES-256-CBC:+CAMELLIA-256-GCM:-CAMELLIA-256-CBC:+AES-128-CBC:+3DES-CBC:-DHE-DSS:%SERVER_PRECEDENCE'
Some more updates:

Looks like the IRC servers are running a very old version of GnuTLS (dating to 2009), so we're kind of short on options for modern ciphers.  I'll come up with something that'll work on the existing servers in the interim when I get back on Monday, but we should probably look to moving to either a newer platform or upgrading our crypto libraries.
Flags: needinfo?(jvehent)
fwiw we will be doing OS updates on those nodes by end of the month.
Flags: needinfo?(cshields)
Sounds good.  Just let me know when they're updated, and I will work towards getting us a working TLS configuration with GnuTLS. If the updated OS still has an ancient GnuTLS, I would recommend that we move towards OpenSSL, which should (hopefully) just be a single line change in the InspIRCd configuration.
Upgrades won't get you newer than v2.8.5. CentOS 7 ships with v3.x, which is likely what you're targeting
(In reply to April King [:April] from comment #11)
> Sounds good.  Just let me know when they're updated, and I will work towards
> getting us a working TLS configuration with GnuTLS. If the updated OS still
> has an ancient GnuTLS, I would recommend that we move towards OpenSSL, which
> should (hopefully) just be a single line change in the InspIRCd
> configuration.

We weren't able to get full updates in AWS. We could test InspIRCd with OpenSSL in staging.
I'm not sure if this is related, but attempting to connect via ZNC with a client certificate fails with an OpenSSL error of: error:140940E5:SSL routines:ssl3_read_bytes:ssl handshake failure. Connecting without a client certificate succeeds, as does connecting directly (i.e. not through ZNC) with a client certificate, as does using openssl s_client on the ZNC server to open a connection with a client certificate. I'm following this up with the ZNC maintainers as well, but it seems like updating the TLS configuration on the IRC server's side couldn't hurt. This issue appears to have fallen through the cracks; is there any chance of it being revisited?
(In reply to Eli Young from comment #14)
> I'm not sure if this is related, but attempting to connect via ZNC with a
> client certificate fails with an OpenSSL error of: error:140940E5:SSL
> routines:ssl3_read_bytes:ssl handshake failure. Connecting without a client
> certificate succeeds, as does connecting directly (i.e. not through ZNC)
> with a client certificate, as does using openssl s_client on the ZNC server
> to open a connection with a client certificate. I'm following this up with
> the ZNC maintainers as well, but it seems like updating the TLS
> configuration on the IRC server's side couldn't hurt. This issue appears to
> have fallen through the cracks; is there any chance of it being revisited?

I can confirm I began experiencing the same issue when using client certificates after Debian Stable upgraded from OpenSSL 1.0.1 to 1.10.0 on 6/18/2017.

Downgrading client to TLS1.1 was a workaround to continue using client cert auth, pointing to a cipher suite incompatibility between server side GnuTLS and client OpenSSL version > 1.0.1

My breakage on Debian 8 upgrade:
2017-06-18 12:29:25 upgrade openssl:amd64 1.0.1t-1+deb8u6 1.1.0f-3
:April - Are the recommendations in Comment 8 still current? I will work on using those on CentOS 7.
Flags: needinfo?(april)
Other notes:

* <gnutls dhbits="2048"> is already current
* We probably don't want to switch to OpenSSL. GnuTLS is the recommended SSL suite of choice[1].


[1] Search for "vs." in https://wiki.inspircd.org/Modules/2.0/ssl_gnutls
I don't have any strong feelings about OpenSSL vs. GnuTLS, feel free to use whichever is right for the application.

I'm not entirely certain if my recommendations from Comment 8 are still accurate, but they were based on this:
https://wiki.mozilla.org/Security/Server_Side_TLS#Intermediate_compatibility_.28default.29

Which is what I would recommend for using for irc.mozilla.org currently.

Thanks for working on this!
Flags: needinfo?(april)
Thank you! :)

Migrated to JIRA in CORE-120.

Assignee: infra → ashish
URL: https://wiki.inspircd.org/Modules/2.0... → https://wiki.inspircd.org/Modules/2.0...
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INACTIVE

I think there may be a mixup of issues here. https://jira.mozilla.com/browse/CORE-120 is about something entirely different.

Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.