1136095 - RX_MALFORMED_SERVER_KEY_EXCH with RSA_EXPORT_WITH_DES_CBC_SHA or *_RC4_56_SHA, when using 2048 server key and TLS 1.2

Status: RESOLVED FIXED

(NSS :: Libraries, defect)

Description

[Kai Engert [:KaiE:]]

After landing bug 1129573 our automated tests report a new failure.

The following combination of parameters results in
SSL_ERROR_RX_MALFORMED_SERVER_KEY_EXCH reported by the NSS client:

- RSA server certificate with key size 2048
- cipher RSA_EXPORT_WITH_DES_CBC_SHA (-c l) or
  cipher RSA_EXPORT_WITH_RC4_56_SHA (-c m)
- TLS 1.2

Changing ANY (at least one) of the above parameters works:

- Using server key size 1024 works (using certificate data from an older test run)

- Using TLS 1.1 or older works

- All other ciphers work

Comment 1

[Kai Engert [:KaiE:]]

Either the mentioned combination of parameters should work, or, if the combination is invalid, NSS should refuse to use it, instead of sending a broken server key exchange message.

I think the patch that triggered these failures shouldn't be seen as a regression, and I'd prefer to keep the patch.

Rather, we should address these failures.

I've temporarily disabled the affected tests:
https://hg.mozilla.org/projects/nss/rev/ae0710a52477

Comment 2

[Kai Engert [:KaiE:]]

Use the correct syntax to disable the tests (and avoid failures caused by empty lines):
https://hg.mozilla.org/projects/nss/rev/807b7ed8e175

Comment 3

[Masatoshi Kimura [:emk]]

The symbolic names of those ciphers are
>    TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA,	/* l */
>    TLS_RSA_EXPORT1024_WITH_RC4_56_SHA,	        /* m */
[1].
The code points are 
> #define TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA     0x0062
> #define TLS_RSA_EXPORT1024_WITH_RC4_56_SHA      0x0064
[2].

* Those cipher suites intentionally restrict the key size to 1024-bit because of the US export restriction.
* Although those cipher suites have never been officially registered to IANA registry [3], TLS 1.1+ does not allow export cipher suites anymore [4].

So I don't think it is a valid combination. (I don't know why EXPORT1024 + TLS 1.1 "works".)

[1] https://mxr.mozilla.org/nss/source/cmd/selfserv/selfserv.c?rev=ab3a00d03c39#119
[2] https://mxr.mozilla.org/nss/source/lib/ssl/sslproto.h?rev=970075886b70#187
[3] https://www.iana.org/assignments/tls-parameters/tls-parameters.txt "0x00,0x60-66 Reserved to avoid conflicts with widely deployed implementations         [Pasi_Eronen]"
[4] https://tools.ietf.org/html/rfc4346#appendix-A.5

Comment 4

[Kai Engert [:KaiE:]]

We also have bustage on the memleak build/test machine, I've removed ciphers m and l until we can get this bug fixed:
https://hg.mozilla.org/projects/nss/rev/66b2e2f83a84

Comment 5

[Tim Taubert [:ttaubert] (inactive)]

"Fixed" by removing export ciphers in bug 1252849.