1530753 - incorrect alert sent for some malformed ClientHello messages

Status: NEW

(NSS :: Libraries, defect, P3)

Description

[Alicja Kario]

When the client sends ClientHello messages that are malformed because they are too long or are missing data (are truncated), NSS rejects them with the illegal_parameter alert instead of the RFC prescribed decode_error.

See RFC 8446, Section 4.1.2:
If negotiating a version of TLS prior to 1.3,
a server MUST check that the message either contains no data after
legacy_compression_methods or that it contains a valid extensions
block with no data following. If not, then it MUST abort the
handshake with a "decode_error" alert.

Section 6:
Peers which receive a message which
cannot be parsed according to the syntax (e.g., have a length
extending beyond the message boundary or contain an out-of-range
length) MUST terminate the connection with a "decode_error" alert.

Section 6.2:
decode_error: A message could not be decoded because some field was
out of the specified range or the length of the message was
incorrect. This alert is used for errors where the message does
not conform to the formal protocol syntax.

Version:
a2b586f90e0a (current default branch)

Reproducer:
git clone https://github.com/tomato42/tlsfuzzer.git
pushd tlsfuzzer

won't be necessary after https://github.com/tomato42/tlsfuzzer/pull/518 is merged

git checkout strict-client-hello-checks
git clone https://github.com/warner/python-ecdsa .python-ecdsa
ln -s .python-ecdsa/ecdsa ecdsa
git clone https://github.com/tomato42/tlslite-ng.git .tlslite-ng
ln -s .tlslite-ng/tlslite tlslite
popd
PYTHONPATH=tlsfuzzer python tlsfuzzer/scripts/test-truncating-of-client-hello.py

in another terminal

openssl req -x509 -newkey rsa -keyout localhost.key -out localhost.crt -nodes -batch -subj /CN=localhost
openssl pkcs12 -export -passout pass: -out localhost.p12 -inkey localhost.key -in localhost.crt
mkdir nssdb
certutil -N -d sql:nssdb --empty-password
pk12util -i localhost.p12 -d sql:nssdb -W ''
./selfserv -d sql:./nssdb -p 4433 -V tls1.0: -H 1 -n localhost -u

NSS server output:
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12249:
SSL received a malformed Handshake record.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12249:
SSL received a malformed Handshake record.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.
selfserv: HDX PR_Read returned error -12260:
SSL received a malformed Client Hello handshake message.

tlsfuzzer output:

sanity ...
OK

small truncate: 5 of bytes truncated ...
OK

extension past extensions ...
OK

hello truncate: 32 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ecea90> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ecead0>) with last message being: <tlslite.messages.Message object at 0x7fc227ed4ed0>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

hello truncate: 39 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227eced90> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ecedd0>) with last message being: <tlslite.messages.Message object at 0x7fc227eac690>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

big pad: 65536 of "0" byte padding ...
OK

small pad: 2 of "255" byte padding ...
OK

small truncate: 4 of bytes truncated ...
OK

small truncate: 1 of bytes truncated ...
OK

large pad: 4096 of "0" byte padding ...
OK

small truncate: 2 of bytes truncated ...
OK

hello truncate: 12 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ece490> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ece4d0>) with last message being: <tlslite.messages.Message object at 0x7fc227eacc10>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

huge pad: 131584 of "0" byte padding ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227eb9fd0> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ec0050>) with last message being: <tlslite.messages.Message object at 0x7fc227eace90>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

hello truncate: 9 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ec5b50> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ec5b90>) with last message being: <tlslite.messages.Message object at 0x7fc227eacfd0>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

small pad: 2 of "0" byte padding ...
OK

hello truncate: 47 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ed40d0> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ed4110>) with last message being: <tlslite.messages.Message object at 0x7fc227c12150>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

full message truncate: 48 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ed43d0> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ed4410>) with last message being: <tlslite.messages.Message object at 0x7fc227eacf90>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

hello truncate: 8 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ec5850> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ec5890>) with last message being: <tlslite.messages.Message object at 0x7fc227c12310>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

small pad: 1 of "0" byte padding ...
OK

hello truncate: 13 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ece790> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ece7d0>) with last message being: <tlslite.messages.Message object at 0x7fc227c12450>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

hello truncate: 10 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ec5e50> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ec5e90>) with last message being: <tlslite.messages.Message object at 0x7fc227c122d0>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

small pad: 3 of "0" byte padding ...
OK

small truncate: 6 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ec5550> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ec5590>) with last message being: <tlslite.messages.Message object at 0x7fc227c12690>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

small pad: 1 of "255" byte padding ...
OK

max pad: 16777167 of "0" byte padding ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ec0310> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ec0350>) with last message being: <tlslite.messages.Message object at 0x7fc227bf2c10>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

small truncate: 3 of bytes truncated ...
OK

small pad: 3 of "255" byte padding ...
OK

hello truncate: 11 of bytes truncated ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7fc227ece190> (child: <tlsfuzzer.expect.ExpectClose object at 0x7fc227ece1d0>) with last message being: <tlslite.messages.Message object at 0x7fc227bf2e50>
Error while processing
Traceback (most recent call last):
File "scripts/test-truncating-of-client-hello.py", line 204, in main
runner.run()
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 223, in run
node.process(self.state, msg)
File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/expect.py", line 1412, in process
raise AssertionError(problem_desc)
AssertionError: Expected alert description "decode_error" does not match received "illegal_parameter"

medium pad: 256 of "0" byte padding ...
OK

sanity ...
OK

Check if ClientHello length checking is correct in server
version: 2

Test end
successful: 17
failed: 13
'full message truncate: 48 of bytes truncated'
'hello truncate: 8 of bytes truncated'
'hello truncate: 9 of bytes truncated'
'hello truncate: 10 of bytes truncated'
'hello truncate: 11 of bytes truncated'
'hello truncate: 12 of bytes truncated'
'hello truncate: 13 of bytes truncated'
'hello truncate: 32 of bytes truncated'
'hello truncate: 39 of bytes truncated'
'hello truncate: 47 of bytes truncated'
'huge pad: 131584 of "0" byte padding'
'max pad: 16777167 of "0" byte padding'
'small truncate: 6 of bytes truncated'