1384302 - Specific Zendesk site not working in Nightly.

Status: RESOLVED INVALID

(Core :: Networking: HTTP, defect)

Description

[frooned]

Attachment: Problem loading page - Nightly.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Build ID: 20170725030209

Steps to reproduce:

I work with several Zendesk sites on a daily basis. I am having trouble loading one:
https://astarsoftware.zendesk.com/


Actual results:

When loading: 
https://astarsoftware.zendesk.com/

I receive an error:
"Secure Connection Failed

An error occurred during a connection to astarsoftware.zendesk.com. SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG

    The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
    Please contact the website owners to inform them of this problem."


Expected results:

I should be able to load the page. It may be useful to note that I am able to access other Zendesk sites. i.e.:

https://junecloud.zendesk.com/agent/dashboard

Comment 1

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

Can you get a packet trace of the TLS handshake (use e.g. wireshark) and attach it? Thanks.

Comment 3

[Lars Viklund]

Attachment: astarsoftware-failed.pcapng

Server response seems to be plaintext 400.

Comment 4

[Lars Viklund]

Attachment: astarsoftware-working-55_0b13.pcapng

Captured from Firefox Developer Edition 55.0b13, loads page properly.

Comment 5

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

Thanks. Interestingly, the client hellos appear to be functionally identical in both cases. One difference that might be causing this is that the failing attempt appears to send the client hello in two tcp packets - can you confirm that this is always happening? Or can you get a packet trace that doesn't have the client hello in two packets?

Comment 6

[Lars Viklund]

I have not been able to make an connection attempt to the failing site without it resulting in two packets in Nightly.

Noteworthy is that most other ClientHello:s are similarly split but working on my system, including twitter.com, incoming.telemetry.mozilla.org, and also the working junecloud site mentioned in the description.

There's the occasional unfragmented one to places like incoming.telemetry.mozilla.org, but the norm seems to be two packets.

Comment 7

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

Maybe this is also tcp fast-open-relateD?

Comment 8

[Dragana Damjanovic [:dragana]]

Can you try this site with the latest nightly? 

The fact that we are sending 2 packets instead of one was an experiment that I did while chasing a bug in TFO. In the latest nightly that should not be the case any more. All ClientHello that are smaller than ~1400bytes(and they are smaller) should be sent as a single packet.

In latest nightly TFO is turned off.

Can you try the latest nightly first with TFO turned off ( the pref is "network.tcp.tcp_fastopen_enable"). and after that with turn on TFO. please use shift reload (to force a new TCP connection).

Thank you!

Comment 9

[Lars Viklund]

In 56.0a1 (2017-08-01) (64-bit) the astarsoftware site loads properly for both values of the pref.
The intellij site from my duplicate bug also loads.

Comment 10

[Dragana Damjanovic [:dragana]]

Thank you.

So we have a problem if ClientHello is split up. TFO syn packet can accept max 1440Bytes of data which should be enough for ClientHello, but we cannot guarantee 1440bytes will be really sent (The internet can be really strange...)

We will probably need some fallback mechanism.

Comment 11

[Patrick McManus [:mcmanus]]

So this isn't really a TFO bug if you look at the capture (tho it was triggered by the TFO troubleshooting, I'm sure)

The SYN has the TFO flag but no data (presumably because there is no cookie) and the Syn-Ack doesn't set a cookie. So its a non TFO server. but we still split the chello into 10 and 507 packets.

So the server has a pretty severe bug that it is treating a TCP byte stream as a message. That should be reported to nginx/zendesk and I guess suggest this bug closed as INVALID (server bug) as long as the short-packet-splitting has been reverted elsewhere. (it has, right?)

dragana, I don't think this bit of webcompat news impacts on our TFO plans does it? The SYN should hold the chello just as well (or not) as the first packet after handshake..

Comment 12

[Dragana Damjanovic [:dragana]]

(In reply to Patrick McManus [:mcmanus] from comment #11)
> So this isn't really a TFO bug if you look at the capture (tho it was
> triggered by the TFO troubleshooting, I'm sure)
> 
> The SYN has the TFO flag but no data (presumably because there is no cookie)
> and the Syn-Ack doesn't set a cookie. So its a non TFO server. but we still
> split the chello into 10 and 507 packets.
> 
> So the server has a pretty severe bug that it is treating a TCP byte stream
> as a message. That should be reported to nginx/zendesk and I guess suggest
> this bug closed as INVALID (server bug) as long as the
> short-packet-splitting has been reverted elsewhere. (it has, right?)
> 

This has been reverted as of Nightly 20170801.

> dragana, I don't think this bit of webcompat news impacts on our TFO plans
> does it? The SYN should hold the chello just as well (or not) as the first
> packet after handshake..

It should, I think that 1400bytes tcp packet size is very usually. I do not believe that there is a os with a lower one...

We can closed this bug.