781500 - https://*.mozilla.org request give 'connection was interrupted' errors

Status: RESOLVED INCOMPLETE

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P4)

Description

[Mark Finkle (:mfinkle) (use needinfo?)]

When I try to visit some Mozilla properties (tbpl.mozilla.org, hacks.mozilla.org and planet.mozilla.org) I can't load the page and Firefox gives this error:

Connection Interrupted
The connection to the server was reset while the page was loading.


If I try using Google Chrome, I get the same interruption error, but sometimes I get:

SSL connection error
Unable to make a secure connection to the server.

-----
This only happens if I am using my Comcast ISP. I can load the properties fine if I use my Verizon 4g connection on my phones and tablet.

Also, loading developer.mozilla.org, addons.mozilla.org and bugzilla.mozilla.org all work fine using Comcast ISP.

Comment 1

[Jake Maul [:jakem]]

Comcast is an interesting beast in a couple ways... IPv6 and DNSSEC support. Both *should* be irrelevant, but historically have not been entirely so.

One thing that is definitely relevant... planet.mozilla.org does not currently support SSL... we don't have a valid (non-expired) certificate for it, so it does not listen on port 443 at all.

Would you be able to verify if you're connecting over IPv4 or IPv6 when on Comcast? Some of our properties are v6-aware, and I'm suspecting that may play a role here... although offhand I can't identify what that role is.

Here's some random data, in the hopes that some of it may be relevant to future commenters:

developer.mozilla.org - single hosted in SCL3, uses Cedexis, no IPv6
addons.mozilla.org - single hosted in PHX1, no GLB, has IPv6
bugzilla.mozilla.org - active/passive hosted in PHX1, uses Dynect, has IPv6

tbpl.mozilla.org - generic cluster in PHX1, has IPv6, TLS SNI for cert selection
hacks.mozilla.org - generic cluster in PHX1, has IPv6, TLS SNI for cert selection
planet.mozilla.org - static cluster in PHX1, has IPv6 but no SSL support

Comment 2

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Jake Maul [:jakem] from comment #1)

> Would you be able to verify if you're connecting over IPv4 or IPv6 when on
> Comcast? Some of our properties are v6-aware, and I'm suspecting that may
> play a role here... although offhand I can't identify what that role is.

IPv4 only. I had IPv6 set to ignore

> Here's some random data, in the hopes that some of it may be relevant to
> future commenters:
> 
> developer.mozilla.org - single hosted in SCL3, uses Cedexis, no IPv6
> addons.mozilla.org - single hosted in PHX1, no GLB, has IPv6
> bugzilla.mozilla.org - active/passive hosted in PHX1, uses Dynect, has IPv6

These all connect fine using https://

> tbpl.mozilla.org - generic cluster in PHX1, has IPv6, TLS SNI for cert
> selection

Can't connect using http:// or https://

> hacks.mozilla.org - generic cluster in PHX1, has IPv6, TLS SNI for cert
> selection

Connects using http:// but no stylesheest are loaded. I assume those might be https:// ?

> planet.mozilla.org - static cluster in PHX1, has IPv6 but no SSL support

Connects fine as http://

Comment 3

[Jake Maul [:jakem]]

We have a couple other folks here on Comcast (in IT, webops), without problems.

Have you tried any other machines on that Comcast connection? (perhaps wifi on your phone/tablet) We are starting to suspect this is a local issue on your end... possibly some bit of malware or something.

Also, could you check to see what IPs all of these sites resolve to? That might help narrow things down too.

Comment 4

[Mark Finkle (:mfinkle) (use needinfo?)]

I have tried connecting via wireless and wired connections on Comcast. Both fail. This includes using it from desktops, laptops phones and tablets.

I have been turning my cell phone into a hotspot so I can actually see tbpl.mozilla.org (I need access to do my job).

Comment 5

[Brian Hourigan [:digi]]

Hi Mark,

Sorry to hear you're having such a bizarre problem. It could be a number of things so please bear with us so we can rule out individual components.

- You mention that Chrome displays a generic SSL error, does it happen to have any additional details? A quick google search of that string seems to suggest that some more details may be available in some cases.

- During one of these outages, could you tether your computer to your Verizon 4g device? If you are unable to do so and it's a laptop, could you try replicating these problems on another internet connections such as a cafe, etc?

- How long has this been happening? If it just stated, have you made any recent software or hardware updates? New router, cable modem, etc?

- When these problems occur, could you try opening cmd.exe/terminal and pinging the web property? If you're running windows try a "ipconfig /flushdns" first, or if it's OSX I think the command is "dscacheutil -flushcache". If the ping fails after flushing cache, run it for 5-10 more times and see if the behavior changes.

Thanks in advance.

Comment 6

[Jake Maul [:jakem]]

Removing "in Firefox" since it's reported to have issues in Chrome also.

Comment 7

[Jake Maul [:jakem]]

Renormalizing priority levels... P4 is "normal" now.

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Brian Hourigan [:digi] from comment #5)

> - You mention that Chrome displays a generic SSL error, does it happen to
> have any additional details? A quick google search of that string seems to
> suggest that some more details may be available in some cases.

In Chrome, I sometimes get:
Error 101 (net::ERR_CONNECTION_RESET): The connection was reset.

But other times I get:
SSL connection error
Unable to make a secure connection to the server. This may be a problem with the server, or it may be requiring a client authentication certificate that you don't have.
Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.

> - During one of these outages, could you tether your computer to your
> Verizon 4g device? If you are unable to do so and it's a laptop, could you
> try replicating these problems on another internet connections such as a
> cafe, etc?

Tethering to my verizon 4g device does work. I wouldn't call it an outage though. The failure to connect happens all the time on my wired Comcast connection.

> - How long has this been happening? If it just stated, have you made any
> recent software or hardware updates? New router, cable modem, etc?

I recently updated to a new DOCIS 3 cable modem.

> - When these problems occur, could you try opening cmd.exe/terminal and
> pinging the web property? If you're running windows try a "ipconfig
> /flushdns" first, or if it's OSX I think the command is "dscacheutil
> -flushcache". If the ping fails after flushing cache, run it for 5-10 more
> times and see if the behavior changes.

mfinkle@ubuntu-desktop:~/source/mozilla-android/mozilla$ ping tbpl.mozilla.org
PING generic.zlb.phx.mozilla.net (63.245.217.86) 56(84) bytes of data.
64 bytes from generic.zlb.phx.mozilla.net (63.245.217.86): icmp_req=1 ttl=48 time=87.7 ms
64 bytes from generic.zlb.phx.mozilla.net (63.245.217.86): icmp_req=2 ttl=48 time=88.5 ms
64 bytes from generic.zlb.phx.mozilla.net (63.245.217.86): icmp_req=3 ttl=48 time=88.3 ms

Comment 9

[Corey Shields [:cshields]]

Not sure what we can do here.  You've shown that the sites are up when you have this problem, just not through your home cable connection.

Have you contacted comcast on this?

I guess the next step would be needing exact times and status on all of the sites you have an issue with so we can try and find a cause or common ground.