Closed Bug 1572088 Opened 6 years ago Closed 6 years ago

Can't connect to imap.gmail.com on ipv6 from PAR2

Categories

(Infrastructure & Operations Graveyard :: NetOps, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: julienw, Assigned: van)

Details

$ traceroute6 imap.gmail.com
traceroute to imap.gmail.com (2a00:1450:400c:c0c::6d), 30 hops max, 80 byte packets
 1  2a00:8c40:235:224::1 (2a00:8c40:235:224::1)  5.127 ms  5.030 ms  4.988 ms
 2  2a00:8c40:235:4091::1 (2a00:8c40:235:4091::1)  4.888 ms  4.889 ms  4.844 ms
 3  2a00:8c40:235:ffff::15 (2a00:8c40:235:ffff::15)  4.805 ms  4.731 ms  4.660 mso
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *

This works fine in ipv4.

Also some other ipv6 destinations work fine, for example here it is for my own server:

$ traceroute6 petitevallee.everlong.org
traceroute to petitevallee.everlong.org (2001:4b98:dc0:47:216:3eff:fe26:27ec), 30 hops max, 80 byte packets
 1  2a00:8c40:235:224::1 (2a00:8c40:235:224::1)  5.760 ms  5.660 ms  5.601 ms
 2  2a00:8c40:235:4091::1 (2a00:8c40:235:4091::1)  5.499 ms  5.371 ms  5.392 ms
 3  2a00:8c40:235:ffff::15 (2a00:8c40:235:ffff::15)  5.348 ms  5.298 ms  5.243 ms
 4  2001:978:2:19::4d:1 (2001:978:2:19::4d:1)  6.370 ms  7.515 ms  7.451 ms
 5  be3517.ccr41.par01.atlas.cogentco.com (2001:550:0:1000::8275:3129)  6.195 ms  6.141 ms *
 6  be3594.rcr21.b019498-0.par01.atlas.cogentco.com (2001:550:0:1000::9a36:3c7e)  7.170 ms  6.224 ms be3593.rcr21.b019498-0.par01.atlas.cogentco.com (2001:550:0:1000::9a36:3c7a)  6.117 ms
 7  * * *
 8  peerlink-vlan214.sc-1.csd4-ip6.paris.gandi.net (2001:4b98:dc2:1f::1:1202)  4.740 ms  4.663 ms  4.641 ms
 9  2001:4b98:dc2:1f::c302:2 (2001:4b98:dc2:1f::c302:2)  7.748 ms  7.442 ms  7.323 ms
10  2001:4b99:1:254::15 (2001:4b99:1:254::15)  5.920 ms  5.896 ms  5.852 ms
11  xvm6-fe26-27ec.ghst.net (2001:4b98:dc0:47:216:3eff:fe26:27ec)  7.066 ms  5.739 ms  5.612 ms

Hey Van, I think you're best suited to look at this :)

Flags: needinfo?(vle)

this appears to be an issue upstream, i've emailed our ISP for support.

Flags: needinfo?(vle)

This is also affecting the guest network :
[ludovic@saraan ~]$ traceroute6 google.com
traceroute to google.com (2a00:1450:4007:808::200e), 30 hops max, 80 byte packets
1 2a00:8c40:235:232::1 (2a00:8c40:235:232::1) 5.217 ms !N 5.148 ms !N 5.087 ms !N

the issue was reported to our ISP yesterday. they are not giving us routes to certain sites and ive asked again today. will update again once i hear from our ISP.

I will follow up with my team during US business hours.

Dear customer,

I am afraid that even my escalation contact will just state the same. I guess we could make some progress if you had the trace or show bgp x.x.x.x output, if you had connection to Google through Cogent before. So we could see who stopped sending it to us

But to put it briefly, we hold the routes that we customers and peers advertise to us and we cannot force them to advertise said resource. Also, I must be noted that this prefix is from Google who, aside of not having any peer with us, hold the ultimate right to decide where to propagate this prefixes.

Regards,

Javier Mencía | Customer Support Engineer
Customer Support Europe / European Network Operation Center
Cogent Communications | Madrid, Spain | www.cogentco.com
Hotlines in Europe: +34 91 102 2501, option 2 | +49 69 2998 9696, option 2 | +33 1 49 03 1818
Alternative Hotline: 0800 877 8005 (in the UK) | 06800 18854 (in Hungary)
Email: eu-support@cogentco.com

Assignee: network-operations → vle

update - johnb has requested more info from our ISP and we will pursue escalation.

spoke to :julienw before i left, looks like cogent is filtering google and they're steadfast about it. he understand this issue will be resolved when we get our 2nd ISP in the office.

:julienw, are you guys affected work-wise? and how many members in the office are impacted?

Flags: needinfo?(felash)

I was mostly affected in thunderbird. Now I'm not affected much because I disabled ipv6 there. Curiously I'm not affected in Firefox despite that I didn't disable ipv6 there...

And I just realized that I did disable ipv6 in the past on the WiFi connection itself, so maybe that's the reason.

Anyway nowadays I'm not impacted but that's a pretty drastic workaround I have :-)

Flags: needinfo?(felash)

Hi Julien.
An update on this. Each office the NetOps team builds should have two working WAN carrier circuits. Our second WAN carrier will be staged soon. This will bring back the IPV6 functionality you previously had. we will communicate of any changes that might occur.
the only caveat I would state upfront. Should our second WAN carrier be affected by an outage, it will impact this Google IPV6 service for the duration of such.

Thanks

Status: NEW → ASSIGNED

Thanks for the update.
I'm still a bit concerned that we contracted with a WAN carrier that doesn't have this important functionality, especially that we're using Google for some of our company services.

we are totally with you. we have never ran into this situation before with a Global carrier. In our investigation of this issue, we have found that Cogent has engaged in this type of disputes more than any other Global WAN carrier. we are exploring our legal options. our second wan carrier will be arriving soon and this will be a non issue.
we will keep you posted

Got it, thanks!

We brought up the Zayo link this evening. Can you re-test and see if this works now?

no dice:

$ mtr -wb6 google.com
Start: 2019-10-18T11:21:52+0200
HOST: jcristau-z4            Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 2a00:8c40:235:224::1    0.0%    10    1.0   1.2   1.0   1.3   0.1
  2.|-- 2a00:8c40:235:4091::1   0.0%    10    0.4   0.4   0.4   0.4   0.0
  3.|-- 2a00:8c40:235:ffff::15  0.0%    10    6.3  16.5   0.8  32.8  10.7
  4.|-- 2a00:8c40:235:ffff::12  0.0%    10    0.6   0.7   0.6   0.8   0.1
  5.|-- 2001:1b48:2:103::3a     0.0%    10    4.3   3.1   1.8   4.3   1.0
  6.|-- 2001:1b48:2:3::ed       0.0%    10    7.0   2.1   1.5   7.0   1.7
  7.|-- 2001:1b48:2:3::16:2     0.0%    10    1.5   4.2   1.4  28.0   8.4
  8.|-- ???                    100.0    10    0.0   0.0   0.0   0.0   0.0

ok, your route is going out through Zayo... and not getting beyond that.

Looks like our IPv6 space isn't getting advertised beyond Zayo. I will go poke them about why they're not propagating it.

Any news from the carrier?

Flags: needinfo?(justdave)

This should all be working as of a week or two ago, can you test again?

Flags: needinfo?(justdave)
Flags: needinfo?(jcristau)
Flags: needinfo?(felash)

Success!

$ mtr -wb6 google.com
Start: 2019-11-12T19:57:13+0100
HOST: jcristau-z4                                          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 2a00:8c40:235:224::1                                  0.0%    10    1.1   1.2   1.0   1.4   0.1
  2.|-- 2a00:8c40:235:4091::1                                 0.0%    10    0.4   0.4   0.4   0.4   0.0
  3.|-- 2a00:8c40:235:ffff::15                                0.0%    10    0.5   0.7   0.5   0.8   0.1
  4.|-- 2a00:8c40:235:ffff::12                                0.0%    10    0.8   0.7   0.7   0.8   0.0
  5.|-- 2001:1b48:2:103::3a                                   0.0%    10    1.8  10.2   1.3  82.1  25.3
  6.|-- 2001:1b48:2:3::ed                                     0.0%    10    0.8   1.1   0.8   2.1   0.5
  7.|-- 2001:1b48:2:3::16:2                                   0.0%    10    0.8   7.3   0.8  39.9  13.0
  8.|-- 2001:1b48:2:1::4:2                                    0.0%    10    2.5   2.7   1.0   9.4   2.7
  9.|-- 2001:4860:0:1016::6                                   0.0%    10    3.3   1.8   1.1   3.3   0.9
 10.|-- 2001:4860::c:4000:ce41                                0.0%    10   27.0  27.2  27.0  27.3   0.1
 11.|-- 2001:4860::c:4001:66d2                                0.0%    10   27.3  27.4  27.2  27.9   0.2
 12.|-- 2001:4860::c:4001:9920                                0.0%    10   31.2  28.2  27.6  31.2   1.1
 13.|-- 2001:4860::c:4001:ec1                                 0.0%    10   27.4  27.4  27.2  27.6   0.1
 14.|-- ???                                                  100.0    10    0.0   0.0   0.0   0.0   0.0
 15.|-- 2001:4860:0:1::1c51                                   0.0%    10   26.7  26.7  26.6  26.7   0.0
 16.|-- ham02s15-in-x0e.1e100.net (2a00:1450:4005:803::200e)  0.0%    10   26.6  26.7  26.6  26.8   0.1
Flags: needinfo?(jcristau)

Awesome, thanks!

Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Flags: needinfo?(felash)
Resolution: --- → FIXED

seems to work for me too, thanks!

Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.