1142918 - Localization server no longer answers to requests from transvision.mozfr.org

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Localization Server, task)

Description

[Pascal Chevrel:pascalc]

Transvision uses web services created by the l10n drivers team, in particular requests such as:

https://l10n.mozilla-community.org/~flod/mozilla-l10n-query/?bugzilla=products

Since [Thu Mar 12 22:40:00 2015] all requests from our app to get this data fail with a timeout with this error:
[Thu Mar 12 22:40:00 2015] [error] [client 180.76.6.37] PHP Warning:  file_get_contents(https://l10n.mozilla-community.org/~flod/mozilla-l10n-query/?bugzilla=products): failed to open stream: Connection timed out in /home/pascalc/transvision/vendor/mozillal10n/bugzilla/src/Bugzilla/Bugzilla.php on line 77

If I ssh to our mozfr server, I can indeed see that wget takes 4 minutes  with https://l10n.mozilla-community.org/ but works instantly with any other website, it seems that it first times out using IPV6 and then when it falls back to IPV4, it gets the data instantly. Unfortunately, it won't work with out app because it won't wait 4mn for the request.

time wget https://l10n.mozilla-community.org
--2015-03-13 10:08:29--  https://l10n.mozilla-community.org/
Résolution de l10n.mozilla-community.org (l10n.mozilla-community.org)... 2001:41d0:52:400::375, 37.187.48.136
Connexion vers l10n.mozilla-community.org (l10n.mozilla-community.org)|2001:41d0:52:400::375|:443...échec: Connexion terminée par expiration du délai d'attente.
Connexion vers l10n.mozilla-community.org (l10n.mozilla-community.org)|37.187.48.136|:443...connecté.
requête HTTP transmise, en attente de la réponse...200 OK
Longueur: 1689 (1,6K) [text/html]
Sauvegarde en : «index.html.1»

100%[================================================================================================================================>] 1 689       --.-K/s   ds 0s      

2015-03-13 10:11:39 (23,4 MB/s) - «index.html.1» sauvegardé [1689/1689]


real	3m9.694s
user	0m0.348s
sys	0m0.024s



Here is a traceroute from the mozfr server:
traceroute l10n.mozilla-community.org
traceroute to l10n.mozilla-community.org (37.187.48.136), 30 hops max, 60 byte packets
 1  vss-3b-6k.fr.eu (188.165.242.252)  47.502 ms * *
 2  10.21.50.48 (10.21.50.48)  1.652 ms  1.651 ms  1.642 ms
 3  l10n.mozilla-community.org (37.187.48.136)  0.461 ms  0.467 ms  0.459 ms

The mozfr server hasn't had any update this week and Ludovic also sees the bug from his own machine, so it seems that there is something funky with incoming network connections in IPV6 on the localization server.

Comment 1

[Pascal Chevrel:pascalc]

Ludovic asked me to check if IPV6 works from the localization server, apparently it doesn't

traceroute6 www.google.com
traceroute to www.google.com (2a00:1450:400c:c05::6a) from 2001:41d0:52:400::375, 30 hops max, 24 byte packets
 1  vps28311.ovh.net (2001:41d0:52:400::375)  1434.93 ms !H  2999.54 ms !H  3000.02 ms !H

Comment 2

[Pascal Chevrel:pascalc]

also note that logging via ssh to the server, the welcoming message says at the end:
*** System restart required ***

Comment 3

[Pascal Chevrel:pascalc]

Reed, AFAIK you are the only sysadmin for the server, can you give me an estimate as to when you will look at this issue please so as that I can plan when to remove the temporary hack we put on Transvision to workaround this bug? Thanks

Comment 4

[Reed Loden [:reed]]

(In reply to Pascal Chevrel:pascalc from comment #3)
> Reed, AFAIK you are the only sysadmin for the server, can you give me an
> estimate as to when you will look at this issue please so as that I can plan
> when to remove the temporary hack we put on Transvision to workaround this
> bug? Thanks

I haven't been reading bugmail, so only saw the needinfo request. I can try to look this weekend sometime. If it's an IPv6 issue, probably need to involve OVH.

Comment 5

[Reed Loden [:reed]]

Rebooted the server. IPv6 connectivity seems to be working both inbound and outbound. Can somebody else confirm?

Comment 6

[Pascal Chevrel:pascalc]

I can confirm it works now :)