920367 - If ipv6 is not disabled and there is an ipv6 address, always try it first

Status: NEW

(Core :: Networking, defect, P3)

Description

[Alex]

Attachment: Screen Shot 2013-09-25 at 2.31.18 PM.png

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20130924030202

Steps to reproduce:

In recent versions of OS X, Apple provides an internal implementation of "Happy Eyeballs" in the getaddrinfo() API that often sorts perfectly working IPv6 addresses after IPv4 addresses if they are even 1ms slower to respond, this causes Firefox to simply stop using IPv6 to communicate with the host, as it seems to use the results provided directly and sees an IPv4 address first (bypassing the implementation of Happy Eyeballs in Firefox)

This can be seen in long lived connections like Gmail, where Firefox will switch from IPv6 to IPv4 randomly over the course of the day as connections are re-established, and OS X sorts the results differently.

The one upside to this is that there's no user visible breakage, it just results in an under utilisation of IPv6 connectivity as shown here: https://labs.ripe.net/Members/emileaben/hampered-eyeballs


Actual results:

1) Connect to example.org
2) OS X might return sorted address list placing IPv4 addresses first
3) Firefox connects to IPv4 address first, not attempting IPv6


Expected results:

1) Connect to example.org
2) OS X might return sorted address list placing IPv4 addresses first
3) Firefox connects to IPv6 address first anyway, then attempting IPv4 if IPv6 doesn't work

Comment 1

[Simon]

I am seeing this as well.

Comment 2

[Dragana Damjanovic [:dragana]]

Attachment: bug920367-fix-v1.patch

This patch will always put IPv6 first, except disableIPv6 is set

Comment 3

[Masatoshi Kimura [:emk]]

Do we ignore operating system's preference on all platforms?

Comment 4

[Dragana Damjanovic [:dragana]]

The list received from getaddrinfo will take into account operating system's preference. If we do any rearrangement of this list we are going to ignore it. (the patch above does it).

Do we check somewhere in the code for preferences expicitly? (mPreferIPv6 is set depending on this list)
We can fix Apple problem by ignoring OS's preference or we can do an additional check of the preference.

Comment 5

[Patrick McManus [:mcmanus]]

(In reply to Masatoshi Kimura [:emk] from comment #3)
> Do we ignore operating system's preference on all platforms?

right now, yes this is consistent

Comment 6

[Patrick McManus [:mcmanus]]

so this has the rather global impact of preferring v6 on all platforms, all the time (which is rather broader than the title of the bug). I think the time has come to give this a try if we have some kind of metric we can watch - afterall the current code is rather agnostic as to what it is using.

dragana, do you have a suggestion on a metric? (perhaps rate of syn-retry, which I think we would need to collect?)

(sorry that this is so blast from the past - spring cleaning)

Comment 7

[Dragana Damjanovic [:dragana]]

just an info:

https://www.ietf.org/mail-archive/web/v6ops/current/msg22455.html

Comment 8

[Dragana Damjanovic [:dragana]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=721c10cce928

Comment 9

[Dragana Damjanovic [:dragana]]

Attachment: bug920367-fix-v1.patch

rebased.

Comment 10

[dagger.bugzilla]

If I'm reading the patch correctly, it unconditionally sorts v6 addresses above v4 ones. This is specifically against RFC 3484/6724; we should either just trust the OS resolver to do the right thing, or we should sort addresses correctly (which is a lot harder than it sounds, especially considering the system admin can specify their own custom ordering rules).

This bug report was about OS X's resolver, which I'm aware does _not_ do the right thing, but a) that means that any attempts at workarounds ought to be OS X-only, and b) since this bug was filed, the OS X resolver has been changed to give v6 a significant head-start, so the original problem of switching between v6 and v4 constantly should be mostly fixed by Apple.

Comment 11

[Alex]

I haven't had a chance to try 10.11 personally, but I've got a friend who's running it (And also has IPv6, lucky) and they can't reproduce the bad behavior I'm seeing.

This isn't needed for Linux and Windows, but even with the changes in 10.11 I'm inclined to think that Firefox should be smarter here. I'm not sure if there's a way to read the OS level preference, but if IPv6 is enabled at the OS/adapter level and IPv4 isn't specifically preferred, then Firefox should at least try a IPv6 address first, even if it's sorted behind IPv4 addresses.

Comment 12

[Dragana Damjanovic [:dragana]]

This is an experiment. It is going to be just in Nightly for some time and if it is to annoying for to many people it is going to be out in a day :)

We are just interested to see how many broke ipv6 are there. 

Bug 1177848 added a bit of telemetry for this and I want to see how bad it is going to be.

Comment 13

[dagger.bugzilla]

There's a few cases where you should be preferring v4. If the server uses 6to4 or Teredo, then native v4 will give a better experience to that server. If they're ULA and the client doesn't have a ULA address, then the connection will probably fail over v6 anyway. If the _client_ only has 6to4/Teredo, then v4 will give a better experience to most servers. (getaddrinfo() handles all these cases by default.) And if the user has configured an override (e.g. "always prefer v6 even if 6to4/Teredo", "always prefer v4", or "prefer v6 except for a few specific hosts") then they must've had a reason for it.

(User config goes in /etc/gai.conf for Linux and somewhere in the registry for Windows; you can view it with `netsh interface ipv6 show prefixpolicies`.)

Also, I'm not sure what this will tell us about broken v6. In its current form, it'll just cause a lot of v6 connection failures on v4-only hosts (except not on Windows, because Windows doesn't do AAAA queries unless it thinks you have v6 or you explicitly ask for them to happen), and if you restrict it only to hosts with v6 available then you'll be matching getaddrinfo()'s behavior for most queries anyway.

Comment 14

[Patrick McManus [:mcmanus]]

we should do this in 48

Comment 15

[sgarg]

Comment 14 says "we should do this in 48". Pray, what is "this"? 

As per http://www.test-ipv6.com/faq_avoids_ipv6.html, Firefox does try an IPv6 address and falls back to IPv4 if the 300ms timer expires. I just filed Bug 1271079 asking that there be an ability to either (i) disable Happy Eyeballs, and (ii) configure the timeout.

I am behind a he.net tunnel and for some reason I keep getting a surly panda instead of a running unicorn at http://ismyipv6working.com/. I can confirm that till at least 6 months back, Firefox did show me the running unicorn.

Also http://www.test-ipv6.com/ gives me a 10/10

For a change the Microsoft browsers behave as intended.

Comment 16

[scineram]

I have a router at

C:\>nslookup lede.lan
Server:  UnKnown
Address:  fe80::0123:45ff:fe67:89ef

Name:    lede.lan
Addresses:  fd0c:4920:b75a::1
          192.168.1.1

When I visit the webinterface at https://lede.lan, SixOrNot indicates the connection is over 4 (orange). If I turn 4 off and refresh, the page will load fine over 6 (green). Since I observe the same issue with the ping command, does FF inherit this behavior from Windows? Are these actions according to the specs?

Comment 17

[Dragana Damjanovic [:dragana]]

(In reply to scineram from comment #16)
> I have a router at
> 
> C:\>nslookup lede.lan
> Server:  UnKnown
> Address:  fe80::0123:45ff:fe67:89ef
> 
> Name:    lede.lan
> Addresses:  fd0c:4920:b75a::1
>           192.168.1.1
> 
> When I visit the webinterface at https://lede.lan, SixOrNot indicates the
> connection is over 4 (orange). If I turn 4 off and refresh, the page will
> load fine over 6 (green). Since I observe the same issue with the ping
> command, does FF inherit this behavior from Windows? Are these actions
> according to the specs?

Firefox does not change anything. A OS does DNS look up and returns a list of addresses. The firefox will try the first address first. This is a OS decision.

Comment 18

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 19

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 20

[Wellington Torrejais da Silva]

https://codereview.chromium.org/7029049