128898 - DNS: IPv6: slow resolution of host listed in hosts file, but not in DNS

Status: VERIFIED DUPLICATE of bug 68796

(Core :: Networking, defect)

Description

[Lee Harr]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.8) Gecko/20020220
BuildID:    0000000000

I have the address of my webserver listed in my /etc/hosts file, but
when I go to that page, the browser reports "resolving host..." at 
the bottom of the screen for about 10 seconds before loading the page.


Reproducible: Always
Steps to Reproduce:
1. List server in hosts
2. Do not have server in DNS
3. Go to host by name

Actual Results:  the browser reports "resolving host..." at 
the bottom of the screen for about 10 seconds
then loads the page.

Expected Results:  Load the page almost instantly.

Comment 1

[Fuzzy Gorilla]

Is your /etc/host.conf file configured to check the hosts before trying bind?
If it is not, then all queries will go to DNS before /etc/hosts.

Comment 2

[Lee Harr]

This problem only occurs with Mozilla, and only since upgrading from 0.9.6
to 0.9.8.  I tried changing the configuration, but the problem persists.

Comment 3

[Peter Haight]

I think this is a symptom of a larger problem. On my machine
(FreeBSD-4.5-STABLE) with a build from the CVS 1.0 branch a few days ago, some
DNS queries just take a long time. Mozilla will request a page
(www.vanguard.com) and will say "Resolving host" for about 10 seconds, but if I
go to a prompt and ping that same during that 10 second period, the OS has
already resolved it and it is in the cache. Then I wait another 9 seconds for
Mozilla to figure that out.

I haven't figured out what the pattern is. I run a debug build and if someone
could point me to where the DNS resolving code is and maybe some good debug
messages to turn on, I could perhaps figure it out or provide more info.

Comment 4

[Peter Haight]

I think I figured it out. On my machine the problem is that if you do a
gethostbyname2(host, AF_INET6) on some hosts FreeBSD takes a long time to fail.
(I'm currently asking the FreeBSD people why). There are two ways to see if this
is the problem you are having.

1. For your webserver stick an IPV6 address in your /etc/hosts file. Say your
webserver is www.vanguard.com, then stick this in there:
::1      www.vanguard.com

Once I do that for www.vanguard.com, it resolves very quickly. If I take it out,
it takes over a minute to resolve.

2. I'll post a patch that I'm currently using to fix this problem. Once I hear
from the FreeBSD folks, I'll come up with a better solution. This patch is only
temporary and can't be committed for obvious reasons.

Comment 5

[Peter Haight]

Attachment: Patch that provides a temporary fix for this problem.

This is more a patch to see if I've got the problem nailed than something that
should be committed.

Comment 6

[Peter Haight]

Actually, you may want to leave that IPv6 part out of the summary until we hear
from Lee Harr about whether this fixes his problem as I'm not entirely sure that
we were both experiencing the exact same problem.

Also, I heard back from the FreeBSD guys and they say the problem is that the
www.vanguard.com DNS server is not responding at all to the AAAA (IPv6) DNS
request. According to the relevant RFCs it is supposed to at least respond with
something like "unsupported". Since it doesn't respond at all FreeBSD eventually
just times out. 

I'm looking into fixing it by firing off two lookup threads. One to do an IPv6
lookup and another to do an IPv4 lookup and then take the first one that comes
back. If I do this, I'll also be fixing Bug #70213 along the way.

Comment 7

[Sean Rhodes]

Well, just to add my 2 cents,

I'm running FreeBSD 4.5 RELEASE.
I have /etc/hosts.conf /etc/resolv.conf set up correctly.
In /etc/hosts I've added addresses of servers that I visit often.

Mozilla still resolves these same addresses ("Resolving host...").

Other applications (lynx, konqueror, ping, sylpheed) clearly use /etc/hosts,
as they start transferring data immediately (also, my DNS servers are very slow
tonight, and after adding the addresses to /etc/hosts, I saw a huge difference
with these applications).

Mozilla seems to be the only one resolving these addresses.

Also, I don't really know much about the ipv6 issue, but this problem occurs for
 ALL the servers I added to /etc/hosts, so the problem is not with one
particular server.

Comment 8

[Lee Harr]

The workaround works for me.

I put a lines like

::1         www.mydomain.com
10.61.33.77 www.mydomain.com
::1         www2.mydomain.com
10.61.33.77 www2.mydomain.com

where the 10... address is the real address and the
::1 IPv6 address is just a fake.

It resolves immediately now.

Comment 9

[benc]

-> new
enough information to make me take this serious.

+neeti: who can look at the guts of the code to figure out what is going on here?

Comment 10

[Sean Rhodes]

OK, the ::1 workaround works for me too.

P.S. I forgot to mention before that I'm using RC3...

Comment 11

[Peter Haight]

The problem is not with either Mozilla or FreeBSD (sort of). The problem is that
there are a lot of broken DNS servers that aren't responding correctly to IPv6
(AAAA) DNS requests.I'm guessing it isn't a problem on other operating systems
right now because most don't come with IPv6 enabled by default. If other IPv6
enabled OSes are not having this problem, then maybe it is a problem  with
FreeBSD. The FreeBSD people think that the best way to deal with this is to
complain to the owners of faulty DNS servers to get them to fix them. While that
would be nice, in practice I'm having mixed success getting people to respond to
my requests.

The easiest fix is to just apply the patch that I posted. This works, but you
can no longer us IPv6 which is probably fine in most places. If we make that bit
of code a configurable run-time option somewhere that would work well.

The other choice is to use async DNS lookups. Fire off two DNS lookups. First
fire off an IPv6 lookup and then the normal one a second later. If the normal
one returns first, then use that one. I did do the asnyc DNS code for mozilla
unix (see Bug #70213). That patch does not include a fix for this specific
problem since I just like to use the patch that disables IPv6 lookups to solve
this problem.

So I think we should make a run-time configuration option that FreeBSD users can
use to turn off IPv6 lookups if they are having problems.

Comment 12

[Sean Rhodes]

Well after adding the ::1 lines to /etc/hosts, Mozilla was skipping the DNS
servers. At first.

Now Mozilla's gone into a state where, when entering an address in /etc/hosts,
it sits (seemingly) forever on "Resolving host"... This does not happen often,
but I've seen it numerous times. While Mozilla was in that state, I fired up
lynx, and within half a second it was transferring data from the very same
address Mozilla was choking on. And I bet Konqueror would have done the same had
I tried it.

But then I loaded this page, and it went quite quick (and this site isn't even
in /etc/hosts), so I really don't know what's going on. Mozilla gets into a
weird state sometimes where it just doesn't want to resolve an address. Seems
like it "comes out of it" after a while.

And I hear what Peter Haight says about the DNS servers not being able to handle
ipv6 addresses, but to me, thats a secondary issue. To me the primary issue is
why does Mozilla even CONTACT the DNS servers if the addresses are right there
in /etc/hosts for it to read?

Surely if this was an FreeBSD/ipv6 issue then Mozilla wouldn't be the only app
doing this?

Anyway, this is all above my head, I don't know anything about ipv6, so excuse
me if I'm totally missing the point...

Comment 13

[Peter Haight]

I think Mozilla is one of the very few apps that uses
gethostbyname2("www.google.com", AF_INET6) which means try an IPv6 lookup.

Because Mozilla always tries to use IPv6 to lookup before it tries IPv4. So it
would tell FreeBSD to lookup 'www.google.com' using IPv6 and FreeBSD would look
in your host file for an IPv6 address for google and since there was no IPv6
address in there, it would try asking the DNS server. Once that returned with an
error, then Mozilla would try IPv4 and use the value you put in your /etc/hosts
file.

When you stick a '::1' in your /etc/host, FreeBSD will return that immediately
when doing an IPv6 lookup for that host. In IPv6 ::1 means localhost (similar to
127.0.0.1). 

Hmm. I'm not sure why Mozilla doesn't try to use ::1 to contact the server after
it returns. If it is detecting somewhere else not to use IPv6 addresses, then we
should use that to decide to turn IPv6 lookups off. I'll look into that. 

Comment 14

[Sean Rhodes]

OK! Finally I think I understand what you were speaking about :) Thanks for
explaining that to me Peter. 

Sounds like a very likely cause to the symptoms we're seeing. Makes perfect sense.
I hope you manage to fix it soon!

Comment 15

[Lee Harr]

> Hmm. I'm not sure why Mozilla doesn't try to use ::1 to contact 
> the server after it returns.

Right. This was confusing me too...



> So I think we should make a run-time configuration option that 
> FreeBSD users can use to turn off IPv6 lookups if they
> are having problems.

This sounds good to me.

The ::1 workaround is not really acceptable, as now other
applications are failing the lookup.

Comment 16

[Peter Haight]

Ok. I figured out what's going on with the ::1 addresses. After Mozilla does an
IPv6 lookup it always does a IPv4 lookup immediately after and returns both
addresses to the caller. The caller then tries to connect to the first one (::1)
and fails because there is nothing listening at ::1. Then it tries the next
address which is the IPv4 one.

I'm still thinking that an option to disable IPv6 lookups is the best way to go.
Do the 'official' netscape/mozilla guys have an opinion? If there is no
objection, I'll try and add such an option.

Comment 17

[benc]

Please make some comments in bug 96432 about how you would like this done (but
don't mark them as dupes yet...)

Comment 18

[John G. Myers]

*** This bug has been marked as a duplicate of 68796 ***

Comment 19

[benc]

V/dupe