773648 - integrate libunbound

Status: RESOLVED WONTFIX

(Core :: Networking: DNS, enhancement)

Description

[Josh Aas]

Attachment: fix v1.0

We are planning to use libunbound as our own host resolver, to replace usage of 'getaddrinfo'.

This patch applies on top of ldns integration.

Comment 1

[Josh Aas]

Attachment: fix v1.1

Minor updates, still doesn't actually work.

Comment 2

[Wouter Wijngaards]

Attachment: v1.2 patch, builds

the includes for nss,nspr used include <nss3/file.h> and changed to include "file.h", and it builds successfully.

Comment 3

[Wouter Wijngaards]

Attachment: v1.3 patch, builds and removed warning

fixed warnings reported by the compiler.

Comment 4

[Wouter Wijngaards]

Attachment: patch for ipc, Makefiles, and for network DNS.cpp and nsHostResolver.cpp

Comment 5

[Wouter Wijngaards]

Attachment: unbound code in this patch (config.h edited)

Comment 6

[Wouter Wijngaards]

Added attachment for netwerk, unbound and a 1.2 ldns to the ldns page.  Together, they make firefox resolve using libunbound.  Libunbound is code-configured to use 8.8.8.8 as resolver.

Comment 7

[Josh Aas]

Attachment: network part v1.5

Updated for hg diff compatibility.

Comment 8

[Josh Aas]

Attachment: unbound part 1.5

Updated for hg diff compatibility.

Comment 9

[Wouter Wijngaards]

Attachment: shell script that fetches unbound and ldns code into mozilla-central

this shell script I used to update the source and header files.  it does not touch the config.h files, and the ldns-version in its util.h.  It also does not update Makefiles, or hg add.  Just puts source files in the correct place.  Used it to update to the latest (svn) version of unbound and ldns, this unbound version has a ttl in its libunbound lookup results.

Comment 10

[Wouter Wijngaards]

Attachment: patch 1.6.diff (ldns + unbound + netwerk)

This is with hg diff.  Contains ldns, unbound and other changes in one bundle.  Also fixed are build error for Josh (I hope), it can cancel queries, it cleans up on resolver shutdown.

Comment 11

[Josh Aas]

Attachment: patch v1.7

Includes build fixes for OS X. Now builds and runs on OS X and Linux, Windows not tested yet.

Comment 12

[Wouter Wijngaards]

Attachment: 1.7 with fixes for compile (osx, windows), but needs event.h

This is 1.7 with
- include cmath in other unrelated windows code to make it compile for me (gcc 4.7.2?)
- ifdefs in net.h, config.h util.h in ldns for _WINDOWS (ssize_t, other includes).  This makes ldns compile.

Comment 13

[Wouter Wijngaards]

Attachment: unbound, ldns, libevent 1.8

Compiles on linux.  Contains the libevent patch.
- I had to adjust the shutdown code, so that it exits more cleanly.

Comment 14

[Josh Aas]

(In reply to Wouter Wijngaards from comment #13)
> Created attachment 716567 [details]
> unbound, ldns, libevent 1.8
> 
> Compiles on linux.  Contains the libevent patch.
> - I had to adjust the shutdown code, so that it exits more cleanly.

Lets not mix those two patches, leave them separate. Just require both to build. Both patches are already gigantic.

Comment 15

[Wouter Wijngaards]

sure, how do you separate them?  hg diff ipc > one patch and hg diff ... > other patch?

Comment 16

[Wouter Wijngaards]

Attachment: unbound (set 1.9) patch

tweaked to compile on windows.

Comment 17

[Wouter Wijngaards]

Attachment: netwerk/dns 1.9 patch

updated to compile on windows.

Comment 18

[Landry Breuil (:gaston)]

Are there plans to provide a --with-system-libunbound configure flag ? (same question for ldns of course...)

Comment 19

[Landry Breuil (:gaston)]

(In reply to Wouter Wijngaards from comment #6)
> Added attachment for netwerk, unbound and a 1.2 ldns to the ldns page. 
> Together, they make firefox resolve using libunbound.  Libunbound is
> code-configured to use 8.8.8.8 as resolver.

If i look at the comment attachment #709056 [details] [diff] [review], do i get it right that 'hardcoding' 8.8.8.8 is used only for debug purposes, and in production the dns servers from resolv.conf will be used ?

Comment 20

[Landry Breuil (:gaston)]

Comment on attachment 716567 [details]
unbound, ldns, libevent 1.8

Hrm, sorry for messing up patch flags.. i though bugzilla was able to ungzip files on the fly.

Comment 21

[Wouter Wijngaards]

(In reply to Landry Breuil (:gaston) from comment #19)
> If i look at the comment attachment #709056 [details] [diff] [review] [diff] [review], do i
> get it right that 'hardcoding' 8.8.8.8 is used only for debug purposes, and
> in production the dns servers from resolv.conf will be used ?

In production something else would be good.  This is an open problem really, as very often the dns resolvers from resolv.conf are not good enough (do not support DNSSEC).  An in-between is to set libunbound to be a full-recursor and fetch the data from the internet itself, that requires that the router lets UDP traffic (with DNSSEC in it) through, and not all do (especially in hotspots, shops, hotels, bad isps).

Getting the servers from resolv.conf is also pretty tricky (OSX, Windows).

One way to do this is to probe the servers to see if they support DNSSEC (i.e. like our dnssec-trigger project does).  But I am not sure if this is the direction you want to go into.  It does not always work, and you can end up in situations where you cannot get DNSSEC to work (traffic to 8.8.8.8 is blocked and you cannot get a certified answer that it is blocked, so this means someone could preted it is blocked to force you to fallback to insecure mode ...).

Comment 22

[Jan Beich]

Attachment: bsd config (for v1.9)

Works fine here:

  $ ./dist/bin/firefox -P test -no-remote www.freebsd.org
  [1367056860] libunbound[54131:0] notice: init module 0: validator
  [1367056860] libunbound[54131:0] notice: init module 1: iterator
  [1367056860] libunbound[54131:0] info: resolving www.freebsd.org. A IN
  DNS lookup answer thread starting execution.
  DNS answer handler evloop once.
  [1367056860] libunbound[54131:0] info: response for www.freebsd.org. A IN
  [1367056860] libunbound[54131:0] info: reply from <.> 8.8.8.8#53
  [1367056860] libunbound[54131:0] info: query response was CNAME
  [1367056860] libunbound[54131:0] info: resolving www.freebsd.org. A IN
  [1367056860] libunbound[54131:0] info: response for www.freebsd.org. A IN
  [1367056860] libunbound[54131:0] info: reply from <.> 8.8.8.8#53
  [1367056860] libunbound[54131:0] info: query response was ANSWER
  DNS answer handler evloop once.
  ...
  
Why not just run as subconfigure? It'd leave some porting issues at
vendor expense.

Comment 23

[Jan Beich]

Attachment: use libevent-1.4 compatibility header (for v1.9)

unbound fallbacks to event2/*.h headers without HAVE_EVENT_H while
only event.h is listed under config/system-headers. This would
break visibility for gcc_hidden.h + --with-system-libevent.

And downstream may prefer libevent-1.4 for various reasons e.g.,
OpenBSD have it as part of base system.

Comment 24

[Landry Breuil (:gaston)]

Comment on attachment 742694 [details] [diff] [review]
bsd config (for v1.9)

Review of attachment 742694 [details] [diff] [review]:
-----------------------------------------------------------------

::: netwerk/dns/unbound/config.h
@@ +259,5 @@
>  
>  /* Define to 1 if you have the `sendmsg' function. */
>  #define HAVE_SENDMSG 1
>  
> +#if defined(__linux__) || defined(__DragonFly__) || defined(__FreeBSD__)

OpenBSD has setresgid/setresuid in addition to setregid/setreuid. NetBSD only have the latter.

Comment 25

[cloos]

Note that those who care about dnssec run a validating resolver locally (either on each lan, each box or even each vm).

It would be most inappropriate for gecko to ignore a local validating resolver.

Which does not imply that it shouldn’t re-do the validation.  Just that it should use the local cache whenever it can.  The data might already be local, and it shouldn’t presume that it is the only app on the box which cares about the data it queries.

It seems that telling libunbound to proxy its queries via the resolv.conf resolvers, and validate the results, should cover all bases.