1523367 - Support equivalent of Chromium's `--host-resolver-rules` in Necko

Status: NEW

(Core :: Networking, enhancement, P3)

Description

[Nick Alexander :nalexander [he/him]]

While investigating Bug 1522133, I ran into Chromium's --host-resolver-rules. These rules allow to do (at least) two things:

1. Hard-code DNS name resolution to fixed IP addresses, for example with a rule like MAP * 127.0.0.1. This is equivalent to setting the pref network.dns.forceResolve="127.0.0.1", added by Bug 1361099.

2. Map ports for requests, for example with a rule like MAP *:80 127.0.0.1:8080.

I'd like to get some feedback from the Necko team on how to support something like 2.) in Necko.

As far as I can tell this is "transparent to the request". I.e., the socket connection would be opened to remote port 127.0.0.1:8080, but the HTTP header would still be host foo.com (i.e., without port 8080). I had a quick skim through the Necko code and this abstraction doesn't appear to exist: that is, it's not clear that we can differentiate between the "request ports" and the "socket ports". But I'm no Necko expert; maybe this is possible in the nsIChannel abstraction.

It's also not clear to me if this can be done with an HTTP/HTTPS proxy. That is, suppose I want to achieve the equivalent behaviour but I can't modify Necko itself. If I tell Firefox to route all traffic through a proxy, can the proxy itself do the port mapping silently? Is this visible to consumers?

This doesn't really block Bug 1522133, but it sure would be useful.

Comment 1

[Nick Alexander :nalexander [he/him]]

valentin: I see that you reviewed McManus's changes for Bug 1361099. Can you take a stab at orienting me on this, or redirect to somebody more suitable? Many thanks!

Comment 2

[Valentin Gosu [:valentin] (he/him)]

(In reply to Nick Alexander :nalexander [he/him] from comment #0)

While investigating Bug 1522133, I ran into Chromium's --host-resolver-rules. These rules allow to do (at least) two things:

1. Hard-code DNS name resolution to fixed IP addresses, for example with a rule like MAP * 127.0.0.1. This is equivalent to setting the pref network.dns.forceResolve="127.0.0.1", added by Bug 1361099.

2. Map ports for requests, for example with a rule like MAP *:80 127.0.0.1:8080.

I'd like to get some feedback from the Necko team on how to support something like 2.) in Necko.

As far as I can tell this is "transparent to the request". I.e., the socket connection would be opened to remote port 127.0.0.1:8080, but the HTTP header would still be host foo.com (i.e., without port 8080). I had a quick skim through the Necko code and this abstraction doesn't appear to exist: that is, it's not clear that we can differentiate between the "request ports" and the "socket ports". But I'm no Necko expert; maybe this is possible in the nsIChannel abstraction.

That is correct, at the moment we have no way of differentiating between the two. We could, but I'm not sure it's worth the extra complexity.

It's also not clear to me if this can be done with an HTTP/HTTPS proxy. That is, suppose I want to achieve the equivalent behaviour but I can't modify Necko itself. If I tell Firefox to route all traffic through a proxy, can the proxy itself do the port mapping silently? Is this visible to consumers?

A proxy would work. The web requests shouldn't see that we're using a proxy, and the port mapping should be fairly straight-forward.

Comment 3

[Nick Alexander :nalexander [he/him]]

Just an update on this. I talked further with Valentin on Vidyo. He suggests that it would be possible to do this port mapping in Necko, and probably not that hard -- but it would violate a lot of assumptions up and down the stack.

So I started to look into proxying, and discovered that a solution based on goproxy is trivial. Like, really trivial. Modifying this example ever so slightly, the following is sufficient:

package main

import (
	"flag"
	"fmt"
	"github.com/elazarl/goproxy"
	"log"
	"net"
	"net/http"
	"strings"
)

func main() {
	verbose := flag.Bool("v", false, "should every proxy request be logged to stdout")
	addr := flag.String("addr", ":4040", "proxy listen address")
	flag.Parse()
	proxy := goproxy.NewProxyHttpServer()
	proxy.Tr.Dial = func(network, addr string) (c net.Conn, err error) {
		rewritten := addr
		if strings.HasSuffix(rewritten, ":80") {
			rewritten = "127.0.0.1:8080"
		} else if strings.HasSuffix(rewritten, ":443") {
			rewritten = "127.0.0.1:8081"
		}

		return net.Dial(network, rewritten)
			}
	proxy.Verbose = *verbose
	log.Fatal(http.ListenAndServe(*addr, proxy))
}

Running that on :4040, and WPR on :8080 and :8081, and setting network.proxy.{http,https} = "localhost:4040", I have success. (With two certs, for goproxy and for WPR, but we can address that.)

Now, we could run this proxy separately, or we could integrate it into WPR, or we could try to make WPR actually be a full proxy for HTTPS traffic, and then grow support for this remapping. I will file an issue against WPR to investigate the latter two bits.

Comment 4

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander [he/him] from comment #3)

Now, we could run this proxy separately, or we could integrate it into WPR, or we could try to make WPR actually be a full proxy for HTTPS traffic, and then grow support for this remapping. I will file an issue against WPR to investigate the latter two bits.

Not surprisingly, I'm not the first to consider this. I commented on a (closed) ticket against WprGo (that's how they seem to refer to it in their issue tracked) outlining my use case. Let's see if we get any response!

Comment 6

[Honza Bambas (:mayhemer)]

I think we now have a 99% equivalent with two preferences:

• network.dns.forceResolve to force the same IP for all hosts
• network.socket.forcePort to remap ports to one or more different ports to actually connect to

Closing, please reopen if more work is needed (I take this bug a bit like a meta bug now)

Comment 7

[Honza Bambas (:mayhemer)]

Reopening, because this bug requirement was originally to allow full per-host name mapping, which we don't have. Ignore comment 6.

Comment 10

[Maverickson]

Attachment: Display list dump 2

Comment 11

[Maverickson]

Attachment: links which are not working on moz

Comment 12

[github]

+1. Any progress on remapping per-hostname IP mappings? Right now I can only use Chrome for this purpose, but much prefer Firefox!

This works in chrome:

chrome --host-resolver-rules="MAP example.com 192.168.12.34"

Comment 13

[Valentin Gosu [:valentin] (he/him)]

There is a way to override the IP of specific hostnames:

const gOverride = Cc["@mozilla.org/network/native-dns-override;1"].getService(
  Ci.nsINativeDNSResolverOverride
);

gOverride.addIPOverride("example.com", "1.1.1.1");
gOverride.addIPOverride("example.org", "::1:2:3");
gOverride.addIPOverride("example.net", "N/A"); // NO IPs

// Use this when you no longer need the overrides
// gOverride.clearOverrides();

If you execute this in the browser console (Ctrl-Shift-J) then you get a similar behaviour.
No port mapping yet. We might possibly implement these as a command line option in the future to match chrome, but for now you can use this workaround. Note this only overrides the native DNS - so you should disable DNS over HTTPS before by setting network.trr.mode to 5

Comment 14

[Lawrence Li]

+1. Can we have this mapping feature?