1759004 - RemoteAgent requests using a relative path and with a custom host header will be rejected by httpd.js

Status: NEW

(Remote Protocol :: Agent, defect, P3)

Description

[Julian Descottes [:jdescottes]]

In Bug 1750689 we added the possibility to define allowed hosts/origins to connect to the RemoteAgent server.

However the base server we use, httpd.js also performs some validation on the incoming requests and might reject requests we would like to accept.

The main issue seems to be around the following check: https://searchfox.org/mozilla-central/rev/61c6b2ca66b8fc7d82a71d6f045871e5a8649cbb/netwerk/test/httpserver/httpd.js#1721-1730

        var scheme = identity.getScheme(host, port);
        if (!scheme) {
          dumpn(
            "*** unrecognized hostname (" +
              hostPort +
              ") in Host " +
              "header, 400 time"
          );
          throw HTTP_400;
        }

which is only performed if the URL was not absolute (checked here).

So if the client creates a request with the following characteristics:

• has a host header matching the allowed hosts
• has a relative URL (eg /session instead of http://{remoteagenthost}:{remoteagentport}/session)

httpd.js will return an error 400 before we can validate the request.

We can workaround some of that by using {Identity.add](https://searchfox.org/mozilla-central/rev/61c6b2ca66b8fc7d82a71d6f045871e5a8649cbb/netwerk/test/httpserver/httpd.js#1083) for each additional host defined in remote.hosts.allowed.

There is still a question about how to support "any IP address", because they would also be rejected by httpd.js

Comment 1

[Julian Descottes [:jdescottes]]

httpd.js will also reject handshake requests using the ws: protocol because of the following check: https://searchfox.org/mozilla-central/rev/61c6b2ca66b8fc7d82a71d6f045871e5a8649cbb/netwerk/test/httpserver/httpd.js#1232-1246

  _validate(scheme, host, port) {
    if (scheme !== "http" && scheme !== "https") {
      dumpn("*** server only supports http/https schemes: '" + scheme + "'");
      dumpStack();
      throw Components.Exception("", Cr.NS_ERROR_ILLEGAL_VALUE);
    }
    if (!HOST_REGEX.test(host) && host != "[::1]") {
      dumpn("*** unexpected host: '" + host + "'");
      throw Components.Exception("", Cr.NS_ERROR_ILLEGAL_VALUE);
    }
    if (port < 0 || port > 65535) {
      dumpn("*** unexpected port: '" + port + "'");
      throw Components.Exception("", Cr.NS_ERROR_ILLEGAL_VALUE);
    }
  },

All in all, it seems httpd.js will either need to be modified or replaced when we want to support consumers trying to use BiDi in docker containers or similar.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I just noticed the same with my WebSocket client on MacOS. A connection is still refused. But as long the origin header could be manually tweaked it works.

Given that httpd.js is just a normal webserver it might be denied to make such changes. But we could ask. If that's the case we might indeed have to think about an alternative and maybe use the DevTools WebSocket server implementation, which is also more lightweight but requires certain additions. So not that easy and quick.

Comment 3

[Julian Descottes [:jdescottes]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #2)

I just noticed the same with my WebSocket client on MacOS. A connection is still refused. But as long the origin header could be manually tweaked it works.

Does this have anything to do with this problem? The connection was rejected because of the origin header, but as soon as I added it to remote.origins.allowed, I could successfully connect using the "WebSocket Client" app (I believe we are talking about the same app since you were the one who suggested it to me)

(so to be clear I assume this app, same as the WebSocket constructor in JS, crafts an initial http:// request to perform the handshake, and that's why it works, but it might not always be the case)

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Oh, I actually misread the failure message and actually forgot to revert a change for remote.origins.allowed. Setting it to ws://localhost:9222 and removing all header overrides in the Websocket client makes it actually work pretty fine. So it's indeed not related.

Comment 5

[Julian Descottes [:jdescottes]]

Some additional information about the issues with ws:/ requests, it seems to only impact requests using an absolute path, ie containing ws:/.

Such requests will hit _validate from https://searchfox.org/mozilla-central/rev/61fcf20da23053d54f9a057e208dd5e7407a977c/netwerk/test/httpserver/httpd.js#1892.

      if (
        !serverIdentity.has(scheme, host, port) ||
        fullPath.charAt(0) != "/"
      ) {
        dumpn("*** serverIdentity unknown or path does not start with '/'");
        throw HTTP_400;
      }

(serverIdentity.has calls serverIdentity._validate(scheme, host, port), which throws for anything != http or https)

ws: requests using a relative URL will skip this check because it is guarded by https://searchfox.org/mozilla-central/rev/61fcf20da23053d54f9a057e208dd5e7407a977c/netwerk/test/httpserver/httpd.js#1860

    if (fullPath.charAt(0) != "/") {
    // ...
    }

So, ws: requests with relative paths do work, but with the same limitation as explained in the description, ie they can't use custom host headers.

Comment 6

[Julian Descottes [:jdescottes]]

Will review this again after the WebDriver BiDi release meeting this week.

Comment 7

[Julian Descottes [:jdescottes]]

Split out comment 5 to new bug