1398900 - Browser and system settings should not be used when FindProxyForURL() returns "DIRECT"

Status: RESOLVED WONTFIX

(WebExtensions :: Request Handling, defect, P3)

Description

[catusx]

Attachment: An extension that always return "DIRECT"

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36

Steps to reproduce:

1. Extract test-proxy-direct.zip attached and run `web-ext run -f "C:\Program Files\Nightly\firefox.exe"` or `web-ext run -f firefox-nightly` under the extracted location. (The extension has a proxy script that always return "DIRECT".)
2. In Preferences > General > Network Proxy, click on `Settings...` and select Manual proxy configuration, fill in any proxy you want. If you don't have a proxy, you can also fill in an invalid one like HTTP 127.0.0.1:8888.
3. Visit http://example.com


Actual results:

Firefox will connect to the proxy provided in the browser settings. (Or, if you used an invalid proxy in Step 2, you will see "The proxy server is refusing connections", which proves that proxy connection is attempted.) If "Use system proxy" is selected in browser settings, then system proxy is used. Also, if another extension has also registered a proxy script, then that proxy script is called before it falls back.


Expected results:

Since "DIRECT" is returned by the extension, Firefox should just use direct connection with no proxy, and override browser and system settings. Currently, returning an explicit proxy string, like "PROXY 127.0.0.1:8888" overrides all other settings, but "DIRECT" does not. This behavior also differs from Chrome Extensions.

If the extension instead wants to pass control to the next extension or use the browser settings, they should explicitly return "PASS", which is designed for the exact purpose. (See: https://bugzilla.mozilla.org/show_bug.cgi?id=1319634) Right now, it seems "DIRECT" does what "PASS'" should do. On the other hand, forcing real "DIRECT" connection is not possible. I believe this is not aligned to the design document: https://docs.google.com/document/d/1W45o5X2bFRPrTaQDFp9IzTJ8njCVfEgyENS7i2owaUI/edit

Consider a user who has installed a proxy-controlling extension and selected "DIRECT" in the extension. He/she would be very confused by the fact that some proxy sever is used despite what he/she just did. He/she probably would not recognize that he/she needs to clear the settings in Preferences, or in system settings. The extension itself is also unaware of the situation (since it cannot read Preferences or system settings) and thus cannot educate the user to do so.

Also, consider another user who is in intranet and needs to use proxy for almost everything except a few internal hosts. The network administrator has configured WPAD for the whole institution, but forgot to put a special case for "internal.example.com". He/she tries to install an extension that offers URL-based switching. So far so good, except the extension could not work properly when "DIRECT" should be used. The average user would assume that the extension should take over but it could not. He/she will have no idea what's happening here.

There are also situations that users want to configure Extension A, who has higher priority, to bypass proxies for a few intranet sites, but still use the proxies provided by Extension B. Right now, in addition to adding internal.example.com to Extension A, the user will also needs to do the same thing to Extension B, which is repetitive and non-intuitive. And what if Extension B does not provide URL-based switching? There are a lot of extensions that just provide proxies and they shouldn't be forced to provide some sort of switching.

To conclude, I would suggest that "DIRECT" should really use "DIRECT" connection, which also features compatibility with Chrome Extensions. Then we can implement Bug 1319634 for "PASS", which offers awesome additional functionality only available in Firefox.

Comment 1

[Shane Caraveo (:mixedpuppy)]

This is not really a webextension bug, or at least not a bug in the current browser.proxy implementation.  All the proxy api does is call the PAC script, then return the results to the underlying proxy system in firefox.  It does not make any consideration for *which* PAC script takes precedence (nor could it), or how other proxy settings in prefs affects the outcome.

A better outcome might be from adding the ability to get the proxy config so you may be able to determine whether or not you'll run into any issues.  That still wont address the fact that the proxy system will call each filter, and is designed to give precedence to a proxy over direct.

As far as Chrome behavior, these systems are simply not compatible and compatibility in the future is unlikely.

Comment 2

[catusx]

Thanks for the explanation. I now understand that everything is passed to the underlying system and it is indeed the expected behavior to prioritize proxies over direct.

Then the following question arises. How does "DIRECT" differ from the proposed "PASS" in Bug 1319634?

Also, is it possible to add a new type of response (maybe "BYPASS") to force the use of direct connection for a given request? This would also be quite helpful to address the scenarios above where no proxy is desired. I don't know if this requires any changes in the underlying proxy system though.

I understand that compatibility is not an issue here, but it seems like we are missing one important feature of forcing a direct connection, compared to Chrome. I've already got users reporting this as undesired (e.g. https://github.com/FelisCatus/SwitchyOmega/issues/1212) and I would love to see any way to solve it, whether compatible or not.

Comment 3

[Eric Jung [:ericjung]]

> This is not really a webextension bug, or at least not a bug in the current browser.proxy implementation. 

I don't agree. The problem is:

https://searchfox.org/mozilla-central/source/toolkit/components/extensions/ProxyScriptContext.jsm#136

  if (type === PROXY_TYPES.DIRECT) {
    return defaultProxyInfo;
  }

That should be:

  if (type === PROXY_TYPES.DIRECT) {
    return ProxyService.newProxyInfo("direct", "", -1, 0, 0, null);
  }

Comment 4

[Eric Jung [:ericjung]]

p.s. I know that returning

  ProxyService.newProxyInfo("direct", "", -1, 0, 0, null)

from nsIProtocolProxyFilter.applyFilter() bypasses Firefox's network preferences because I have a legacy XUL extension (FoxyProxy) which did precisely that.

Comment 5

[catusx]

I also want to add that several SwitchyOmega users are complaining about this. They cannot figure out the reason or solution on their own, but they did notice that the behavior is different than FoxyProxy.

Any chance we can reconsider this? For users who are upgrading from XUL-based extensions to WebExtensions, this seems be a breaking change and a drawback.

Comment 6

[Shane Caraveo (:mixedpuppy)]

(In reply to Eric Jung [:ericjung] from comment #3)
> > This is not really a webextension bug, or at least not a bug in the current browser.proxy implementation. 
> 
> I don't agree. The problem is:
> 
> https://searchfox.org/mozilla-central/source/toolkit/components/extensions/
> ProxyScriptContext.jsm#136
> 
>   if (type === PROXY_TYPES.DIRECT) {
>     return defaultProxyInfo;
>   }
> 
> That should be:
> 
>   if (type === PROXY_TYPES.DIRECT) {
>     return ProxyService.newProxyInfo("direct", "", -1, 0, 0, null);
>   }

Ok, I looked at this more, you've changed my mind.  If someone wants to make a patch (with test) it might move faster.

Comment 7

[Rob Wu [:robwu]]

proxy.register has been removed in bug 1443259