Closed Bug 1177909 Opened 9 years ago Closed 7 years ago

websocket doesn't work if firefox uses the system's proxy settings on ubuntu linux

Categories

(Core :: Networking: WebSockets, defect)

Product:
Core
Component:
Networking: WebSockets
Version:
38 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla56
Tracking Flags:
Tracking Status
firefox56 --- fixed

People

(Reporter: soyer, Assigned: xeonchen)

References

Details

(Whiteboard: [necko-active][proxy])

Attachments

(2 files)

Bug 1177909 - Part 1: support perferred proxy type while using system proxy setting;
59 bytes, text/x-review-board-request
bagder
: review+
Details
Bug 1177909 - Part 2: add test cases when WebSockets use system proxy;
59 bytes, text/x-review-board-request
bagder
: review+
Details 
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/600.6.3 (KHTML, like Gecko) Version/7.1.6 Safari/537.85.15

Steps to reproduce:

setup the proxy setting to use the system's proxy
open a page with a websocket chat (ws or wss)


Actual results:

the page loads, the websocket content try to connect without the proxy, and the chat doesn't work


Expected results:

the chat should be working and the ws or wss socket should connect via the proxy
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:40.0) Gecko/20100101 Firefox/40.0

I was able to reproduce soyer@irl.hu's problem on OS X 10.10 and a Gentoo Linux box. (Please tell me if I should paste the User Agent for the Gentoo box as well)
Confirming the problem on OS X 10.10 and Firefox ESR 38.3.0 too.
It breaks slack.com communication platform.
I am able to reproduce this. Will pass up to Developers.
Status: UNCONFIRMED → NEW
Component: Untriaged → Networking: HTTP
Ever confirmed: true
Product: Firefox → Core
Whiteboard: [necko-backlog]
Component: Networking: HTTP → Networking: WebSockets
I've bumped into this with "Mozilla/5.0 (X11; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0". Slack, Whatsapp, IRCCloud connections fail.

This needs some attention. Immediately.
Gary, could you take a look at this issue?
Flags: needinfo?(xeonchen)
Whiteboard: [necko-backlog] → [necko-backlog][proxy]
(In reply to Cengiz Can from comment #5)
> I've bumped into this with "Mozilla/5.0 (X11; Linux x86_64; rv:54.0)
> Gecko/20100101 Firefox/54.0". Slack, Whatsapp, IRCCloud connections fail.
> 
> This needs some attention. Immediately.

I cannot reproduce this on below platforms (by following steps from: bug 1270035 comment 0):

Firefox 53.0.3 64-bit on Ubuntu 16.04.2 LTS
Firefox 55.0a1 2017-05-31 64-bit on macOS Sierra 10.12.5
Firefox 55.0a1 2017-05-31 64-bit on Debian GNU/Linux 9.0 x86_64

All lights are green on https://slack.com/help/test

Cengiz, would you describe how "connections fail" is?
You just can't connect to the site? Something is broken?
Flags: needinfo?(xeonchen) → needinfo?(cengizc)
(In reply to Gary Chen [:xeonchen] (needinfo plz) from comment #7)

Hello Gary Chen! Thank you for responding!
 
> Cengiz, would you describe how "connections fail" is?
> You just can't connect to the site? Something is broken?

I'm using Firefox 54.0b11 (64-bit) on Arch Linux, with Gnome 3.24.2 and NetworkManager 1.8.0-1.

By "connections fail" I was trying to tell that websocket requests are getting unexpectedly closed. I think they're not getting upgraded to websocket:// after the initial HTTP call.

I'm behind a Squid instance and I've enabled it as the system wide HTTP/HTTPS proxy. (SOCKS and FTP is undefined)

If I instruct Firefox to "Use system proxy settings", https://slack.com/help/test reports that:

-- Web Socket (Message Proxy A): Failed (Socket closed unexpectedly)
-- Web Socket (Message Proxy B): Failed (Socket closed unexpectedly)
-- Web Socket (Message Proxy C): Failed (Socket closed unexpectedly)
-- Web Socket (Message Proxy D): Failed (Socket closed unexpectedly)
-- Web Socket (Flannel): Failed (Socket closed unexpectedly)
-- Web Socket (Flannel [No Compression]): Failed (Socket closed unexpectedly)

However, if I choose "Manual proxy settings" and fill them in, same test page report 100% success. (Well, except the network bandwidth :P)

I'll try to get some logs from Firefox and share it here.
(In reply to Cengiz Can from comment #8)
> (In reply to Gary Chen [:xeonchen] (needinfo plz) from comment #7)
> 
> Hello Gary Chen! Thank you for responding!
>  
> > Cengiz, would you describe how "connections fail" is?
> > You just can't connect to the site? Something is broken?
> 
> I'm using Firefox 54.0b11 (64-bit) on Arch Linux, with Gnome 3.24.2 and
> NetworkManager 1.8.0-1.
> 

Sorry I'm not familiar with Gnome, are you setting http_proxy and https_proxy in the environment variable or through NetworkManager (supposed to be GUI?)?

> By "connections fail" I was trying to tell that websocket requests are
> getting unexpectedly closed. I think they're not getting upgraded to
> websocket:// after the initial HTTP call.
> 
> I'm behind a Squid instance and I've enabled it as the system wide
> HTTP/HTTPS proxy. (SOCKS and FTP is undefined)
> 
> If I instruct Firefox to "Use system proxy settings",
> https://slack.com/help/test reports that:
> 
> -- Web Socket (Message Proxy A): Failed (Socket closed unexpectedly)
> -- Web Socket (Message Proxy B): Failed (Socket closed unexpectedly)
> -- Web Socket (Message Proxy C): Failed (Socket closed unexpectedly)
> -- Web Socket (Message Proxy D): Failed (Socket closed unexpectedly)
> -- Web Socket (Flannel): Failed (Socket closed unexpectedly)
> -- Web Socket (Flannel [No Compression]): Failed (Socket closed unexpectedly)
> 

I also tested with Squid, but unfortunately I got no luck.

> However, if I choose "Manual proxy settings" and fill them in, same test
> page report 100% success. (Well, except the network bandwidth :P)
> 
The bandwidth test takes some time.

> I'll try to get some logs from Firefox and share it here.

Thanks! Looking forward to hear more from you!
FYI, I tried to reproduce this bug few days ago on echo test page http://www.websocket.org/echo.html using squid proxy. When I set the proxy manually, websockets didn't work because squid doesn't support upgrade request, see http://lists.squid-cache.org/pipermail/squid-users/2017-January/013953.html. When I set "Use system proxy" WS worked because for some reason firefox didn't use proxy for websockets although it used proxy for other HTTP traffic.
I tried to set proxy using Gnome's setting panel, but still cannot reproduce.

--
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0
Proxy: squid/3.5.23
(In reply to Gary Chen [:xeonchen] (needinfo plz) from comment #11)
> I tried to set proxy using Gnome's setting panel, but still cannot reproduce.

Gary, I think you might have been testing in a network that does allow direct connection to the Internet, and this might be the reason why you couldn't reproduce this issue!? 

We have the same problem as described in this bug report, and I just reproduced it with Firefox 53.

Our company network does not allow direct connections to the Internet, so Firefox has to use the proxy for all connections. If we configure the HTTP and HTTPS proxy in the Gnome proxy settings, and then try the test on http://www.websocket.org/echo.html , Firefox tries to establish websocket connections directly (not using the proxy), so the firewall blocks those connections and the test fails. Using the http_proxy and https_proxy environment variables instead of the Gnome proxy settings leads to the same problem.

If we configure the HTTP and HTTPS proxy directly in Firefox, Firefox is using the proxy for websocket connections (it establishes a CONNECT tunnel) and the test is successful. I'm not sure why this isn't working for Michail, we also use a Squid proxy and Firefox uses a CONNECT tunnel, which does work with Squid, as described in the mailing list post he linked.
(In reply to Carsten Sommer from comment #12)
> If we configure the HTTP and HTTPS proxy directly in Firefox, Firefox is
> using the proxy for websocket connections (it establishes a CONNECT tunnel)
> and the test is successful. I'm not sure why this isn't working for Michail,
> we also use a Squid proxy and Firefox uses a CONNECT tunnel, which does work
> with Squid, as described in the mailing list post he linked.

CONNECT is used only in case of WSS. WS uses unencrypted traffic on port 80 and squid should be configured to allow CONNECT only on port 443.
(In reply to Carsten Sommer from comment #12)
> (In reply to Gary Chen [:xeonchen] (needinfo plz) from comment #11)
> > I tried to set proxy using Gnome's setting panel, but still cannot reproduce.
> 
> Gary, I think you might have been testing in a network that does allow
> direct connection to the Internet, and this might be the reason why you
> couldn't reproduce this issue!? 
> 

After using iptables to deny network traffic to the internet, I can now reproduce this bug.

Thank you Carsten!
Assignee: nobody → xeonchen
Status: NEW → ASSIGNED
Whiteboard: [necko-backlog][proxy] → [necko-active][proxy]
Flags: needinfo?(cengizc)
If we use system proxy setting under Linux, it will compare the scheme, which is "wss" here, and cannot find any matched proxy server with the given scheme ([1]).

However, WebSocket will set |nsIProtocolProxyService::RESOLVE_PREFER_HTTPS_PROXY| flag while calling |AsyncResolve|, which leads to use HTTPS proxy for connection ([2], [3]).

[1] http://searchfox.org/mozilla-central/rev/d840ebd5858a61dbc1622487c1fab74ecf235e03/toolkit/system/unixproxy/nsUnixSystemProxySettings.cpp#503
[2] http://searchfox.org/mozilla-central/rev/d840ebd5858a61dbc1622487c1fab74ecf235e03/netwerk/protocol/websocket/WebSocketChannel.cpp#2925
[3] http://searchfox.org/mozilla-central/rev/d840ebd5858a61dbc1622487c1fab74ecf235e03/netwerk/base/nsProtocolProxyService.cpp#2027
Comment on attachment 8876683 [details]
Bug 1177909 - Part 1: support perferred proxy type while using system proxy setting;

https://reviewboard.mozilla.org/r/148030/#review153848
Attachment #8876683 - Flags: review?(daniel) → review+
Comment on attachment 8876684 [details]
Bug 1177909 - Part 2: add test cases when WebSockets use system proxy;

https://reviewboard.mozilla.org/r/148032/#review153852

::: netwerk/test/unit/test_bug1177909.js:68
(Diff revision 1)
> +      }
> +    });
> +  });
> +}
> +
> +async function TestProxyTypeByURI(uri, flags) {

This method is called from 5 places in this test file and all of them pass a foxed 0 in the flags argument - so maybe remove that argument and always pass on a 0 to TestProxyType() ?
Attachment #8876684 - Flags: review?(daniel) → review+
s/foxed/fixed =)
Comment on attachment 8876684 [details]
Bug 1177909 - Part 2: add test cases when WebSockets use system proxy;

https://reviewboard.mozilla.org/r/148032/#review153852

> This method is called from 5 places in this test file and all of them pass a foxed 0 in the flags argument - so maybe remove that argument and always pass on a 0 to TestProxyType() ?

Yeah, but in the 6th test function |testWebSocketProxy()|, it calls with an argument |proxyFlags|.
Pushed by gachen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4a954e387736
Part 1: support perferred proxy type while using system proxy setting; r=bagder
https://hg.mozilla.org/integration/autoland/rev/004b0a044f7b
Part 2: add test cases when WebSockets use system proxy; r=bagder
https://hg.mozilla.org/mozilla-central/rev/4a954e387736
https://hg.mozilla.org/mozilla-central/rev/004b0a044f7b
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox56: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla56

looks as if this is back in FF71: i can't get a websockets connection for accessing a ravello VM to work if I'm using a http proxy.

(In reply to Mathias Homann from comment #26)

looks as if this is back in FF71: i can't get a websockets connection for accessing a ravello VM to work if I'm using a http proxy.

See https://bugzilla.mozilla.org/show_bug.cgi?id=1601671#c9

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: