Closed Bug 906896 Opened 11 years ago Closed 10 years ago

Increase number of permitted EventSource connections

Categories

(Core :: Networking: HTTP, defect)

Product:
Core
Component:
Networking: HTTP
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: brent.tubbs, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36

Steps to reproduce:

Visit http://ssebin.btubbs.com/multi/.  Click the "Add Counter" button 6 or more times.


Actual results:

Once you reach 6 counters (each with its own EventSource connection), you can't make any more.  If you open another tab and try to access the site there, it won't connect.


Expected results:

You should be able to make a *lot* more EventSource connections.  Since this feature is meant to fill a similar niche to websockets, something closer to the network.websocket.max-connections limit of 200 would make sense.

I don't have an opinion on whether this would best be done by raising network.http.max-persistent-connections-per-server, or by having EventSource connections capped with a different setting like websockets have.  But the current cap of 6 simultaneous connections is a severe constraint.  I have a web application (https://bitbucket.org/yougov/velociraptor/) that maintains two EventSource connections.  Users who open the application in multiple tabs often get confused as to why the app is no longer responsive.
We should mark EventSource connections so they don't count toward the 6 per-host connection limit.  Patrick, do we have a convenient way in the channel API to do that?
Flags: needinfo?(mcmanus)
(In reply to Jason Duell (:jduell) from comment #1)
> We should mark EventSource connections so they don't count toward the 6
> per-host connection limit.  Patrick, do we have a convenient way in the
> channel API to do that?

nope. I'm not even sure its desirable.
Flags: needinfo?(mcmanus)
Well, we either need to provide 1) a way for EventSource to tell necko to exclude itself from the connection limit, or 2) teach necko to recognize EventSource connections and exclude them.

Glancing at EventSource.cpp, it seems the most likely approach for #2 would be to check for "text/event-stream" in the Accept header (ATM only EventSource.cpp uses that MIME type).  Seems a shame to add a strcmp to every request just for this, but I suppose it's more flexible (in case some addon or something ever rolls something to listen to EventSources on top of an HTTP request).

For #1 we could add a C++-only method to some IDL.  I agree it's not something we want to expose to JS.

Don't have a strong opinion about which road to take. #2 seems to capture more cases.  Module owner's choice :)
Flags: needinfo?(mcmanus)
(In reply to Jason Duell (:jduell) from comment #3)
> Well, we either need to provide 1) a way for EventSource to tell necko to
> exclude itself from the connection limit, or 2) teach necko to recognize
> EventSource connections and exclude them.
> 

why do we need to provide a way to do that? What makes Event Source special in a way that, say, a COMET hanging GET isn't?

Or to put it another way - what is the connection limit trying to achieve and why is that not applicable to event source?

another fun question: why isn't the answer here to use websockets?
Flags: needinfo?(mcmanus)
btw http limits per tab - now that's a patch I could get behind.
> what is the connection limit trying to achieve 
> and why is that not applicable to event source?

Whatever it is that connection limits are trying to achieve, it's surely not to get into a state where HTTP requests get silently dumped into a queue that will never be serviced because there are already 6 EventSource objects to the same host strewn somewhere among all open tabs.

> What makes Event Source special in a way that, say, a COMET hanging GET isn't?

Well, they're both "special" :) in that they can wind up glomming up the max-connections count (and reducing parallelism).  We should probably try to ensure that neither can do that (for COMET we could place requests that have been made but not responded to into the "long lived" category after some timeout and not count toward the limit any more).

Or maybe we can just get away (as we have so far) with doing nothing special for either, and just leave it to web-devs to ensure they aren't clogging their own hosts' max-cxn limits.  It does seem like a bit of a footgun.  But there's lots of those on the Internet :)

> btw http limits per tab - now that's a patch I could get behind.

Sure, that's probably a good idea.  Open a bug and we can discuss it there.  I suspect the issue would come down to whether we'd wind up hammering some sites with too many sockets at once (IIRC the reason for the max-cxn limit in the first place): the sites that are most likely to see large increases (google/facebook/etc) probably already have the infrastructure to handle it, so I'd be more worried about the long tail.  (And we'd need to make sure rogue scripts couldn't use the feature for DOS attacks?
See Also: → 778884
From bug 778884, we see there is a very faulty behavior: once the user opens several tabs containing an event source, everything looks like all following connections to the same host are just put "on hold", without any error message. This feels very wrong to me.
Patrick,

Would you take a patch that excludes EventSource connections (and maybe long-lived XHRs) from the connection limit?
Flags: needinfo?(mcmanus)
(In reply to Jason Duell (:jduell) from comment #9)
> Patrick,
> 
> Would you take a patch that excludes EventSource connections (and maybe
> long-lived XHRs) from the connection limit?

I can't believe EventSource was a API that got spec'd. This is so broken.

I'd rather not not - its just a loophole around the limit - and it will be abused that way.

what's the story on other browsers? I guess if the horse is out of the barn we could do the eventsource only thing at a higher limit.
Flags: needinfo?(mcmanus)
>do the eventsource only thing at a higher limit.

WebSockets is another such thing.
>I can't believe EventSource was a API that got spec'd. This is so broken.

Can you please describe why?
(In reply to 4esn0k from comment #11)
> >do the eventsource only thing at a higher limit.
> 
> WebSockets is another such thing.

websockets multiplexes results on one tcp connection. it what should be used for this kind of feature. It also only uses HTTP to bootstrap, so once websockets is established it is not goverened by the HTTP specification.

(In reply to 4esn0k from comment #12)
> >I can't believe EventSource was a API that got spec'd. This is so broken.
> 
> Can you please describe why?

the http specfication requires low parallel connection limits to the same origin - so specifying http transcations that become head of line blockers inevitably creates the problem you are seeing. it is inherent in the design.

if the server was upgraded to spdy/3 that would also solve the issue because spdy is able to carry large numbers of parallel streams over one tcp connection - so blocking streams are not a problem.
(In reply to Patrick McManus [:mcmanus] from comment #10)
> 
> what's the story on other browsers? I guess if the horse is out of the barn
> we could do the eventsource only thing at a higher limit.

Bruno told me that other browsers have the same issue reported in bug 778884. He actually filed bugs for other browsers too. Bruno, maybe you can add some information and pointers here?

(In reply to Patrick McManus [:mcmanus] from comment #14)
> it is what should be used
> for this kind of feature.

Sooo should we make EventSource obsolete, then? Because the only thing EventSource is doing is this kind of feature.
Flags: needinfo?(brmichel)
I WontFix'd the Chromium bug for this: https://code.google.com/p/chromium/issues/detail?id=275955#c2.
wontfix based on compat - comment 16
Status: UNCONFIRMED → RESOLVED
Closed: 10 years ago
Resolution: --- → WONTFIX
> websockets multiplexes results on one tcp connection

is such multiplexing supported by Firefox or any other browser?

> the http specfication requires low parallel connection limits to the same origin - so specifying http transcations that become head of line blockers inevitably creates the problem you are seeing. it is inherent in the design.
so new protocol can be created, name it HTTPforEventSource, where more than 6 connections is allowed
similary to WebSockets, connection limit can be applied while connecting, after server respond with "Content-type:text/event-stream" nolimit rules can be used...
Here's your new protocol: SPDY (http://www.chromium.org/spdy/spdy-whitepaper), which is getting standardized as HTTP/2 (http://http2.github.io/http2-spec/). SPDY is supported today in IE11+, Firefox, Chrome, and Opera: http://caniuse.com/spdy.
> Bruno, maybe you can add some information and pointers here?

Sure, the bug for webkit is on https://bugs.webkit.org/show_bug.cgi?id=92670 and the one for chromium is on http://code.google.com/p/chromium/issues/detail?id=139688. If I remember correctly, I have also opened a bug for opera but I didn't find it anymore.
Flags: needinfo?(brmichel)
EventSource works nicely as a simple push mechanism. It's much easier to implement on the server compared to websockets, works nicely with most proxies, and has a fairly easy to use fallback option for older browsers (long poll hack via xhr)

http://blog.fastmail.fm/2012/01/09/building-the-new-ajax-mail-ui-part-1-instant-notifications-of-new-emails-via-eventsourceserver-sent-events/

The connection limit issue does suck though. It's annoying that there's a "hack" for websockets to have a larger limit, but none for event source and Chrome and Firefox are both just punting as "won't fix".

Oh well. Being pragmatic, we came up with a hack for inter-tab communication to deal with the connection limit if anyone else runs into this issue.

http://blog.fastmail.fm/2012/11/26/inter-tab-communication-using-local-storage/
You need to log in before you can comment on or make changes to this bug.