681198 - Speedbit Video Accelerator conflicts with > 32 concurrent connections

Status: RESOLVED WONTFIX

(Core :: Networking: HTTP, defect)

Description

[Shane Beasley]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0
Build ID: 20110811165603

Steps to reproduce:

Initially, I found that Firefox 6 was choking on my netflix.com home page. I worked on several reproduction methods. Attached is the simplest method, which consists of a freestanding HTML application (index.{html,js,css}).

To use:

1. start a new instance of Firefox
2. load index.html
3. enter a number of connections to open (N) and an unused TCP port number on your system (P)
4. click Load

[PROTIP: Steps 2-4 can be condensed by loading index.html?n=N&p=P where N and P are integers (e.g. index.html?n=4&p=1234).]

JavaScript then generates an HTML document containing N <img> elements with unique IP addresses, like this (P=1234):

  <img src="http://127.255.255.254:1234/nosuch.png" ... />
  <img src="http://127.255.255.253:1234/nosuch.png" ... />
  <img src="http://127.255.255.252:1234/nosuch.png" ... />
  <!-- etc. -->

Although all 127.0.0.0/8 addresses refer to the local host, Firefox will open a new connection for each address, resulting in a total of N open connections.


Actual results:

A sufficiently high value of N will cause Firefox 6 to consume 100% of one CPU core. After that, no new connections can be made, not even in other windows/tabs. The problem does not appear to obey a timeout and cannot be resolved by closing the offending tab. The only solution is to restart the browser.

(The problem occurs for N=33 or greater on my 32-bit Windows XP desktop PC, and for N=32 or greater on my 64-bit Windows 7 laptop. I do not understand the discrepancy.)

Note that the number of concurrent connections is cumulative across all tabs/windows. It is therefore advised to test using a fresh instance of Firefox with no preexisting tabs/windows.


Expected results:

It should try but fail to open N connections without any other obvious side effects. (In fact, this is what it appears to do in Firefox 5.x -- I assert that this problem was introduced in Firefox 6.)

Comment 1

[Shane Beasley]

Attachment: repro material

I get a 500 Internal Server Error when I try to submit a .zip containing my index.{html,css,js} file... Trying with a condensed (single-file) .html.

Comment 2

[Shane Beasley]

NB: I've found that I can work around the problem by setting network.http.max-connections=64 in about:config.

Comment 3

[Patrick McManus [:mcmanus]]

Shane, I really appreciate you taking the time to make the report. Something funny is going on.

STR is WFM on win7 pro 64 bit.. works fine with n=32, 33, 50 and 150.

STR is also WFM on xp sp3. also at 32, 33, 50 and 150.

But it isn't obvious to me that your STR will always create large numbers of parallel connections. Under normal circumstances those connections to unused localhost ports will fail very quickly; some will certainly have a chance to close before we try and open the full set (this all happens synchronously). For some reason (firewall maybe? different speed of machine?) they may not fail as quickly on your host. That means I might not really be testing the path you are suggesting has a problem (and indeed that path did change in firefox 6).

So I made a couple changes to your test - I had it contact a real server that I gave 75 secondary IP addresses to. I had that server feed back a real png but I had it delay 15 seconds before doing so. I used that 15 second interval to confirm that N (up to 75) parallel connections were really being made. After 15 seconds elapsed I made sure that all the images loaded and networking worked in other tabs.

I tested that setup on win7 pro 64 bit and it was fine with 32, 33, 34, 75.

I did the same tests on xp sp3.

They all WFM, so we've ruled out any basic problem using that many connections simultaneously in those OS's. Now we need to look for advanced problems :)

Any idea what is different in your environment?

If you generate an HTTP LOG (https://developer.mozilla.org/en/HTTP_Logging) maybe we can figure out something from that.

Comment 4

[Shane Beasley]

Attachment: 7zipped log file

In fact, I had written a small Perl "server" as you describe in comment 3, but I discarded it when I found that HTML/JS did the same thing. :)

I've since found that it also happens on my PC in safe mode, but *not* on a 32-bit XP SP2 virtual machine (VMware) that I had lying around. Perhaps it's because it's a VM, but more likely it's because the VM doesn't have a bunch of extraneous software on it, as do the physical machines. :)

Unfortunately, since I'm a power user, there's quite a list of potential offenders. I'm going to see if I can narrow it down on the VM (snapshotting as I go)... starting with the AVG antivirus. :)

Meanwhile, I've attached a 7-zipped log file. Beware: It's 765MB compressed down to 150KB because its extremely repetitive. There's something like this over and over:

3768[933780]: STS poll iter [1]
3768[933780]:   active [65] { handler=6b50200 condition=0 pollflags=6 }
3768[933780]:   active [64] { handler=6b50120 condition=0 pollflags=6 }
3768[933780]:   active [63] { handler=6b50040 condition=0 pollflags=6 }
3768[933780]:   active [62] { handler=6b4ff60 condition=0 pollflags=6 }
3768[933780]:   active [61] { handler=6b4fe80 condition=0 pollflags=6 }
3768[933780]:   active [60] { handler=6b4fda0 condition=0 pollflags=6 }
3768[933780]:   active [59] { handler=6b4fcc0 condition=0 pollflags=6 }
3768[933780]:   active [58] { handler=6b4fbe0 condition=0 pollflags=6 }
3768[933780]:   active [57] { handler=6b4fb00 condition=0 pollflags=6 }
3768[933780]:   active [56] { handler=6b4fa20 condition=0 pollflags=6 }
3768[933780]:   active [55] { handler=6b4f940 condition=0 pollflags=6 }
3768[933780]:   active [54] { handler=6b4f860 condition=0 pollflags=6 }
3768[933780]:   active [53] { handler=6b4f780 condition=0 pollflags=6 }
3768[933780]:   active [52] { handler=6b4f6a0 condition=0 pollflags=6 }
3768[933780]:   active [51] { handler=6b4f5c0 condition=0 pollflags=6 }
3768[933780]:   active [50] { handler=6b4f4e0 condition=0 pollflags=6 }
3768[933780]:   active [49] { handler=6b4f400 condition=0 pollflags=6 }
3768[933780]:   active [48] { handler=6b4f320 condition=0 pollflags=6 }
3768[933780]:   active [47] { handler=6b4f240 condition=0 pollflags=6 }
3768[933780]:   active [46] { handler=6b4f160 condition=0 pollflags=6 }
3768[933780]:   active [45] { handler=6b4f080 condition=0 pollflags=6 }
3768[933780]:   active [44] { handler=6b4efa0 condition=0 pollflags=6 }
3768[933780]:   active [43] { handler=6b4eec0 condition=0 pollflags=6 }
3768[933780]:   active [42] { handler=6b4ede0 condition=0 pollflags=6 }
3768[933780]:   active [41] { handler=6b4ed00 condition=0 pollflags=6 }
3768[933780]:   active [40] { handler=6b4ec20 condition=0 pollflags=6 }
3768[933780]:   active [39] { handler=6b4eb40 condition=0 pollflags=6 }
3768[933780]:   active [38] { handler=6b4ea60 condition=0 pollflags=6 }
3768[933780]:   active [37] { handler=6b4e980 condition=0 pollflags=6 }
3768[933780]:   active [36] { handler=6b4e8a0 condition=0 pollflags=6 }
3768[933780]:   active [35] { handler=6b4e7c0 condition=0 pollflags=6 }
3768[933780]:   active [34] { handler=6b4e6e0 condition=0 pollflags=6 }
3768[933780]:   active [33] { handler=6b4e600 condition=0 pollflags=6 }
3768[933780]:   active [32] { handler=6b4e520 condition=0 pollflags=6 }
3768[933780]:   active [31] { handler=6b4e440 condition=0 pollflags=6 }
3768[933780]:   active [30] { handler=6b4e360 condition=0 pollflags=6 }
3768[933780]:   active [29] { handler=6b4e280 condition=0 pollflags=6 }
3768[933780]:   active [28] { handler=6b4e1a0 condition=0 pollflags=6 }
3768[933780]:   active [27] { handler=6b4e0c0 condition=0 pollflags=6 }
3768[933780]:   active [26] { handler=5f1ff20 condition=0 pollflags=6 }
3768[933780]:   active [25] { handler=5f1fe40 condition=0 pollflags=6 }
3768[933780]:   active [24] { handler=5f1fd60 condition=0 pollflags=6 }
3768[933780]:   active [23] { handler=5f1fc80 condition=0 pollflags=6 }
3768[933780]:   active [22] { handler=5f1fba0 condition=0 pollflags=6 }
3768[933780]:   active [21] { handler=5f1fac0 condition=0 pollflags=6 }
3768[933780]:   active [20] { handler=5f1f9e0 condition=0 pollflags=6 }
3768[933780]:   active [19] { handler=5f1f900 condition=0 pollflags=6 }
3768[933780]:   active [18] { handler=5f1f820 condition=0 pollflags=6 }
3768[933780]:   active [17] { handler=5f1f740 condition=0 pollflags=6 }
3768[933780]:   active [16] { handler=5f1f660 condition=0 pollflags=6 }
3768[933780]:   active [15] { handler=5f1f580 condition=0 pollflags=6 }
3768[933780]:   active [14] { handler=5f1f4a0 condition=0 pollflags=6 }
3768[933780]:   active [13] { handler=5f1f3c0 condition=0 pollflags=6 }
3768[933780]:   active [12] { handler=5f1f2e0 condition=0 pollflags=6 }
3768[933780]:   active [11] { handler=5f1f200 condition=0 pollflags=6 }
3768[933780]:   active [10] { handler=5f1f120 condition=0 pollflags=6 }
3768[933780]:   active [9] { handler=5f1f040 condition=0 pollflags=6 }
3768[933780]:   active [8] { handler=5f1ef60 condition=0 pollflags=6 }
3768[933780]:   active [7] { handler=5f1ee80 condition=0 pollflags=6 }
3768[933780]:   active [6] { handler=5f1eda0 condition=0 pollflags=6 }
3768[933780]:   active [5] { handler=5f1ecc0 condition=0 pollflags=6 }
3768[933780]:   active [4] { handler=5f1ebe0 condition=0 pollflags=6 }
3768[933780]:   active [3] { handler=5f1eb00 condition=0 pollflags=6 }
3768[933780]:   active [2] { handler=5f1ea20 condition=0 pollflags=6 }
3768[933780]:   active [1] { handler=5f1e940 condition=0 pollflags=6 }
3768[933780]:   active [0] { handler=5f1e860 condition=0 pollflags=6 }
3768[933780]:   idle [0] { handler=5f1dfa0 condition=0 pollflags=0 }
3768[933780]:   calling PR_Poll [active=66 idle=1]
3768[933780]: poll timeout: 65535
3768[933780]:     timeout = 65535000 milliseconds
3768[933780]:     ...returned after 0 milliseconds
3768[933780]:   PR_Poll error [-5974]
3768[933780]: STS poll iter [1]
...

Comment 5

[Shane Beasley]

This is definitively an incompatibility with Speedbit Video Accelerator (http://www.videoaccelerator.com/). In particular:

- I uninstalled it from my PC and laptop and the problem magically went away.
- I installed it on my 32-bit XP virtual machine and the problem appeared.

Video Accelerator works in part by injecting DLLs into other processes, thereby voiding all warranties. Also, I found that this was the cause of my somewhat recent (past several months?) inability to start the Cygwin X server [xwin.exe] on my laptop.

In sum, I'm going to stop using Speedbit Video Accelerator now. :)

(Good news: Bywifi Video Accelerator still works like a charm, in part because it operates as a proxy server, which is very much supported by Firefox et al.)

I completely understand if this is marked INVALID, WONTFIX, etc. (Another route is to yell at the user if Video Accelerator is installed, though that's probably outside your purview.)

Comment 6

[Patrick McManus [:mcmanus]]

Shane, thanks for figuring this out. I'm not quite sure how to proceed - but for starters I'll change the bug title to make it easier to find ;)

Comment 7

[Honza Bambas (:mayhemer)]

We may want add the dll to the block list [1].  The dll seems to be sblsp.dll and we probably have other problems with that LSP, see [2].

[1] http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/nsWindowsDllBlocklist.h
[2] https://bugzilla.mozilla.org/buglist.cgi?quicksearch=SpeedBit%20Video%20Accelerator

Comment 8

[Patrick McManus [:mcmanus]]

we could blocklist speedbit, but the issue appears to have gone away as a practical matter