364443 - Interpreting "network.http.max-connections-per-server" to mean "proxy" chokes speed

Status: RESOLVED DUPLICATE of bug 648603

(Core :: Networking, defect)

Description

[Pasi Keinänen]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061025 BonEcho/2.0 (tete009 G7 SSE2)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061025 BonEcho/2.0 (tete009 G7 SSE2)

PROBLEM
=======
Firefox 2 branch is speed limited in it's network connections in a following scenario:

1. Proxy server is in use and configured (not inbound connection limited, cpu limited or choking)

2. Firefox is configured to access network through the proxy

3. Firefox is set to:

network.http.max-connections-per-server = 8
network.http.max-connections = 48

In this situation, maximum number of connections Firefox makes is 8.

Regadless of how many separate servers (say, in different tabs) are being loaded.

WHY AN ISSUE
============
It is impossible to make efficient use of network bandwidth AND proxy resources without hammering on a single server.

BECAUSE

1. If one keeps -per-server limit @ 8, then loading 8 different servers simultaneously in 8 tabs causes each tab to get 0-1 http connections.

This will result in time-outs, slow loading and "host not responding" type errors OR errors in loading some particular elements on a page (image, css, etc). 


2. If one ups the -per-server limit to 64 (and universal max-connections at the same time), then when loading a single server, it gets hammered with 64 requests simultanously.


SOLUTION
========

New config setting:

network.http.max-connections-per-proxy

Which sets the maximum outbound http connections to a proxy AND which can exceed the amount of per-server max settings (server here is an actual target end-state server, not a middle-man proxy).

Per-proxy setting should not exceed maximum global connections limit (i.e. network.http.max-connections).


OR

make network.http.max-connections be the amount of simultanous connections that can be made to a Proxy, WHEN a proxy is in use and without capping this limit with the network.http.max-connections-per-server setting



Reproducible: Always

Steps to Reproduce:
1. Configure Firefox to use a (local) proxy. Make sure proxy is not cpu, throughput or connection limited

2. Set :

network.http.max-connections-per-server = 8
network.http.max-connections = 48

3. Open a set of 8 or more tabs (i.e. pages) simultaneously (Bookmarks / Open All in Tabs).

4. View your Proxy's real-time connection load (how many requests are being handle by it from Firefox)

5. Time how long it takes to load ALL the pages in the above configuration (time from "All in tabs" to situation where all tabs have successfully loaded). Calculate the amount of rendering errors due to unsuccessful http requests in the loaded tabs/pages.

6. End cache (ff & proxy), restart both program (ff & proxy).

7. Configure FF to bypass proxy. Do the same test (3) WITHOUT proxy in between.

8. Measure the time to load all pages (much less than in (5)). Count the number of rendering errors (much less than in (5)).



Actual Results:  
1. Loss of http connection throughput speed in a Proxy use situation when http limits are set to RFC recommended levels

In plain english: slower retrieving of multiple pages at the same time, EVEN when the proxy is not slowing down the retrieval.

2. Alternatively, spurious badly rendered pages, due to a very limited set of connections being shared by multitude of simultaneous requests

In plain english, more renderings of pages that have missing images, missing stylesheet or odd flow of retreived material.



Expected Results:  
Excpected functioning of firefox configuration:

In proxy use situation:

network.http.max-connections-per-server = MAX connections to a single server that is not the proxy in use

network.http.max-connections = Max connections altogether to all servers (by default, this is also the max amount sent to the proxy)


Expected results of rendering speed & http request success rate:

In both proxy and non-proxy situations, multiple simultaneous page request is ONLY limited by the amount of what the proxy can handle, NOT by an internal switch inside Firefox that that limits requests to proxy in a way that cannot be circumvented.

That is, in a situation where proxy is not cpu/throughput/connection limited, page loading of multiple pages simultaneously should be equally fast in proxy and non-proxy configurations.

The same applies for the number of successful requests out of all the requests that the page loadings initiate.


No workaround exists that is acceptable.

Setting 

network.http.max-connections-per-server

to a very high number that a proxy can still handle easily, will drown a single end-state server (the server that proxy requests) in too many http request AND will violate RFCs.