62388 - investigate nsSocketTransport bloat

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect, P3)

Description

[Chris Waterson]

Per news://news.mozilla.org/3A31455B.F9F4C9C3%40netscape.com, nsSocketTransport 
and its related data structures are taking up about 5% of gtkEmbed's heap memory 
(146KB) after loading www.cnn.com.

Jud noted that the HTTP service should only be caching eight socket transports, 
yet there are eighteen alive after loading the page!

Comment 1

[Darin Fisher]

This sounds like a problem with the pipeline request system.
Investigating...

Comment 2

[Darin Fisher]

The socket transport service allows for up to 50 active socket transports.  The
pipeline request system allows multiple HTTP channels to share the same socket
transport provided the host:port matches.  I suspect we aren't releasing the
socket transport when the pipeline finishes.  More to come...

Comment 3

[Darin Fisher]

The HTTP handler should only cache 8 active transports.  It can, however, cache
up to 20 idle transports.  I'm going to try to verify that at least the handler
does not believe it is exceeding these limits.

Comment 4

[Darin Fisher]

waterson: can you re-evaluate the bloat (or tell me how to reproduce your test)
after setting the following prefs:

user_prefs("network.http.max-connections", 1)
user_prefs("network.http.keep-alive", false)

thanks!

Comment 5

[Chris Waterson]

nsSocketTransport falls off the map when I set the preferences you suggest
above. (I set them in `all.js', because I'm using gtkEmbed which doesn't read
prefs from a profile.)

Here is the `before' histogram:

Type                    Count    Bytes %Total
TOTAL                   67674  4506378 100.00
nsImageGTK                128   684348  15.19
nsStyleContextData        527   402628   8.93
FrameArena                 80   329200   7.31
JS-unclassified          6836   239747   5.32
nsSocketTransport          66   171524   3.81
nsFontMetricsGTK         1124   134925   2.99
X-unclassified           3548   122284   2.71
gtk-unclassified         1160   117912   2.62
nsComponentManagerImpl   4149   114620   2.54
nsString                 2562   101190   2.25
unclassified-string      3162    95807   2.13
nsTextNode               1344    80640   1.79
nsZipArchive             1824    80148   1.78
HTMLAttributesImpl       2149    75052   1.67
nsCSSRule                 902    71900   1.60
nsPersistentProperties   2379    66998   1.49
StyleContextImpl         1110    66600   1.48
nsVoidArray              1883    66412   1.47
sscanf                   1381    65630   1.46
OTHER                   31360  1418813  31.48


And here is the `after' histogram:

Type                    Count    Bytes %Total
TOTAL                   72083  4467643 100.00
nsImageGTK                143   670816  15.01
nsStyleContextData        565   431660   9.66
FrameArena                 81   333315   7.46
JS-unclassified          6822   239351   5.36
nsFontMetricsGTK         1133   160541   3.59
X-unclassified           3597   130044   2.91
nsPersistentProperties   5922   129534   2.90
gtk-unclassified         1217   121310   2.72
nsComponentManagerImpl   4149   114620   2.57
nsString                 2669   105006   2.35
unclassified-string      3180    96199   2.15
nsTextNode               1377    82620   1.85
nsZipArchive             1828    80325   1.80
HTMLAttributesImpl       2230    77908   1.74
nsCSSRule                 910    72540   1.62
StyleContextImpl         1137    68220   1.53
nsVoidArray              1905    67368   1.51
sscanf                   1381    65630   1.47
dl-unclassified           475    64498   1.44
OTHER                   31362  1356138  30.35

Note that `nsSocketTransport' doesn't appear in the `top twenty' in the second
case.

FWIW, if you want to run these tests yourself (which I'd prefer!), please see

  http://lxr.mozilla.org/mozilla/source/tools/trace-malloc/README

Comment 6

[Darin Fisher]

Bug 63565 might be a dupe of this bug.  Investigating...

Comment 7

[Darin Fisher]

*** Bug 65830 has been marked as a duplicate of this bug. ***

Comment 8

[Darin Fisher]

nominating for moz 0.9

Comment 9

[Darin Fisher]

Added keyword nsbeta1

Comment 10

[neeti]

*** Bug 66393 has been marked as a duplicate of this bug. ***

Comment 11

[neeti]

I ran http://btek/cgi-bin/buster.cgi on linux with XPCOM_MEM_BLOAT_LOG set.
We no longer leak nsSocketTransport objects.

     
|<----------------Class--------------->|<-----Bytes------>|<----------------Obje
cts---------------->|<--------------References-------------->|
387 nsSocketTransport                             260        0    26582        0 
(    5.39 +/-     3.44)   676544        0 (   21.39 +/-    12.29)

Comment 12

[Tom Everingham]

verified per engr's comments