109415 - investigate default memory cache size

Status: VERIFIED FIXED

(Core :: Networking: Cache, defect)

Description

[Cathleen]

We should start put in some time to investigate our defult memory cache size, to
determine whether if we should make it an adjustible value base on the size of
physical memory.  Also, something else to consider... should we have a fixed
upper bound and lower bound.

hyatt has proven that if you increase memory cache, our page-load performance
gets boosted.

need QA help on this one...

Comment 1

[Darin Fisher]

this sounds like a dupe of bug 105344 filed by hyatt.

Comment 2

[lchiang]

It does sound like bug 105344 although we could use this bug report to track the
QA test progress/results until we have final results to post to the other bug.

cathleen - when do you need this by?  I'll ask tever/jrgm/kerz to get some QA
investigation into this and to make sure the approach is what you're looking for
before we start the measurements.

Comment 3

[Cathleen]

sometime within the 0.9.7 time frame if that's doable.  :-)

Comment 4

[lchiang]

Moving to 0.9.8.  Tom or John or Barrett, will you have the cycles to assist on
this one to generate some data?

Comment 5

[Tom Everingham]

yes, I could take a look at this but it will have to be after the holiday season
if that's ok

Comment 6

[lchiang]

tom, if you are going to look into this, pls assign it to you. Thanks!

Comment 7

[Tom Everingham]

reassigning to myself

Comment 8

[Tom Everingham]

Ok, so I started running some tests using John's page loader tool and got some
interesting results.   

The test machine was a 450Mhz P3 with 128M ram and Win NT4.  Build id was
2002012503.  Mem cache size was variable and disk cache left at its default of
50000.  Between runs the machine was restarted to clear out any video memory
that might be in use, all processes were closed, a new profile created and used,
and the sidebar was closed.  

First run numbers:

Size   Avg. Median  Average  Sequence	
2048       1094      1120       7	
4096       1084      1111       1	
8192       1100      1125       8	
12280      1111      1136       9	
16384      1110      1132       2	
24576      1131      1152       10	
32768      888       953        3	
49152      893       959        4	
65538      898       1034       5	
81922      891       968        6	

I am not sure what it means yet but the numbers are showing a slight degredation
in performance going from a 4M cache to a 24M cache.  Then after 32M there is a
jump in performance presumably from all of the test pages being entirely cached.
 The sequence number is just the order in which the test was run and didn't seem
to matter.

The second set of tests were run on the same machine.  The only difference was
that the mem cache was adjusted from 4M to 28M in 4M increments. 

Size   Avg. Median  Average  Sequence	
4096       1094      1122       7
8192       1105      1130       6
12280      1107      1131       5
16384      1120      1145       4
20480      1130      1152       3
24576      1139      1157       2
28672      1138      1150       1

Attaching graphs.

Comment 9

[Tom Everingham]

Attachment: Graph of first set of tests

Comment 10

[Tom Everingham]

Attachment: second

Comment 11

[Tom Everingham]

There was some concern over whether this was a valid test for default cache size
performance but I am not sure of a better way to test this.  Please add any
comments/suggestions.  

Comment 12

[gordon]

I would expect a slight degradation in performance as the memory cache capacity
is raised until it becomes large enough for the entire test to remain resident.
 The more items in the cache, the longer it takes to access any given item
(though the incremental cost should be very small since we're using a fairly
efficient hash table).

These results appear to be consistant with that.  It would be good to run the
iBench test with the same system.  We might get more meaningful numbers.  Also,
we need startup times, to spot any unintended side-effects of changing the
default memory cache size.

Thanks Tom.  This is a good start.

Comment 13

[Cathleen]

tever, thanks for supplying this initial set of data.  This is a good start!!  :-)

The purpose of this bug is to find out the most optimal memory cache size for
various physical memory constraint.  So, the same test needs to be done on
machines with 32MB, 64MB, 128MB, 256MB, 512MB and 1GB physical memory.

Comment 14

[Tom Everingham]

with this test I can not tell what the optimal mem cache size would be for my
128M system so I don't see any reason to run it on other systems just yet. 
According to the graph, it would be any size above the point where the whole
test is cached but that doesn't seem correct.  I will try ibench next to see if
there is any difference.  

Comment 15

[Darin Fisher]

tom: agreed.  and fwiw, many bios' allow you to specify a reduced memory size. 
or, if you are running these tests on linux, you can tell the kernel how much
memory of the available ram to use.  there is a kernel parameter that you can
specify at boot time.  let me know if you want help setting this up.

Comment 16

[Tom Everingham]

I ran the ibench (actually minibench) tests last night on the isolated LAN in
the networking lab.  The results are not revealing anything but I will post them
anyway.  I'm thinking that the entire test was probably cached early on so this
also may not be a good tool for testing default mem cache size.  The system was
a Dell R450 PII, 256M ram, NT4 OS.  Build id 2002092408.  

Size    1st loop   avg subsequent loop     total    (number of loops = 4)
----    --------   -------------------   ---------
2048     28.4 s         26.9 s            109.3 s
4096     28.5           27.0              109.6
8192     28.6           27.0              109.5
12280    28.4           27.1              109.6
16384    28.5           26.9              109.0
24576    28.7           27.0              109.7
37768    28.4           26.8              108.8
49152    28.6           26.9              109.3
65538    28.4           27.1              109.6
81922    28.5           26.9              109.4

Comment 17

[Tom Everingham]

cathleen:  Darin mentioned that you were possibly going to have jrgm modify his 
performance tests to step back and forwards through pages.  Do you know if this 
is going to happen.    It might prove to be a better tool for investigating this 
bug.   

Comment 18

[John Morrison]

I have worked on a back/forward test. I should dig that up and finish testing 
it.

As for the above results, looks like the size is moot for the given test.

Comment 19

[gordon]

Tom, you ran the iBench tests on a fast network, right?  If that's the case, I
guess I'm not too surprised that we see little advantage to using the cache.

That fact that performance doesn't vary with changes in cache capacity, suggests
that the time to pull and image out of http's disk cache and decode it is on par
with pulling an already decoded image out of the memory cache.  I find that hard
to believe.  I would also be surprised if the entire iBench test can fit in a
2Meg memory cache.

I'm not really sure what to make of those numbers.

Comment 20

[gordon]

Taking.

Comment 21

[gordon]

This bug is essentially fixed with the landing of the fix for bug 105344.

Marking FIXED.

Comment 22

[benc]

VERIFIED/fixed