86674 - RWlock experiment in pkcs11

Status: RESOLVED WONTFIX

Description

[Kirk Erickson]

Experiment that didn't pan out.  
Larry's NSPR instrumented ilock utilities indicated there
was lock contention in pk11_searchObjectList() on 
slot->ObjectLock.

Changing this lock to be a RWlock, and having readers share
did not help performance though.  Apparently heap contention
is dwarfing this one.  It may be a win later, so here are
the diffs for the record.

Comment 1

[Kirk Erickson]

Created attachment 39142 [details] [diff] [review]
DIFFS (pkcs11.c pkcs11i.h pkcs11u.c)

Comment 2

[Kirk Erickson]

Here are the before and after selfserv run results at the tip
(before this change and after making this change only)

Previous to changes
------------------------------------------------------------
box 010613 tip-delay-full 11.20 13.06 13.98 76.55
box 010613 tip-delay-restart 6.95 7.51 7.94 133.13
box 010614 tip-nodelay-full 14.71 14.71 14.71 67.98 
box 010614 tip-nodelay-restart 7.59 7.94 9.62 125.89
box 010613 tip-nodelayclient-full 12.22 13.37 14.49 74.82
box 010613 tip-nodelayclient-restart 7.26 7.50 7.71 133.24
box 010615 tip-nodelayserver-full 14.87 14.87 14.87 67.25
box 010615 tip-nodelayserver-restart 999.00 499.50 0.00 2.00
<HANGS>

Before changes on dbldog
--------------------------------------------------------------
dbldog 010615 tip-nodelayclient-full 17.57 19.65 21.13 50.89
dbldog 010615 tip-nodelayclient-restart 7.26 7.82 7.98 127.90

With pk11_searchObjectList() the only READ locker
--------------------------------------------------------------
dbldog 010615 tip-nodelayclient-full 18.89 22.04 25.34 45.38
dbldog 010615 tip-nodelayclient-restart 7.17 7.65 8.03 130.66

With pscs11u.c READ Locks too
--------------------------------------------------------------
dbldog 010615 tip-nodelayclient-full 18.79 20.32 22.67 49.21
dbldog 010615 tip-nodelayclient-restart 6.94 7.72 7.90 129.53

Complete dbldog BEFORE stats
--------------------------------------------------------------
dbldog 010613 tip-delay-full 18.79 19.54 23.93 51.19
dbldog 010613 tip-delay-restart 6.45 7.41 7.92 134.98
dbldog 010613 tip-nodelay-full 19.17 20.26 23.26 49.36
dbldog 010613 tip-nodelay-restart 6.71 7.50 8.09 133.27
dbldog 010613 tip-nodelayclient-full 18.27 19.60 21.41 51.01
dbldog 010613 tip-nodelayclient-restart 7.23 7.87 8.33 127.15
dbldog 010613 tip-nodelayserver-full 18.68 20.36 22.58 49.12
dbldog 010613 tip-nodelayserver-restart 6.88 7.64 7.99 130.83

Complete dbldog AFTER stats
--------------------------------------------------------------
dbldog 010615 tip-delay-full 20.05 21.77 29.76 45.94 
dbldog 010615 tip-delay-restart 7.01 7.67 8.10 130.43
dbldog 010615 tip-nodelay-full 18.18 19.81 21.35 50.48
dbldog 010615 tip-nodelay-restart 7.25 7.82 8.12 127.96
dbldog 010615 tip-nodelayclient-full 18.79 20.32 22.67 49.21
dbldog 010615 tip-nodelayclient-restart 6.94 7.72 7.90 129.53 
dbldog 010615 tip-nodelayserver-full 18.42 21.52 27.04 46.48
dbldog 010615 tip-nodelayserver-restart 6.81 7.94 10.95 125.97

Solaris iws-perf BEFORE
--------------------------------------------------------------
iws-perf 010615 tip-delay-full 6.46 6.51 6.66 153.65
iws-perf 010615 tip-delay-restart 7.81 7.86 7.95 127.17
iws-perf 010615 tip-nodelay-full 6.63 6.73 6.80 148.63
iws-perf 010615 tip-nodelay-restart 7.83 7.88 8.13 126.98 
iws-perf 010615 tip-nodelayclient-full 6.47 6.54 6.73 152.98
iws-perf 010615 tip-nodelayclient-restart 7.72 7.84 7.91 127.50
iws-perf 010615 tip-nodelayserver-full 6.67 6.80 6.92 147.13 
iws-perf 010615 tip-nodelayserver-restart 7.73 7.91 7.98 126.50

Solaris iws-perf AFTER
--------------------------------------------------------------
iws-perf 010615 tip-delay-full 6.49 6.57 6.62 152.32
iws-perf 010615 tip-delay-restart 7.81 7.90 7.95 126.50
iws-perf 010615 tip-nodelay-full 6.44 6.56 6.65 152.56 
iws-perf 010615 tip-nodelay-restart 7.78 7.89 7.95 126.74
iws-perf 010615 tip-nodelayclient-full 6.69 6.89 7.11 145.14
iws-perf 010615 tip-nodelayclient-restart 7.79 7.89 7.97 126.74 
iws-perf 010615 tip-nodelayserver-full 6.54 6.62 6.85 151.02
iws-perf 010615 tip-nodelayserver-restart 7.85 7.94 8.17 125.89 
 

Comment 3

[Wan-Teh Chang]

Bob,

Kirk found that using a reader-writer lock doesn't help.
I asked him to post his findings here for future reference.

I guess we will need to try more finer-grained locks or
address the higher-level bottleneck first.

Comment 4

[Christopher Hoess (gone)]

REMIND is deprecated per bug 35839.

Comment 5

[Christopher Hoess (gone)]

.