294153 - (LdapMemcache) Regarding Ldap memory cache

Status: RESOLVED INCOMPLETE

(Directory Graveyard :: LDAP C SDK, enhancement)

Description

[jagadish]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
Build Identifier: Mozilla/2.0

I am using Ldap memcache API's to cache the search results of (AD) nested group 
search. This is working fine for single domain case, say when configured single 
x.x.x.x ntlm GC server. But in cross domain case(Enabled referral) the Ldap 
referral queries should be get cached , but its not caching these Ldap referral 
search results, intsead it will contact the server again to get the referral 
result. My question is whether the designed Ldap memcache supports caching 
referral search results or not. I saw one comment in Ldap library source
(memcache.c) To do: referral and reference ? does this mean that memcache is 
not yet implemented to cache referral search results ? If not implemented , 
could you please give me a pointers to give a support for cahing the Ldap 
referrals. And also how we can defferentiate the search results(Referral 
results, other group search results)? 


Reproducible: Always

Steps to Reproduce:
1.configure Ntlm server as well as GC server with (Domain controller)host addr 
x.x.x.x say Domain1.
2. Enable the Ldap referral and ntlm server 
3.Configure proxy in the browser, Send a request from the browser will get pop-
up mesg asking user credentials now give the user info and domain name which 
exists in different domain say doamin2( not in domain1) Authentication works 
fine because of referral but when request sent for second time again Proxy will 
contact the server to get the referral info, this should be cached.

Actual Results:  
In-memory cache not working in case of cross-domain(referrals).


Expected Results:  
The referral search results should be cached.

Comment 1

[Mark Smith [:mcs]]

You are correct in your discovery that caching is not implemented for referrals
and search references.  In libldap/result.c:nsldapi_result_nolock() the
ldap_memcache_append() call is made only for LDAP search entries and result
responses.  To add support for referrals and references, the code would have to
be modified to associate response messages returned from referrals and
references with the original request.

Comment 2

[jagadish]

Hi,

Thanks for the response.

Yes i saw that code in libldap/result.c  its quite readable.
I am not sure of how we differentiate the actual search results 
and search results from referral. It would be greatful if you 
could provide some patch to associate response messages returned from referrals 
and references with the original request or some pointer on how to implement 
this in order to support this feature. 
It would be helpful if you respond quickly.

Thanks,
Jagadeesh





(In reply to comment #1)
> You are correct in your discovery that caching is not implemented for 
referrals
> and search references.  In libldap/result.c:nsldapi_result_nolock() the
> ldap_memcache_append() call is made only for LDAP search entries and result
> responses.  To add support for referrals and references, the code would have 
to
> be modified to associate response messages returned from referrals and
> references with the original request.

(In reply to comment #1)
> You are correct in your discovery that caching is not implemented for 
referrals
> and search references.  In libldap/result.c:nsldapi_result_nolock() the
> ldap_memcache_append() call is made only for LDAP search entries and result
> responses.  To add support for referrals and references, the code would have 
to
> be modified to associate response messages returned from referrals and
> references with the original request.

Comment 3

[Mark Smith [:mcs]]

Unfortunately I do not have time right now to work on a patch for this.  To
differentiate results returned in response to a referral, you can look at fields
such as lr_parent inside the LDAPRequest structure to determine if a request was
generated in response to a referral or if it is a "top level" request (which
will have lr_parent == NULL).  Start by looking at how and where lr_parent is used.

Comment 4

[jagadish]

Thanks much for your response.
I did some code change to cache the seach references. I am not sure of the 
change. Plz let me know your thaughts on this. Code change is as below.

 #define NSLDAPI_IS_SEARCH_RESULT( code ) \
-       ((code) == LDAP_RES_SEARCH_RESULT)
+       ((code) == LDAP_RES_SEARCH_RESULT || (code) == 
LDAP_RES_SEARCH_REFERENCE)

I tried exploring the code to see how the lr_parent has been used.
I could see the memcache_append is also called for SERACH_REFERENCES 
but if sent same request again the key generated will be fresh(new) and
search fails when checking the response from cache. 
Here i could see every time the new key is generated and search fails.
My Question is how can we generate the key which is associated with already 
cached response in case of referrals.

I need one more clarification.
for eaxample we send request for the first time and get a referrals as response
the client again contacts the referred server by using chase_one_referral().
Is that the Search request sent for the first time should be used to associate 
with referral response or search request that sent to the referred server 
should be?
It would be greatful if you could provide some logic that associate response 
messages returned from referrals and  references with the original request.
Or atleat give me the pointers how i could proceed to achive this.

Comment 5

[jagadish]

I Did some code changes in result.c to associate the search result with 
original msgid, before calling ldap_memcache_append as shown below.

  if ( ld->ld_memcache != NULL && NSLDAPI_SEARCH_RELATED_RESULT( rc ) &&
             !((*result)->lm_fromcache )) {
			
+               if( (lr = nsldapi_find_request_by_msgid( ld, (*result)-
>lm_msgid ))  != NULL){
+                       if (lr->lr_parent != NULL)
+                              msgid_cache = lr->lr_origid;
+                       else
+                               msgid_cache = (*result)->lm_msgid;
+               }

+               }
+                    ldap_memcache_append( ld, msgid_cache,
+                    (all || NSLDAPI_IS_SEARCH_RESULT( rc )), *result );
        }


But this also doeen't seems to be working. Please let me your thoughts on this.
Also could you please give me some pointers how to proceed with this.

Comment 6

[jagadish]

My observation with code changes that I sent is, always the return value from 
nsldapi_find_request_by_msgid() is NULL and again the same result msgid 
(*result)->lm_msgid) will be used in caching the result.

Comment 7

[Mark Smith [:mcs]]

Regarding comment #5, is the problem that nsldapi_find_request_by_msgid() only
finds "top level" requests (and not those generated by the automatic referral
and reference chasing code)?  I think you are on the right track, although
fixing this bug will require some detailed work.

Comment 8

[Rich Megginson]

Any update on this?