323064 - Honor scope when chasing referrals

Status: RESOLVED FIXED

(Directory :: LDAP C SDK, defect)

Description

[Rich Megginson]

> Well, I don't know if it has to be considered as a bug (failure ?
> feature request ?). Because the comment in the code does say :
> 
> /*
>  * XXXmcs: we don't support scope or filters in search referrals yet,
>  * so if either were present we return an error which is probably
>  * better than just ignoring the extra info.
> */
> 
> So Mozilla behaves correctly regarding to this limitation. It does
> supports referrals, *except* with LDAP URLs which contain a scope or a
> filter.
> 
> If you think this a truly a bug, please do so.

I think it is a bug, or at least a lapse of LDAPv3 support - see http://www.ietf.org/rfc/rfc2251.txt sections 4.1.11 and 4.5.3.
"Some servers (e.g. participating in distributed indexing) may provide a different filter in a referral for a search operation.  If the filter part of the URL is present in an LDAPURL, the client MUST use this filter in its next request to progress this search, and if it is not present the client MUST use the same filter as it used for that search.  Other aspects of the new request may be the same or different as the request which generated the referral."

So it seems as though we should honor the new filter.  It doesn't say whether or not the new scope should be honored.  I suppose there are
several ways we could handle this:
1) Leave scope handling as is - return an error
2) Ignore the scope and just use the same scope as in the original search request
3) Honor the scope
I prefer 3 - if a new scope is present, it's there for a reason and should be respected.  We could add another LDAP option for this so you would be able to choose the 1, 2, or 3 behavior.

Do we also need to honor attribute lists or bindname in referal/reference URLs?

Comment 1

[Mark Smith [:mcs]]

I'd call it incomplete LDAPv3 support, aka a bug.  The LDAPbis drafts provide much better clarity in these areas; see section 4.1.10 of:

   http://ietf.org/internet-drafts/draft-ietf-ldapbis-protocol-32.txt

(the LDAPBis documents are all in or have made it past IETF-wide last call, so they will be RFCs soon replacing 2251 et al).  So I think the simple answer is:

    a) use the new scope if present
and b) use the new filter if present

The reason I added the XXXmcs comment and did not implement this years ago was because the referral handling code will need to do more parsing and reassembly of requests for this to work (and that will be somewhat messy).  Is someone asking for this to support a real deployment?  How urgent is this bug?

Comment 2

[snotling]

(In reply to comment #1)
> I'd call it incomplete LDAPv3 support, aka a bug.  The LDAPbis drafts provide
> much better clarity in these areas; see section 4.1.10 of:
> 
>    http://ietf.org/internet-drafts/draft-ietf-ldapbis-protocol-32.txt
> 
> (the LDAPBis documents are all in or have made it past IETF-wide last call, so
> they will be RFCs soon replacing 2251 et al).  So I think the simple answer is:
> 
>     a) use the new scope if present
> and b) use the new filter if present
>
> The reason I added the XXXmcs comment and did not implement this years ago was
> because the referral handling code will need to do more parsing and reassembly
> of requests for this to work (and that will be somewhat messy).

I also discovered that if the referral URL doesn't include the port number, Mozilla uses 0, instead of 389... Why isn't the referral URL parsed the same way, by the same functions, as the "original" URL ?

> Is someone
> asking for this to support a real deployment?  How urgent is this bug?

I find this bug quite annoying because, for example, it makes the association of Mozilla and OpenLDAP (and other servers ?) impossible for referrals.

Maybe the first step is to modify the test in request.c to make it check *correctly* the scope and filter : i.e. "??" should be mean the same to Mozilla as no scope and filter.
Then, if adding scope and filter support involves too much work, you should evaluate if :
 a/ Mozilla continues to return an error in case a filter/scope is present
 b/ Mozilla ignores the filter/scope and uses the original or no filter/scope

In any case, I think this behavior should be well documented because it took me days to dig through the code :-)

Comment 3

[snotling]

Furthermore, in RFC 3296, at paragraph 5.3., one can read :

   In cases where the URI to be returned is a LDAP URL, the server MUST
   provide an explicit scope specifier from the LDAP URL prior to
   returning it.

   ...

   Example: If the client issues a subtree search in which the base
      object is "OU=Roles,O=MNN,C=WW", the server will return

         SearchResultDone (referral) {
             ldap://hostd/OU=Roles,O=MNN,C=WW??sub
         }

So, IIUC, Mozilla reference chasing will fail with all RFC-compliant LDAP servers ?

Comment 4

[Mark Smith [:mcs]]

(In reply to comment #2)
> I also discovered that if the referral URL doesn't include the port number,
> Mozilla uses 0, instead of 389... Why isn't the referral URL parsed the same
> way, by the same functions, as the "original" URL ?

The same parsing function is used, but there is special handling for the host and port when processing references and referrals.  I am not 100% sure, but I think the bug where port 0 was used my mistake was fixed on the trunk a while ago.  What LDAP C SDK version or client application are you testing with?

Comment 5

[Mark Smith [:mcs]]

(In reply to comment #3)
>
> So, IIUC, Mozilla reference chasing will fail with all RFC-compliant LDAP
> servers?

Yes, I think you are correct.  The code needs to be updated (older specifications did not require that a scope be included, but even RFC 3296 is kind of old now).  Do you know of a publically accessible OpenLDAP server I could test this against once I have a proposed fix?

Comment 6

[snotling]

(In reply to comment #4)
> I am not 100% sure, but I
> think the bug where port 0 was used my mistake was fixed on the trunk a while
> ago.  What LDAP C SDK version or client application are you testing with?
> 

I have tried Thunderbird 1.0.7 and also with a recent Thunderbird checked out and build from MOZILLA_1_8_BRANCH : the "port 0" bug is still there :/

Comment 7

[snotling]

(In reply to comment #5)
> Do you know of a publically accessible OpenLDAP server I
> could test this against once I have a proposed fix?

No, I used my own pair of OpenLDAP servers, but it's quite easy to set up.
My root LDAP server has a base DN of "dc=bar" and the "child" server has "o=foo,dc=bar". And both contains different user entries.
My LDIF for the referral looks like this (to be inserted in the root server) :

  dn: o=foo,dc=bar
  o: foo
  ref: ldap://ldap2:389/o=foo,dc=bar
  objectclass: extensibleObject
  objectclass: referral

Hope it may helps...

Comment 8

[snotling]

(In reply to comment #6)
> I have tried Thunderbird 1.0.7 and also with a recent Thunderbird checked out
> and build from MOZILLA_1_8_BRANCH : the "port 0" bug is still there :/

I've just tested, the official Thunderbird 1.5 still has this bug.

Comment 9

[snotling]

Back to port 0 bug :

In file directory/c-sdk/ldap/libraries/libldap/request.c, at line 1018, there is a test :

  if ( ludp->lud_port == 0 && ludp->lud_host == NULL ) {
    // using port from original request
  } else {
    // using port as specified on new request
  }

In my case, with a LDAP entry of 'ref: ldap://server.site.local/o=foo,dc=bar', ludp->lud_host is NOT NULL but has the value 'server.site.local'.
Thus, Thunderbird wants to use the port specified in the new request, which port is not even present in the URL => port 0 is used.

Could you solve this bug ? BTW, I would consider this bug with more than a "minor" severity as it makes LDAP referrals totally unusable in most LDAP setups...

Comment 10

[Mark Smith [:mcs]]

The port 0 bug was fixed a long time ago on the LDAP C SDK trunk.  See bug 152852.  There is a movement afoot to switch TBird and Seamonkey to use the latest LDAP code, which is long overdue.  See bug 325625.  If that does not happen soon for some reason, we can land the patch in bug 152852 separately.

Comment 11

[Rich Megginson]

Can we mark this as fixed now that the tbird/seamonkey team has taken 5.17 as their branch point?

Comment 12

[Mark Smith [:mcs]]

The port 0 issue has been fixed but the general problem of not honoring scope and filter has not (see comment #5).

Comment 13

[Rich Megginson]

Attachment: diffs for fix

This diff implements support for using the scope from the referral, which at least puts the mozldap sdk on par with openldap (which does not support refererence filters AFAICT by looking at their code).

Comment 14

[Mark Smith [:mcs]]

Comment on attachment 231474 [details]
diffs for fix

This looks good to me.  Great work!

One small point:  I would prefer to add an is_reference parameter to chase_one_referral() rather than rely on the desc string (in case someone changes the desc thinking it is only used for logging).

Comment 15

[Rich Megginson]

Attachment: new diffs to address comments from mcs

Mark, make sure these diffs look ok.  Dan, this was originally filed against the address book functionality in tbird/seamonkey.

Comment 16

[Mark Smith [:mcs]]

Comment on attachment 231496 [details]
new diffs to address comments from mcs

OK.

Comment 17

[Rich Megginson]

Checking in request.c;
/cvsroot/mozilla/directory/c-sdk/ldap/libraries/libldap/request.c,v  <--  request.c
new revision: 5.6; previous revision: 5.5
done

I've renamed the summary of this bug to just honor scope.  I've opened bug 346857 for the honor filter part.