456380 - CalDAV Scheduling does not work with Kerio Mailserver / support absolute href's in caldav responses

Status: RESOLVED INVALID

(Calendar :: Provider: CalDAV, defect)

Description

[Tony Fairbanks]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
Build Identifier: Thunderbird 2.0.0.16/ Lightning .9RC2

Thanks to some bug fixes earlier, CalDAV calendaring and tasks work great with Kerio 6.5, however the new CalDAV scheduling support and freebusy support do not work.

Reproducible: Always

Steps to Reproduce:
1. Set up a CalDAV account with a Kerio server
2. Enable scheduling
3. Try to schedule an appointment or to use freebusy information
Actual Results:  
Lightning doesn't successfully query for the scheduling information


Selected lines from a verbose debug error console -

On Startup (note it says that the server "generally supports calendar-schedule"):
CalDAV: Status 207 on initial PROPFIND for calendar Kerio
CalDAV: Authentication scheme Basic
CalDAV: recv: <?xml version="1.0" encoding="UTF-8"?><a:multistatus xmlns:a="DAV:" xmlns:d="http://calendarserver.org/ns/" xmlns:c="urn:ietf:params:xml:ns:caldav" xmlns:b="xml:"><a:response><a:href>/calendars/epi.ophth.wisc.edu/fairbanks/Calendar/</a:href><a:propstat><a:status>HTTP/1.1 200 OK</a:status><a:prop><a:resourcetype><a:collection/><c:calendar/></a:resourcetype><a:owner><a:href>https://mail.epi.ophth.wisc.edu:444/caldav/users/epi.ophth.wisc.edu/fairbanks</a:href></a:owner><d:getctag>20080922T144615Z</d:getctag></a:prop></a:propstat></a:response></a:multistatus>
CalDAV: initial ctag 20080922T144615Z for calendar Kerio
CalDAV: send: OPTIONS
CalDAV: DAV header: 1,access-control,calendar-access,calendar-schedule,calendar-proxy
CalDAV: Server generally supports calendar-schedule
CalDAV: send: PROPFIND https://mail.epi.ophth.wisc.edu:444/https://mail.epi.ophth.wisc.edu:444/caldav/users/epi.ophth.wisc.edu/fairbanks/
<D:propfind xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-home-set/>
    <C:calendar-user-address-set/>
    <C:schedule-inbox-URL/>
    <C:schedule-outbox-URL/>
  </D:prop>
</D:propfind>
CalDAV: Failed to report principals namespace

Later when going to schedule:
CalDAV: Server does not support scheduling; freebusy query not possible

Comment 1

[Bruno Browning]

This looks to be a Kerio issue. I'm not sure how up-to-date the test Kerio server I have access to is, but when I wireshark Sunbird startup with it I see Sunbird issuing that PROPFIND for calendar-home-set and friends, and Kerio responding with a 200 status but a content of simply "0". If we can't locate in- and out-boxes we can't really do freebusy or scheduling.

Comment 2

[Tony Fairbanks]

Attachment: Wireshark trace

Comment 3

[Tony Fairbanks]

Attachment: Debug log

Comment 4

[Tony Fairbanks]

I downloaded the newest testing version of Kerio (6.6 RC2) and attached a Wireshark trace and a verbose calendar debug log.
One odd thing I'm seeing in both the debug logs and the wireshark trace is the command Lightning sends:
"CalDAV: send: PROPFIND http://server.domain/http://server.domain/caldav/users/server.domain/fairbanks/". 
Is it supposed to be http://server.domain/server.domain/caldav/... with the servername doubled up like that? It seems to me that it should be just http://server.domain/caldav/users/server.domain/fairbanks/ without the repeating part.

Comment 5

[mark]

I'm getting a very similar problem with Bedework 3.5 rc 2 and Lightning 0.9 (2008091718), Thunderbird 2.0.22. Lightning is running on Windows, Bedework on Linux.

The problem only manifests itself when the proxy is enabled (even though it's not being used for this particular address).  When the proxy is disabled, there is no problem.  

These are the relevant PROPFIND URLs that are requested

With the proxy disabled:

PROPFIND http://192.168.1.216:8080/ucaldav/principals/users/mark/


With the proxy enabled (even though it's bypassed in the proxy.pac file for this particular machine - and I've confirmed that it is actually bypassed) - yes, that repetition really is in the URL, it's not a copying mistake:

PROPFIND http://192.168.1.216:8080/%3Chref%20xmlns=%22DAV:%22%3E/ucaldav/principals/users/mark%3C/href%3E%3Chref%20xmlns=%22DAV:%22%3E/ucaldav/principals/users/mark%3C/href%3E%3Chref%20xmlns=%22DAV:%22%3E/ucaldav/principals/users/mark%3C/href%3E/

Comment 6

[Philipp Kewisch [:Fallen][☁️📆]]

Looks like some sort of parsing problem with the proxy enabled. Tony, do you use a proxy too?

Comment 7

[Tony Fairbanks]

No, I'm not using a proxy. I'm still seeing this issue with the double http:// in the URL request. (I'm using the Inverse Edition Lightning below, but I get the same results on plain .9)

PROPFIND /http://mail.domain.com:81/caldav/users/domain.com/fairbanks/ HTTP/1.1
Host: domain.com:81
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.8.1.22) Gecko/20090605 Lightning/0.9.4-Inverse Thunderbird/2.0.0.22
Accept: text/xml
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: utf-8,*;q=0.1
Keep-Alive: 300
Connection: keep-alive
Content-Length: 223
Content-Type: text/xml; charset=utf-8
Depth: 0
Pragma: no-cache
Cache-Control: no-cache

<D:propfind xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-home-set/>
    <C:calendar-user-address-set/>
    <C:schedule-inbox-URL/>
    <C:schedule-outbox-URL/>
  </D:prop>
</D:propfind>

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Type: application/octet-stream
Date: Mon, 10 Aug 2009 16:49:11 GMT
Keep-Alive: timeout=15, max=97
Server: Kerio MailServer 6.7.0 patch 1
Transfer-Encoding: chunked

0

Comment 8

[Tony Fairbanks]

This is the output I get if I telnet into the server with a corrected PROPFIND URL. Kerio definitely seems to return some calendaring-related info.

PROPFIND /calendars/domain.com/fairbanks/Calendar/ HTTP/1.1
Host: mail.domain.com:81
Accept: text/xml
Accept-Language: en-us,en;q=0.5
Accept-Charset: utf-8,*;q=0.1
Keep-Alive: 300
Connection: keep-alive
Content-Length: 190
Content-Type: text/xml; charset=utf-8
Depth: 0
Pragma: no-cache, no-cache
Cache-Control: no-cache, no-cache

<D:propfind xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-home-set/>
    <C:calendar-user-address-set/>
    <C:schedule-inbox-URL/>
    <C:schedule-outbox-URL/>
  </D:prop>
</D:propfind>

HTTP/1.1 207 Multi status
Connection: Keep-Alive
Content-Type: text/xml; charset="utf-8"
Date: Mon, 10 Aug 2009 17:54:58 GMT
DAV: 1, access-control, calendar-access
Keep-Alive: timeout=900, max=99
Server: Kerio MailServer 6.7.0 patch 1
Transfer-Encoding: chunked

b46
<?xml version="1.0" encoding="UTF-8"?><a:multistatus xmlns:a="DAV:" xmlns:e="http://schemas.microsoft.com/exchange/" xmlns:f="http://schemas.microsoft.com/mapi/proptag/" xmlns:g="http://schemas.microsoft.com/repl/" xmlns:d="urn:ietf:params:xml:ns:caldav" xmlns:h="urn:schemas:httpmail:" xmlns:c="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" xmlns:b="xml:"><a:response><a:href>/calendars/domain.com/fairbanks/Calendar/</a:href><a:propstat><a:status>HTTP/1.1 200 OK</a:status><a:prop><a:childcount c:dt="int">166</a:childcount><a:contentclass>urn:content-classes:calendarfolder</a:contentclass><a:displayname>Calendar</a:displayname><a:getetag>d9472d4968624d0aaa39a18a067df82f00000000</a:getetag><a:hassubs c:dt="boolean">0</a:hassubs><a:href>/calendars/domain.com/fairbanks/Calendar/</a:href><a:id>d9472d49-6862-4d0a-aa39-a18a067df82f</a:id><a:iscollection c:dt="boolean">1</a:iscollection><a:isfolder c:dt="boolean">1</a:isfolder><a:ishidden c:dt="boolean">0</a:ishidden><a:href>/calendars/domain.com/fairbanks/</a:href><a:resourcetype><a:collection/><d:calendar/></a:resourcetype><a:uid>d9472d49-6862-4d0a-aa39-a18a067df82f</a:uid><e:foldersize c:dt="int">176468</e:foldersize><e:outlookfolderclass>IPF.Appointment</e:outlookfolderclass><e:permanenturl>d9472d49-6862-4d0a-aa39-a18a067df82f</e:permanenturl><f:x0ff40003 c:dt="int">63</f:x0ff40003><f:x360a000b c:dt="boolean">0</f:x360a000b><f:x3613001f>IPF.Appointment</f:x3613001f><f:x36e41102 c:dt="mv.bin.base64"><b:v/><b:v>AAAAAALfudSc85NIrXcdrkydBtcHAO9xA46wz09Ci0YncpUYLYQAAAAAAIkAAO9xA46wz09Ci0YncpUYLYQAAAAAMcsAAA==</b:v><b:v>AAAAABpEc5CqZhHNm8gAqgAvxFoJAPGDLst8ZGpJr88x0FKF2KUAAAAAAAcAANqVO5HPIh9Cp1JnfOggOaAAAAAAJDwAAA==</b:v><b:v>AAAAAALfudSc85NIrXcdrkydBtcBAO9xA46wz09Ci0YncpUYLYQAAAAAAIkAAA==</b:v></f:x36e41102><g:resourcetag>rt:d9472d4968624d0aaa39a18a067df82fd9472d4968624d0aaa39a18a067df82f00000000</g:resourcetag><h:calendar>/calendars/domain.com/fairbanks/Calendar</h:calendar><h:contacts>/calendars/domain.com/fairbanks/Contacts</h:contacts><h:deleteditems>/calendars/domain.com/fairbanks/Deleted%20Items</h:deleteditems><h:drafts>/calendars/domain.com/fairbanks/Drafts</h:drafts><h:inbox>/calendars/domain.com/fairbanks/INBOX</h:inbox><h:journal>/calendars/domain.com/fairbanks/Journal</h:journal><h:junkemail>/calendars/domain.com/fairbanks/Junk%20E-mail</h:junkemail><h:notes>/calendars/domain.com/fairbanks/Notes</h:notes><h:outbox>/calendars/domain.com/fairbanks/Outbox</h:outbox><h:sendmsg>/calendars/domain.com/fairbanks/%23%23DavMailSubmissionURI%23%23</h:sendmsg><h:sentitems>/calendars/domain.com/fairbanks/Sent%20Items</h:sentitems><h:tasks>/calendars/domain.com/fairbanks/Tasks</h:tasks><h:unreadcount c:dt="int">0</h:unreadcount></a:prop></a:propstat></a:response></a:multistatus>
0

HTTP/1.1 501 Not implemented
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>501 Method not implemented 'outbox-URL/&amp;gt;'</title>
</head><body>
<h1>Method not implemented 'outbox-URL/&amp;gt;'</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>
Connection closed by foreign host.

Comment 9

[Philipp Kewisch [:Fallen][☁️📆]]

it seems we expect a relative path in the owner/href of this response.

<?xml version="1.0" encoding="UTF-8"?>
  <a:multistatus xmlns:a="DAV:" xmlns:d="http://calendarserver.org/ns/" xmlns:c="urn:ietf:params:xml:ns:caldav" xmlns:b="xml:">
    <a:response>
      <a:href>/calendars/server.domain/fairbanks/Calendar/</a:href>
      <a:propstat>
        <a:status>HTTP/1.1 200 OK</a:status>
        <a:prop>
          <a:resourcetype><a:collection/><c:calendar/></a:resourcetype>
          <a:owner>
            <a:href>http://server.domain/caldav/users/server.domain/fairbanks</a:href>
          </a:owner>
          <d:getctag>20080930T164334Z</d:getctag>
        </a:prop>
      </a:propstat>
    </a:response>
</a:multistatus>


Confirming, we should be more tolerant here and strip the server path.

Comment 10

[Tony Fairbanks]

Ok, while I'm not familiar enough with the code to figure out how to correctly strip the server part out, I was able to do an awful, awful hack to get rid of it for testing purposes. Once I got that done I ran into another problem, which I was actually able to diagnose and fix (again, probably not the right way). Kerio is returning inbox and outbox paths in the following form:
<c:schedule-inbox-URL>/full-calendars/domain.com/fairbanks/INBOX</c:schedule-inbox-URL>
<c:schedule-outbox-URL>/full-calendars/domain.com/fairbanks/Sent%20Items</c:schedule-outbox-URL>

However, from the code in calDavCalendar.js, it looks like it's expecting it to be wrapped in <a:href> tags:
var ibPath = response..C::["schedule-inbox-URL"]..D::href[0];

Which causes the parsing to fail, and then makes ibPath null, and all sorts of bad things happen. I changed it to:
var ibPath = response..C::["schedule-inbox-URL"].toString();

And it works (after making a similar fix to the outbox as well). Lightning now seems to grab scheduling data, and it fills in the calendar free/busy schedule thing under invite attendees now.

Comment 11

[Philipp Kewisch [:Fallen][☁️📆]]

I believe wrapping in a href tag is required per spec, so that might be a Kerio bug. Maybe we can be more tolerant there, although I'd suggest to contact Kerio on that issue (possibly read the caldav rfc's beforehand).

Comment 12

[Tony Fairbanks]

Here's the section from the scheduling rfc draft:

Name:  schedule-inbox-URL
   Namespace:  urn:ietf:params:xml:ns:caldav
..snip..
   Definition:
      <!ELEMENT schedule-inbox-URL (DAV:href)>

Assuming I'm interpreting that correctly, I believe you're right about the href tags.

Comment 13

[Tony Fairbanks]

Just an FYI -H ere are some excerpts from the irritating experience I had with Kerio support:

Me: "I've been working on getting Mozilla Thunderbird with the Lightning plug-in to handle scheduling/freebusy using Kerio Mailserver. I've run across a few quirks in the implementation of CalDAV under Kerio, which are documented at the Bugzilla report here: https://bugzilla.mozilla.org/show_bug.cgi?id=456380
There are two problems that currently prevent Lightning from working with Kerio:
1) The owner URL has the full server path, when Lightning expects just the relative path
2) Kerio doesn't wrap the inbox and outbox URLs in an HREF tag

I hacked together a version of Lightning that works around these two problems and was then successfully able to schedule meetings using freebusy support. While I'm not certain about #1 above, #2 seems to be in violation of the RFC spec for scheduling in CalDAV support. Getting these two problems fixed would be a great benefit, as I'm currently holding off on deploying calendaring to more users until we can do scheduling."

Support: "A member of the development team wrote back to me. He states that Free/busy is not supported in Lightning at this time and we do not do active development support for Sunbird/Lightning. As as he knows the free/busy in older Lightning version was supposed to work only with Sun Calendaring Server.

He mentioned a suggestion we have on file for improving our support for Sunbird/Lightning. I have added your vote for this to that suggestion.

Please note we do not have a time frame on if or when this improved support may be available."

They then closed my bug report. I guess they don't seem too concerned about making it work.

Comment 14

[Philipp Kewisch [:Fallen][☁️📆]]

Interesting, I wonder who they contacted. We do support caldav freebusy, and I think we have been doing that for quite some time now.

What we don't support per default is caldav scheduling, this needs to be enabled in the prefs. I'd like to see at least the wrapping fixed on kerio's side, we can fix the absolute url issue on our side.

Comment 15

[Bruno Browning]

Sorry, Tony, I've been away a bit and am just noticing that you've already reported to Kerio the issues with HREF in the in/outbox responses that are being worked around in the patch in bug #518610. 

FWIW, we've supported caldav-sched style freebusy since 0.8, and with the proposed patch in place it seems to work with Kerio: we get a reasonable-looking response back from the server at least (I have not tested that it's correct, just eyeballed that it's well-formed.). We don't yet support fburl-style freebusy.

Comment 16

[Tony Fairbanks]

It looks like Kerio broke something again in version 7.0.2. Freebusy lookups do not work anymore. Everything else seems fine though.

Comment 17

[Tony Fairbanks]

Attachment: Verbose debug log

Comment 18

[Tony Fairbanks]

Attachment: Wireshark capture, shows a little more detail

Comment 19

[simon.j]

This bug is present on the Ma platform also, running the Kerio Mailserver on Mac OSX 10.5.8 with a variety of clients OX10.4, 10.5, 10.6, XP & Vista and all have the same issue with Freebusy lookups.

Comment 20

[Philippp]

Bug is still present with Kerio Connect 7.1.3 (tested Kerio on Win Srv 2003, thunderbird+Lighning on WinXP 32)  :-(

Comment 21

[Philippp]

I just did a very quick try with the new Kerio Connect 7.2 on OS X (server and client), and I noticed 2 promising things:
1.) In the "Integration with Mac" Webpage (in Kerio Webmail) Sunbird/Lightning is listed (as "Beta").
2.) During a very quick test, Thunderbird successfully displayed the free-busy information in the "invite" dialog! :-))
(I did not get the meeting invitation, but that is probably because I do not have a proper mail setup here on the test machine)

Comment 22

[Philippp]

Update: The AntiVirus was not running, which blocked mail delivery. Invitations seem to work now!

Comment 24

[Philipp Kewisch [:Fallen][☁️📆]]

From skimming the past comments this seems to be a server issue, therefore I am going to close this bug. If you have a suggestion what we should do in the client and this is still an issue with Lightning 3.3.1, please reopen the bug.