382219 - PUT -> PROPFIND should update the href returned from the CalDAV server

Status: RESOLVED INVALID

(Calendar :: Provider: CalDAV, defect)

Description

[Jakub Klos]

User-Agent:       Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;  Embedded Web Browser from: http://bsalsa.com/; .NET CLR 2.0.50727)
Build Identifier: 0.5pre

After a successful PUT PROPFIND is issued. The server returns a slightly different href (folder specification in the URL) and the new object should be updated with this URL/href. Otherwise modifying the item or deleting the item with the refresh from the server will not work.

This is a specific behavior of Groupware servers that support folders.

Reproducible: Always

Steps to Reproduce:
1. Add new item (PUT + PROPFIND is iussed)
2. Delete the item - will not work
3.
Actual Results:  
As you can see below the DELETE does not specify the proper href path returned with the PROPFIND result. 

2007-05-28 11:17:36 sid:6d28e5c4ee6b5d667bded2a193aeb12a <<< PUT /webdav/uuid1180343856031.ics
BEGIN:VCALENDAR
PRODID:-//Mozilla Calendar//NONSGML Sunbird//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:/mozilla.org/20070129_1/Africa/Ceuta
X-LIC-LOCATION:Africa/Ceuta
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20070528T091736Z
LAST-MODIFIED:20070528T091736Z
DTSTAMP:20070528T091736Z
UID:uuid1180343856031
SUMMARY:New Event
DTSTART;TZID=/mozilla.org/20070129_1/Africa/Ceuta:20070607T093000
DTEND;TZID=/mozilla.org/20070129_1/Africa/Ceuta:20070607T141500
X-MOZ-LOCATIONPATH:uuid1180343856031.ics
END:VEVENT
END:VCALENDAR


2007-05-28 11:17:36 sid:6d28e5c4ee6b5d667bded2a193aeb12a >>> HTTP/1.1 201 Created

2007-05-28 11:17:36 sid:6d28e5c4ee6b5d667bded2a193aeb12a <<< PROPFIND /webdav/uuid1180343856031.ics
<?xml version="1.0"?>
<D:propfind xmlns:D="DAV:"><D:prop><D:getetag/></D:prop></D:propfind>

2007-05-28 11:17:36 sid:6d28e5c4ee6b5d667bded2a193aeb12a >>> HTTP/1.1 207 Multi-Status [Content-Type: text/xml; charset=utf-8]
<?xml version="1.0" encoding="UTF-8"?>
<D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
<D:response>
<D:href>http://localhost/webdav/Events/uuid1180343856031.ics</D:href>
<D:propstat>
<D:prop>
<D:getetag>uuid1180343856031.ics:0</D:getetag>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>

2007-05-28 11:17:40 sid:6d28e5c4ee6b5d667bded2a193aeb12a <<< PROPFIND /webdav/uuid1180343856031.ics
<?xml version="1.0"?>
<D:propfind xmlns:D="DAV:"><D:prop><D:getetag/></D:prop></D:propfind>

2007-05-28 11:17:40 sid:6d28e5c4ee6b5d667bded2a193aeb12a >>> HTTP/1.1 207 Multi-Status [Content-Type: text/xml; charset=utf-8]
<?xml version="1.0" encoding="UTF-8"?>
<D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
<D:response>
<D:href>http://localhost/webdav/Events/uuid1180343856031.ics</D:href>
<D:propstat>
<D:prop>
<D:getetag>uuid1180343856031.ics:0</D:getetag>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>

2007-05-28 11:17:40 sid:6d28e5c4ee6b5d667bded2a193aeb12a <<< DELETE /webdav/uuid1180343856031.ics


2007-05-28 11:17:40 sid:6d28e5c4ee6b5d667bded2a193aeb12a >>> HTTP/1.1 409 Conflict


Expected Results:  
DELETE /webdav/Events/uuid1180343856031.ics
Should have been issued

Comment 1

[Bruno Browning]

I think the proper CalDAVish way to solve this is going to be to do the PUT and then query the server for an item with the uid of the item just PUT, as described in paragraph 7.8.6 of the CalDAV RFC. Not all of the CalDAV servers currently supported by Sunbird/Lightning have implemented query-by-uid yet, so at present we can't implement that without breaking interop. I expect this will become possible in the 0.7 timeframe.

Comment 2

[Jakub Klos]

I don't think it's the best approach actually. It requires sort of a search. Although the PROPFIND method I suggest is not in the draft it sounds the most effective and as I see it you are already doing this. So why not just updating the href of the item in the cache if the PROPFIND response href is different after the new PUT? Please think about it.
Thank you

Comment 3

[Helge Hess]

Uhm, Sunbird does
a) the PUT with an if-none-match: * to create an item
b) then does a PROPFIND to query the etag of the item?

What if the content changed in the meantime? The PROPFIND just returns the new etag, but not the new content and Sunbird would associate the deprecated content with the wrong etag?

I think the proper thing to do after a PUT is either:
1) a GET on the just created URL, then update the local cache content and etag with the results
2) or the query-by-uid, which is actually better in case the server moved the item (very common)
  [though, please, support the 'location' header in PUT! Its trivial, doesn't hurt and is VERY useful ;-)]

Jakub:
I don't get the rational behind that PROPFIND on /abc/1.ics which returns a result with /abc/events/1.ics. If you mount the CalDAV collection into the parent namespace, why wouldn't the DELETE also get mapped to the events collection? It seems weird that some HTTP methods (PUT,PROPFIND) work on mapped URLs and others don't (DELETE,GET?).

Comment 4

[Bruno Browning]

(In reply to comment #3)
> Uhm, Sunbird does
> a) the PUT with an if-none-match: * to create an item
> b) then does a PROPFIND to query the etag of the item?

Not exactly. Currently Sunbird does the PUT *without* an if-none-match due to limitations in the webdav stack (limitations which I hope to tackle this summer). It then does a PROPFIND in order get the etag. It obviously would be preferable to do this in one go, thus bug 373370.


> I think the proper thing to do after a PUT is either:
> 1) a GET on the just created URL, then update the local cache content and etag
> with the results
> 2) or the query-by-uid, which is actually better in case the server moved the
> item (very common)
>   [though, please, support the 'location' header in PUT! Its trivial, doesn't
> hurt and is VERY useful ;-)]

Supporting the location header is likely something we'll do once it *is* trivial, meaning after the aforementioned webdav fixes. I agree that the query-by-uid is a good way to go.


>  It seems weird that some HTTP methods (PUT,PROPFIND) work on mapped
> URLs and others don't (DELETE,GET?).
> 

Well said, strongly agree.

Comment 5

[Jakub Klos]

The CalDAV server is our own solution and so is the Groupware server it connects to. We used to support the Location header but realized that no caldav/groupdav client in fact supports it. So I decided to update both to support the UID from the caldav application and not create our own one internally. This fixes the location issue.

However the sunbird and lightning clients they do not think hiearchic and do not consider that there might be multiple collections on the caldav server. Our gw solution is purely folder based. We have URLs like
http://.../webdav/Events
http://.../webdav/Tasks
http://.../webdav/Somefolder/anotherfolder
http://.../webdav/Contacts

The fact that we support PUT to the very root of the webdav folder is simply something so it would work with Mozilla. In such case (only the webdav root folder is given) we inspect the type of the inserted item and put it in the default folder for that type of versit object. As you know the DELETE method does not send the content so we cannot investigate the versit object type.

That's just for the clarification.

Back to the problem above. Any solution is fine as long as there is a solution and we can adapt. What I thought was that if PROPFIND is issued why not simply update the etag and the href. Elegant and working right away.

Comment 6

[Bruno Browning]

I kind of think that the Mozilla CalDAV provider needs to be a generic CalDAV provider and not rely on behaviors particular to any given server. Since there may be in future servers which relocate files on PUT without the unusual PROPFIND behavior proposed above, I think the search-by-uid approach is preferable.

Comment 7

[Andrew McMillan]

I'm not sure if there are any CalDAV servers other than RSCDS which don't currently support PROPFIND with a TEXT-MATCH on UID, but this kind of search will be supported in the next release.

Unfortunately the specification in RFC4791, section 9.7.5 says that this is a *substring* match, so it will not be as efficient as it could be if the specification allowed for an anchored or exact match.

I've used the following XML in my regression tests, but if someone could send me an actual example of Mozilla attempting this I can add a more targetted regression test as well.

<?xml version="1.0" encoding="utf-8" ?> 
<calendar-query xmlns="urn:ietf:params:xml:ns:caldav">
  <D:prop xmlns:D="DAV:">
    <D:getetag/>
    <calendar-data/>
  </D:prop>
  <filter>
    <comp-filter name="VCALENDAR">
      <comp-filter name="VEVENT">
        <prop-filter name="UID">
          <text-match collation="i;octet">20061101T073000Z-10468-1000-1-7@ubu</text-match>
        </prop-filter>
      </comp-filter>
    </comp-filter>
  </filter>
</calendar-query>

Comment 8

[Bruno Browning]

Jakub, Helge implied in bug 381573 comment 5 that this folder hierarchy, at least as seen by the Mozilla CalDAV provider, will be getting flattened out. That addresses the underlying cause (inability to modify/delete) of the current bug. So I'm wondering - do we need it anymore?

Comment 9

[Helge Hess]

(In reply to comment #8)
> Jakub, Helge implied in bug 381573 comment 5 that this folder hierarchy, at
> least as seen by the Mozilla CalDAV provider, will be getting flattened out.

Not sure I implied that :-)

Just for completeness, bug 381573 is an issue about moving items inside a single collection, not across collections. The background issue is that it is preferable for RDBMS based servers to assign a server maintained relative URL on creation (based on the primary key of the just created record). So a PUT /folder/xyz would return /folder/123 in the location header.

I think the user visual effect of moving items across collections in a CalDAV client, is that the new item would end up in a different calendar (eg you created an item in the / main calendar, it will show up in the /Tasks calendar). I think its pretty much a requirement that there is a 1:1 match between a CalDAV collection and a Sunbird calendar?

Jakub: I think the proper CalDAV solution for your issue is not to expose / as a calendar collection, but /Tasks and /Events. Then properly set "supported-calendar-component-set" to either VTODO or VEVENT.
Which of course raises the question whether Sunbird honours "supported-calendar-component-set"? *This* might be a new bug.

My suggestion: close this bug and create a new one for Sunbird to support task-only and event-only calendars.

Comment 10

[Bruno Browning]

Attachment: query/refetch item after PUT

I think this is pretty much what we need to do once server-side support is wider. Not requesting review at this time, but I'm hoping that someone who has access to OGo can test this. It should address the underlying problem (incorrect cached href) though not quite in the manner OP suggested. It should not be necessary to build Sunbird to test this: patching components/calDavCalendar.js in a recent Sunbird nightly should do it.

Comment 11

[Martin Schröder [:mschroeder]]

Confirming based on attached patch and some comments. ;)

Comment 12

[Mark Pavlichuk]

(In reply to comment #10)
> I think this is pretty much what we need to do once server-side support is
> wider.

  Tested, and this solves bug 381573 for me (ie. An event multiplying to become two events after I dismiss a calendar alarm).  Unfortunately this has only helped me discover another  bug (which seems to exist in vanilla 0.7 and 0.8pre trunk /w Ogo anyway).  I've filed it under Bug 405530 .

Comment 13

[Bruno Browning]

These parts of Sb/Ltn have been so thoroughly rewritten that I don't think this bug is useful anymore. There is no longer any PROPFIND after PUT, instead the item is refetched using a query-by-uid as discussed in comment #1 and comment #3. Since the PROPFIND in the bug description is gone, and since the GET-after-PUT now updates local caches so that items can be modified/deleted even if the server moved them, this bug is now
->INVALID