456363 - Events deleted on CalDAV server are still displayed in Lightning after a reload

Status: RESOLVED DUPLICATE of bug 519225

(Calendar :: Provider: CalDAV, defect)

Description

[Bernard Desruisseaux]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
Build Identifier: Lightning 0.9 RC2

Lightning doesn't remove meetings that are no longer returned by the CalDAV server from his local cache.

Reproducible: Always

Steps to Reproduce:
1. Create new event in calendar with CalDAV client on computer #1.
2. Reload calendar with CalDAV client on computer #2.
3. Delete new event with CalDAV client on computer #1.
4. Reload calendar with CalDAV client on computer #2.
Actual Results:  
The deleted meeting still show up in the calendar of the CalDAV client running on computer #2.

Expected Results:  
Meeting should have disappeared.

Issue was found with Oracle Beehive's CalDAV Server using Apple iCal 3.0.5 on computer #1 and Lightning 0.9 RC2 on computer #2.

Problem seems specific to deletion. That is, new event as well as modified events are properly picked up by the client after a reload.

In my opinion, this bug should block the release of 0.9.

Comment 1

[Daniel Boelzle [:dbo]]

Works for me using Bedework with iCal (client #1) and Lightning (client #2, both with experimental caching on and off).

Comment 2

[Bruno Browning]

Bernard, does it matter whether the event has been accepted or not?

Comment 3

[Wolfgang Sourdeau]

I have tested against SOGo. Same problem here. The log pretends that the ctag hasn't changed:

<D:multistatus xmlns:a="http://calendarserver.org/ns/" xmlns:D="DAV:"><D:response><D:href>http://evariste.inverse.ca/SOGo/dav/wsourdeau/Calendar/personal/</D:href><D:propstat><D:status>HTTP/1.1 200 OK</D:status><D:prop><a:getctag>1221839700</a:getctag></D:prop></D:propstat></D:response></D:multistatus>
CalDAV: ctag matches, no need to fetch data for calendar Perso

Comment 4

[Wolfgang Sourdeau]

Also, it doesn't matter whether the calendar is cached or not, nor whether it's an invitation or not.

Comment 5

[Bruno Browning]

The fact that this does not happen on Bedework but does on Oracle and SOGo - the only two server impls I'm aware of where calendar == inbox - has to be a clue.

Comment 6

[Bernard Desruisseaux]

I wasn't able to reproduce this issue with Apple iCal Server (Mac OS X 10.5.5).

Comment 7

[Bruno Browning]

does the state of the calendar.caldav.sched.enabled pref have any bearing here?

Comment 8

[Bernard Desruisseaux]

(In reply to comment #2)
> Bernard, does it matter whether the event has been accepted or not?

No. The event created from computer #1 only has myself as attendee.
There is no scheduling involved.

Comment 9

[Bernard Desruisseaux]

(In reply to comment #7)
> does the state of the calendar.caldav.sched.enabled pref have any bearing here?

No. Was able to reproduce with calendar.caldav.sched.enabled set to true as well.

I'm now wondering if the number of events in the calendar would make a difference. I'll get back with results.

Comment 10

[Bernard Desruisseaux]

Unable to reproduce with a calendar that contains only 10 events.

Comment 11

[Daniel Boelzle [:dbo]]

I can't reproduce the issue on Bedework nor Darwin calendar server having about 30-40 events. If it's related to the ctag: I don't know whether Bedework supports it, but IIRC Apple does.

Comment 12

[Wolfgang Sourdeau]

(In reply to comment #4)
> Also, it doesn't matter whether the calendar is cached or not, nor whether it's
> an invitation or not.

SOGo was actually the culprit in this case.... When returning the ctag of the collection, SOGo returnes the last modified date of the latest modified event. But it doesn't take the last-modified field of deleted events, which causes the problem in this case. This is thus a SOGo bug, which has already been fixed internally.

I can't speak for Oracle though.

Comment 13

[Bruno Browning]

Looks like the Oracle issue is a urlencoding one. I'll take this one.

Comment 14

[Bernard Desruisseaux]

Bruno, Daniel, thanks a lot for your help on this issue.

Too bad we can't get this fixed in 0.9, on the other hand, I believe we should be able to provide a workaround on the server for this one. That is, the URL returned by our server could be specified with the exact same URL encoding used by the client. If the Request-URI specified by the client is NOT encoding the "@" character then all URLs returned to the client should NOT encode the "@" character as well and vice versa.

Comment 15

[James Andrewartha]

I'm seeing this in Bedework 3.4 with Lightning 0.9 - I delete an event in the webui, then reload in Lightning, which claims the ctag hasn't changed: 

CalDAV: recv: <?xml version="1.0" encoding="UTF-8" ?>

<multistatus xmlns:ns3="http://calendarserver.org/ns/" xmlns="DAV:" xmlns:ns1="urn:ietf:params:xml:ns:caldav" xmlns:ns2="http://www.w3.org/2002/12/cal/ical#">
  <response>
    <href>/ucaldav/user/jamesa/calendar/</href>
    <propstat>
      <prop>
        <ns3:getctag>20080925T074630Z23</ns3:getctag>
      </prop>
      <status>HTTP/1.1 200 ok</status>
    </propstat>
  </response>
</multistatus>

CalDAV: ctag matches, no need to fetch data for calendar Work

The date in the ctag stays the same before and after deleting the event, so I guess Bedework has a similar bug to SOGo.

Comment 16

[Bruno Browning]

Attachment: Always use decoded paths

this fixes the Oracle portion of the bug. It's a diff against a tree with the patch for bug 455262, so that will want to be committed first.

Comment 17

[Daniel Boelzle [:dbo]]

Comment on attachment 340751 [details] [diff] [review]
Always use decoded paths

>                     var resourcePath = thisCalendar.ensurePath(href);
>-                    itemsReported[resourcePath.toString()] = true;
>+                    itemsReported[decodeURIComponent(resourcePath.toString())] = true;

This seems wrong to me, because all other occurrences (esp. mHrefIndex) just deal with the (once) decoded path. Moreover, the .toString() is obsolete, because ensurePath always returns a string. Could you please clarify this?

>     ensurePath: function caldav_ensurePath(aString) {
>         if (aString.charAt(0) != "/") {
>             var bogusUri = makeURL(aString);
>-            return bogusUri.path;
>+            return decodeURIComponent(bogusUri.path);
>         }
>-        return aString;
>+        return decodeURIComponent(aString);
>     },
> 

ensurePath() not necessarily needs to be a member of the calendar object, but could be a free function, e.g. calDavEnsurePath().

Comment 18

[Daniel Boelzle [:dbo]]

Comment on attachment 340751 [details] [diff] [review]
Always use decoded paths


>     ensurePath: function caldav_ensurePath(aString) {
>         if (aString.charAt(0) != "/") {
>             var bogusUri = makeURL(aString);

There are calls passing in an nsIXMLElement to ensurePath. I think this might lead to errors, because nsIIOService::newURI expects a string.

BTW: bug 459322 might be another indication to let ensurePath always return a plain string.

r- for now; IMO this patch need another round.

Comment 19

[Philipp Kewisch [:Fallen][☁️📆]]

This bug was probably fixed as part of bug 498690 and bug 519225. Marking duplicate of the second, since it fits best.