287612 - PUT without GET/merge when deleting events from remote calendars

Status: VERIFIED DUPLICATE of bug 329570

(Calendar :: Provider: ICS/WebDAV, defect)

Description

[Chris Jones]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041111 Firefox/1.0
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041111 Firefox/1.0

Mozilla Calendar does not perform a GET prior to PUTting a new version of a
calendar on a webDAV server when deleting an event; it does behave correctly for
adding events.

Reproducible: Always

Steps to Reproduce:
1.  Create calendar stored on webDAV server, enabling "Automatically publish"
option.
2.  Add event A to calendar.
3.  Add another event B from another client (e.g. another instance of Mozilla
Calendar).
4.  Delete event A.

Actual Results:  
Event B is no longer in the calendar file, because first client issues a PUT in
step 4 without a preceding GET to ensure it has the most recent version of the
calendar.  Event A is also removed.

Expected Results:  
Event A should be removed; event B should remain.

/* Add an event to the calendar. */
nipisi.cs.ualberta.ca - cdjones [24/Mar/2005:11:10:53 -0700] "GET
/webDAV/cdjones/testing.ics HTTP/1.1" 200 358
nipisi.cs.ualberta.ca - cdjones [24/Mar/2005:11:10:54 -0700] "PUT
/webDAV/cdjones/testing.ics HTTP/1.1" 204 0
/* Now delete the event. */
nipisi.cs.ualberta.ca - cdjones [24/Mar/2005:11:11:14 -0700] "PUT
/webDAV/cdjones/testing.ics HTTP/1.1" 204 0

Comment 1

[Joey Minta]

*** Bug 311296 has been marked as a duplicate of this bug. ***

Comment 2

[Matthew (lilmatt) Willis]

Reassigning all automatically assigned bugs from Mostafa to nobody@m.o

Bugspam filter: TorontoMostafaMove

Comment 3

[Bas van den Bosch]

confirmed, can also be reproduced with TB/lightning and SB on the same workstation connecting to a ftp-server. Should block 0.5?

Comment 4

[Chris Jones]

It shocks me that this can sit for two years before being confirmed, let alone being fixed (for what should be, in principle, fairly simple to fix considering that it works correctly in the add-event case).

Comment 5

[Sebastian Schwieger]

The handling has changed since the report of this bug. Now, there is a check if the remote file has changed by looking at the etag. If the etag is different from the etag obtained by previous reading, writing is not done and an error message appears. This however only works on with WebDav servers. So, at the moment no version checking is done in case of LocalFile and ftp.
LocalFile could be checked by .lastModifiedTime, ftp however is more problematic.
Any thoughts welcome.

Comment 6

[Sebastian Schwieger]

Attachment: check for the last modified time in case of nsILocalFile

This prevents changes made by other programs/people to be overridden by calendar based on a check for the last modified time. This is done in case of local files, only. This patch does not address files handled by ftp.

Comment 7

[Sebastian Schwieger]

Comment on attachment 258047 [details] [diff] [review]
check for the last modified time in case of nsILocalFile

This bug should only deal with WebDAV problems, ie not ftp and localFile which will go in separate bugs. etag checking is currently off in TRUNK builds. Adding dependency to the bug that takes care of re-enabling etags.

Comment 8

[Bas van den Bosch]

bug 373430 filed as followup for ftp...

Comment 9

[Matthew (lilmatt) Willis]

We won't hold 0.5 up for this.

Comment 10

[Martin Sivak]

Well the overwrite happens on Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a5pre) Gecko/20070501 Calendar/0.6a1 even during adding an event.

I could be because Lighttpd+WebDAV was used, but this configuration is becoming more common nowadays.

I tried to check Etags to make sure it is reported correctly by webserver and this is what I discovered:

Original file (written by first client):
HTTP/1.1 200 OK
Content-Type: application/octet-stream
ETag: "-132565182"
Accept-Ranges: bytes
Last-Modified: Tue, 01 May 2007 16:22:04 GMT
Content-Length: 2035
Date: Tue, 01 May 2007 16:24:07 GMT
Server: lighttpd/1.4.13

After modified by second client:
HTTP/1.1 200 OK
Content-Type: application/octet-stream
ETag: "-937937396"
Accept-Ranges: bytes
Last-Modified: Tue, 01 May 2007 16:25:20 GMT
Content-Length: 2746
Date: Tue, 01 May 2007 16:25:51 GMT
Server: lighttpd/1.4.13


And now the first client silently writes his own NOT updated version and destroys the changes made by second client.

HTTP/1.1 200 OK
Content-Type: application/octet-stream
ETag: "672671714"
Accept-Ranges: bytes
Last-Modified: Tue, 01 May 2007 16:26:07 GMT
Content-Length: 2338
Date: Tue, 01 May 2007 16:42:11 GMT
Server: lighttpd/1.4.13



The detailed HTTP sequence was (there were really no locking requests):

::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:22:04 +0200] "GET /msivak/test.ics HTTP/1.1" 200 2443 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a5pre) Gecko/20070501 Calendar/0.6a1"
::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:22:04 +0200] "PUT /msivak/test.ics HTTP/1.1" 200 0 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a5pre) Gecko/20070501 Calendar/0.6a1"

<-- after modified by first client

::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:22:43 +0200] "GET /msivak/test.ics HTTP/1.1" 200 2035 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070329 Thunderbird/1.5.0.10"

<-- second client reloaded

::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:25:20 +0200] "GET /msivak/test.ics HTTP/1.1" 200 2035 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070329 Thunderbird/1.5.0.10"
::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:25:20 +0200] "PUT /msivak/test.ics HTTP/1.1" 200 0 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070329 Thunderbird/1.5.0.10"

<-- second client has written his updated version

::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:26:07 +0200] "GET /msivak/test.ics HTTP/1.1" 200 2746 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a5pre) Gecko/20070501 Calendar/0.6a1"
::ffff:192.168.0.2 dav.local.montik.net msivak [01/May/2007:18:26:07 +0200] "PUT /msivak/test.ics HTTP/1.1" 200 0 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a5pre) Gecko/20070501 Calendar/0.6a1"

<-- first client written his changes without previous manual reload and data loss happened without any notice or warning

Comment 11

[Sebastian Schwieger]

(In reply to comment #10)
> Well the overwrite happens on Mozilla/5.0 (X11; U; Linux i686; en-US;
> rv:1.9a5pre) Gecko/20070501 Calendar/0.6a1 even during adding an event.
> 
This is probably because you used a Trunk build. Etag checking is off on Trunk. If you try the latest 1.8-Branch build (Calendar/0.5pre) overwriting should theoretically not happen, because etags are checked before writing.

Comment 12

[Peter Körner]

Hi there
The ETag-checking is a good thing, but it is not very useful if it just print an error message on the screen (neither a human-readable nor a technical like in the current branch). What should happen is desribed in this comment: https://bugzilla.mozilla.org/show_bug.cgi?id=329570#c7

I would be happy if someone could evaluate a patch, making sunbird reloading the calender if the e-tag has changed, before writing to it. In the current state WebDAV/iCal is not usable with more than one workstation (and if it isn't -- why should one use a server at all?)

Comment 13

[Paul K]

Version 0.2 has an option/comment when subscribing to a remote calendar:

"Automatically publish your changes to the remote calendar? When enabled, a fresh version of the calendar is downloaded before applying any change you make. Your changes are then applied and the calendar is uploaded back to the server.".

  I believe this is a feature that has been missing since version 0.2. I wish it would return.

Comment 14

[Marco van Beek]

This seems to be related, at least when using WebDAV, to bug 290446, which is to do with using file locking. As far as I can see from my server, the WebDAV database has not been used since Apache was last restarted, and we only use WebDAV in the calendars directory. I do not think that the "Automatically publish your changes to the remote calendar..." is the best solution, as it should not be an option. It should happen in the background automatically by using file locking.

Comment 15

[Simon Paquet [:sipaq]]

Sebastian, are you still working on this bug?

Comment 16

[Sebastian Schwieger]

No, I'm not working on this bug. I'd say this bug depends on bug 327933 (Turn etag checking on on TRUNK) and on bug 274967 (implement a sync mechanism). Other than that, I see nothing that can be done here.

Comment 17

[Marco van Beek]

Surely this is a pretty fundamental bug in the implementation of file locking? Bugs 327933 and 274967 are listed with a severity of normal. I would say that anything that is preventing this bug being fixed should be reclassified as major. In fact, given that it causes loss of data, should it not be clasified as Critical?

Comment 18

[Bas van den Bosch]

This bug isn't about file-locking on the server (the server should handle that) but about read before put to check if the file in memory has changed. As per bug 388925 sunbird seems to do a reload but doesn't write to memory, etag-checking is on on branch iirc so this bug could be fixed rather easily I think.

Comment 19

[Michiel van Leeuwen (email: mvl+moz@)]

If you think this is pretty easy, be my guess to fix it. But don't forget the 'merge' part of the bug title.

Comment 20

[Michiel van Leeuwen (email: mvl+moz@)]

I'm not sure what this bug is about now. Is it about deleting an item behaving differently then editing? In that case, it might very well be a worksforme, since there is only one function that uploads the ics file, and it is the called for both editing and deleting.
If it is about not merging remote changes before uploading a local change, how is it not a duplicate of bug 329570?

Comment 21

[Bas van den Bosch]

Judging comment 0 there is (or at least was) a difference as before a put a get is issued as opposed to deleting. But bug 388925 states the data from the get doesn't het into the memory-storage. However, where the merge-part comes from isn't clear to me. I think this bug is solved when a get is issued after a del-operation.

Comment 22

[Michiel van Leeuwen (email: mvl+moz@)]

The summary is actually about using multiple clients at the same time on the same file. This is a duplicate. (The other might be newer, but is clearer)