Closed Bug 411799 Opened 12 years ago Closed 9 years ago

Corruption of ics calendar using webdav

Categories

(Calendar :: Provider: ICS/WebDAV, defect, critical)

defect
Not set
critical

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: skidank, Unassigned)

References

Details

(Keywords: dataloss)

Attachments

(1 file)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
Build Identifier: Lightning 0.8pre build 2008010804 on Thunderbird 2.0.0.9 (20071031)

Lightning will corrupt ics files stored on my webdav server if I simply perform multiple updates quickly. It appears that lightning is attempting to update the ics file for each request asynchronously, thus causing the file to be truncated (of course I have no way of really knowing whats going on). After this happens, lightning is usually pretty hosed and won't refresh any calendars. Usually requires that I kill the thunderbird process (it doesn't die on exit) and restart after I have recovered my calendars from backup.


The webdav server is an apache 2.2 server running on linux. The URL being requested is https and uses basic authentication. 

Reproducible: Always

Steps to Reproduce:
1.Pick your favorite lightning view (I use the week view)
2.Make a number of changes to calendar events of your choice against a calendar set up as described in the but details.
3.

Actual Results:  
Within moments the ics will be be truncated and nothing further will work until thunderbird is killed and restarted.

Expected Results:  
All requested changes to the calendar should be executed successfully and reflected correctly in the ics file.
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.9) Gecko/20071031  Thunderbird/2.0.0.9 Lightning/0.8pre 2008010904


works for me
Additional information that might be useful: the ics file is about 100K in size the the connection is relatively slow (384kb). I believe this is important because the problem seems to be timing dependent: updates to the calendar that occur while the ics is being written. It happens far less frequently for smaller calendars and over faster connections.
I can confirm this behaviour using a slow DSL connection (1024 kbit/s Down - und 128 kbit/s Up).

File size is 280kB. http://severinghaus.org/projects/icv/ has validated it positiv. 
It's important to know: do you use any proxy servers?
It sounds like from Comment 3, we can say that this is Confirmed.

I'm going to leave the QAWanted assessment in place, because it would be good to have a simple set of steps to test this with.  Steffen, can we use your webdav server? 

Thanks for the help ya'll.  It would also be good to check this with the latest 0.8 RC, which you can pick up here:
http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/0.8-candidates/rc1/
Status: UNCONFIRMED → NEW
Ever confirmed: true
I tested the latest from the link above and the problem persists. Also, there are no proxy servers in any of the environments in which this problem occurs for me.
One other symptom I noticed: Not only does thunderbird not die after this occurs, but it pegs the CPU on one core of my box. Don't know if this is useful information.
I can confirm the bug on a 348k up DSL line - but only in 1 of 20-30 instances.
For me the truncated file size is always n*2048 Bytes. Hope it helps.

GS
(In reply to comment #5)
> Steffen, can we use your webdav server?

Using my productive WebDAV server is not possible. Sorry. But I've checked with the WebDAV access to the Mediacenter of GMX.NET and have the same problem there. So this could be a good test server.
Flags: wanted-calendar0.9+
Any update on this bug? It's killing me.
As you can read in a letter to the editor in the 14/08 copy of the German c't magazine (p. 12) there is at least one other person having that problem (but only every 3 or 4 weeks).
Flags: blocking-calendar0.9?
I have the same problem using Lightning0.8 (Build 2008033120) in TB 2.0.0.14 (20080421). 
It corrupts my calendar files almost every second update.  I'm using Mediacenter of GMX have a really slow internet connection but a rather big ics file (200kB).
It corrupts almost everytime a notification pops up, I'm really frustrated.
We have a similar setup here at my company (Apache 2.2 WebDAV)
This problem occurs also at LAN, while many user accessing the same ics file via WebDav. At some point the ics file get truncated.
I tried to reproduce the issue using GMX Media Center with a large calendar file without success w.r.t. file truncation. I get sporadic errors when stressing the calendar that PUT has failed, but a refresh reloads the file fine. This might be due to my good DSL connection, though.
Logging the locking and unlocking ics provider's action queue it looks good, i.e. it is kept locked until a PUT has been completed, thus before another PUT could happen.
What I spotted in the code is that there's a potential race when fetching the etag (GMX doesn't return it on PUT, so there's a follow-up PROPFIND). We don't wait for that until the next PUT is launched. However, this only might lead the etag check to fail, but is IMO no cause for file truncation.

(In reply to comment #13)
> We have a similar setup here at my company (Apache 2.2 WebDAV)
> This problem occurs also at LAN, while many user accessing the same ics file
> via WebDav. At some point the ics file get truncated.
I think this has been filed as bug 329570 resp. bug 290446. The latter is more like a solution proposal about whether to check etags or use locks.
Comment on attachment 331296 [details] [diff] [review]
[checked in] fixing potential PROPFIND / unlock race

Not sure I like the fact that a callback function is used here, but I understand why you need to do so.

Since an alternative doesn't come to my mind,

r=philipp
Attachment #331296 - Flags: review?(philipp) → review+
Attachment #331296 - Attachment description: fixing potential PROPFIND / unlock race → [checked in] fixing potential PROPFIND / unlock race
Since I cannot reproduce the bug using e.g. MediaCenter with large ics files, could anyone try out a recent nightly whether the last patch makes it better?
Using the nightly build from Aug 05, the problem still occurs. I can cause it to happen with a few moments by simply moving an event around on the calendar a handful of times in quick succession. I also noticed a new behavior. On one occasion, lightning hung refreshing a calendar but the calendar was not truncated. Restarting thunderbird resolved that problem. 
Whiteboard: [needs patch]
Whiteboard: [needs patch]
I once again tried to reproduce the issue against GMX media center slowing down my DSL connection via ipfw pipes to modem/isdn speed without success.
Patches are really welcome, but unless we can safely reproduce the problem (which makes it almost impossible to fix), we won't block 0.9 on it.
Flags: blocking-calendar0.9? → blocking-calendar0.9-
I can set up access to my server if someone would like to test there. Send me an email and I'll send along connection info and credentials.
My ics file get corrupted once a week. The calendar is also stored on the GMX media center. Could bug 390036 play a role?
@Daniel: Do you use IMAP/SMTP together with SSL?
I will disable SSL and report if this still happens. 
I switched all of my calendars from https to http. After about a week the problem has not recurred when normally it happens multiple times a week. It does sound like it may be related to bug 390036.  I had two occurrences of a calendar failing to reload after a modification. I had to kill the thunderbird process to get it to load, but the ics file was fine. No excessive CPU in this case.
(In reply to comment #21)
> @Daniel: Do you use IMAP/SMTP together with SSL?
Yes.
Another me, too.. (IMAP and SMTP with SSL)
Flags: wanted-calendar1.0+
Flags: wanted-calendar0.9+
Flags: blocking-calendar1.0?
Flags: blocking-calendar0.9-
I think this is fixed for 1.0pre by bug 329570 (concurrent editing) and the fixes in bug 390036 (hanging TB with Lightning with SSL). The latter contains a fix for 0.9 too (if you install the 0.9 fix in bug 363455).
Ralph, Steffen, Gerhard, Harald, Markus: Could you please check whether this problem still occurs using a recent lightning 1.0pre nightly? Thanks!
Not using TB 3.0pre and Lightning 1.0pre does not install on TB 2.0 - if I'm right..
Duplicate of this bug: 423623
Keywords: qawanted
OS: Windows XP → All
Hardware: x86 → All
Does anyone experiencing this issue have apache server logs available? This might be a similar issue to bug 410874. This needs to be retested with Lightning 1.0b1 and Thunderbird 3.
Duplicate of this bug: 529782
I'm going to close this bug for lack of testers. If this problem still occurs, please open a new bug.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → 1.0b1
Flags: blocking-calendar1.0?
You need to log in before you can comment on or make changes to this bug.