327602 - Pushing the calendar file failed. Status code: 500: Illegal Calendar data format

Status: RESOLVED FIXED

(Calendar :: Provider: ICS/WebDAV, defect)

Description

[Jan Eldenmalm]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051201 Thunderbird/1.5

When creating a yearly recurring event the calendar reports "Pushing the calendar file failed. Status code: 500: Illegal Calendar data format"



Reproducible: Always

Steps to Reproduce:
1. Create recurring yearly event
2. save
3. after save the error occurs

Actual Results:  
The event is not stored on the central calendar.
If the event came into Sunbird(thunderbirdplugin) from the server ( for example if used my PDA to edit the central calendar), than Calendar keeps complaining every time it tries to update the event ( to unset reminder)..

Comment 1

[Joey Minta]

A 500 error is from the server, so we need to know where you are trying to publish to.  Right now, I suspect this is a problem with the server, rather than with Sunbird.  Please also include your version information for what copy of Sunbird you're using.

Comment 2

[Jan Eldenmalm]

I receive the same error usually related to recurrring yearly events like birthdays. my calendar server is communigate pro.

Using:
Mozilla Calendar 2006012105-cal
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051201 Thunderbird/1.5

Comment 3

[Thomas Bleek]

Hello,

I have the same error for ALL calendar entries. After creating an event with
sunbird 0.3a2 I see this error. some tests with exporting importing to communigate
pro shows, that cgp don't like these extensions to the date lines
DTSTART;TZID=/mozilla.org/20050126_1/Europe/Berlin:20060625T100000
DTEND;TZID=/mozilla.org/20050126_1/Europe/Berlin:20060625T110000

After deleting the ";TZID=/mozilla.org/20050126_1/Europe/Berlin"
it is importing without errors.

The support of Stalker told me, that this format is not confrming to the standard and so the will do nothing on this, i think.

Are there right or wrong???
Thanks,
thomas
[e-mail]

Comment 4

[Joey Minta]

(In reply to comment #3)
> cgp don't like these extensions to the date lines
> DTSTART;TZID=/mozilla.org/20050126_1/Europe/Berlin:20060625T100000
> DTEND;TZID=/mozilla.org/20050126_1/Europe/Berlin:20060625T110000
> The support of Stalker told me, that this format is not confrming to the
> standard and so the will do nothing on this, i think.
> 
> Are there right or wrong???
The spec gives the following example of the a VTIMEZONE in use:
   The following are examples of this property parameter:

     DTSTART;TZID=US-Eastern:19980119T020000

     DTEND;TZID=US-Eastern:19980119T030000

During interop testing no one in CalConnect had problems with our timezones and I firmly believe they are within the spec.  If you can elaborate at all on why they feel we're violating the spec, that would be greatly appreciated

Comment 5

[Matthew (lilmatt) Willis]

*** Bug 344682 has been marked as a duplicate of this bug. ***

Comment 6

[Tom]

(In reply to comment #5)
> *** Bug 344682 has been marked as a duplicate of this bug. ***
> 
I actually think that the problem is when there is a tag that has TAG;VALUE=SOMETHING:SOMETHING

I say this because I am also having problems with the VALARM tags because of the TRIGGER;VALUE=DURATION:-PT15M  If you take away the ;VALUE=DURATION portion it works fine.

Comment 7

[Joey Minta]

(In reply to comment #6)
> (In reply to comment #5)
> I say this because I am also having problems with the VALARM tags because of
> the TRIGGER;VALUE=DURATION:-PT15M  If you take away the ;VALUE=DURATION portion
> it works fine.
> 
RFC2445 (http://www.rfc-archive.org/getrfc.php?rfc=2445) is your friend.  This duration bit looks legitimate to me too.

"  The "TRIGGER" property specifies when the alarm will be triggered.
   The "TRIGGER" property specifies a duration prior to the start of an
   event or a to-do. The "TRIGGER" edge may be explicitly set to be
   relative to the "START" or "END" of the event or to-do with the
   "RELATED" parameter of the "TRIGGER" property. The "TRIGGER" property
   value type can alternatively be set to an absolute calendar date and
   time of day value."
So,  you can easily get either a DURATION or a DATETIME value for TRIGGER.  The VALUE=DURATION helps clients distinguish which it is they're getting.  The RFC gives an example of this syntax with a DATETIME:
     BEGIN:VALARM
     TRIGGER;VALUE=DATE-TIME:19970317T133000Z
     REPEAT:4
     DURATION:PT15M
     ACTION:AUDIO
     ATTACH;FMTTYPE=audio/basic:ftp://host.com/pub/sounds/bell-01.aud
     END:VALARM

Comment 8

[Tom]

Below is the response that I received from Communigate technical support.  Does this make sense and if so can it be changed within the source code and not break other portions of the program?

###Start Blaming Others###
Hello,

I must note that the version of Sunbird that existed previously (1.x) was able to work with CGPro. The version of Sunbird you are currently using (0.3x) is an alfa-version. It can incorporate many changes.

The change that is needed here is that the time-zone should be defined before it is used anywhere in the calendar data (quite logical, isn't it?). It is not the case now, and that is why import doesn't work.

So I suggest you to contact Mozilla in this case so that they change the data order in their vCalendar data - VTIMEZONE specification should appear before the TZID. That will solve the problem. 
###End Blaming Others###

Comment 9

[Matthew (lilmatt) Willis]

Moving to WebDAV. CGPro doesn't have CalDAV support as of yet.

Comment 10

[Tom]

(In reply to comment #9)
> Moving to WebDAV. CGPro doesn't have CalDAV support as of yet.
> 
We are using WebDAV.  This is where the problem occurs.

Comment 11

[Michiel van Leeuwen (email: mvl+moz@)]

We might indeed put the VTIMEZONE section after the events. But I think that's not a violation of rfc2445. But if it helps with interoperability, changing the order might be a good thing,

Comment 12

[Tom]

(In reply to comment #11)
> We might indeed put the VTIMEZONE section after the events. But I think that's
> not a violation of rfc2445. But if it helps with interoperability, changing the
> order might be a good thing,
> 

Has any progress been made towards this?  Where in the code is this done (what file)?  Thanks.

Comment 13

[Piglith]

This still is returning 

Error number 0x0
Publishing the calendar file failed
Status code: 500: Illegal Calendar data format

Connecting to CommuniGatePro.

When using the older Calendar there is no problem in connecting to CGP. 

I'm using Lightning 0.3a2+ (build 2006091506)

Comment 14

[Matthew (lilmatt) Willis]

(In reply to comment #11)
> We might indeed put the VTIMEZONE section after the events.
We do.

> But I think that's not a violation of rfc2445.
It's not.

> But if it helps with interoperability, changing the order might be a good
> thing
Many other ICS files I see do it this way.

Comment 15

[Tom]

(In reply to comment #14)
> (In reply to comment #11)
> > We might indeed put the VTIMEZONE section after the events.
> We do.
> 
> > But I think that's not a violation of rfc2445.
> It's not.
> 
> > But if it helps with interoperability, changing the order might be a good
> > thing
> Many other ICS files I see do it this way.
> 

So, does this mean that no changes will be made to the order for interoperability?  If this is the case then anyone that uses Communigate for a server will need to find a new client.

Comment 16

[Matthew (lilmatt) Willis]

(In reply to comment #15)
> So, does this mean that no changes will be made to the order for
> interoperability?  If this is the case then anyone that uses Communigate for a
> server will need to find a new client.

That's not what it means at all.  The bug just hasn't been fixed yet, as we're trying to get 0.3 out the door.  Since we're not violating the spec at the moment (meaning it's a workaround for CGPro), it's not the highest priority.

Comment 17

[Bas van den Bosch]

I dug into this (see http://groups.google.nl/group/demos.local.lists.cgp/browse_thread/thread/2b68396e1804be98/a14fb2091d69bd51?lnk=st&q=sunbird+communigate&rnum=1#a14fb2091d69bd51 sorry for the ugly link...)

I think there's actually 2 problems. The VTIMEZONE-problem as discussed above which also happens when importing an ics-file (created by exporting from communigate) in a local calendar is one problem. The other (error 500 illegal format) throws a PROPFIND-unknown method in the error logs of communigate. I can't seem to reproduce this specific method now though, just getting error 500 "illegal Calendar data format" in the communigate logs.

Just tested with different releases, 0.3a doesn't give this error, from 0.3b it does... Somewhere in between some backend-code seems to have changed.

Comment 18

[Matthias Schlett]

Communigate Pro is our main communication server and we really need a Linux calender application which has equal or better features than Outlook.

Comment 19

[Bas van den Bosch]

I've been thinking about this. Cgate doesn't follow webdav-protocols. When we (users) add an event the whole ics has to be deleted and then rewritten. This worked up till version 0.3a but was rewritten for efficiency-reasons in 0.3b. I'm no programmer but perhaps someone could write a plugin for this (reverting to the old code)? Also Viraj Alankar wrote a proxy which only writes the altered events to communigate.
http://linux.softpedia.com/get/Office/Scheduling/IMAP-Calendar-Proxy-7946.shtml

This proxy was released under GPL-code and I think he wouldn't mind releasing it under a different license (his own page http://www.viraj.org/ is down now). I even think the stalker-developers wouldn't mind helping out if needed. 

Comment 20

[A Ballesteros]

I have documented more about this issue here:
http://mail.communigate.com/~ab/files/moz-cal-cgp.html

CommuniGate Systems (formerly Stalker Software) does not yet have complete webdav implemented for the server side storage area of our server product. I wish to vote for a higher priority for this bug report.

Comment 21

[Matthew (lilmatt) Willis]

Attachment: Use pvl_unshift for VTIMEZONEs, and pvl_push for all others

Example without this patch: (note VTIMEZONE at end)

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
BEGIN:VEVENT
CREATED:20070111T122005Z
LAST-MODIFIED:20070111T122005Z
DTSTAMP:20070111T122005Z
UID:610af4dd-9ca3-af42-98b6-2ed7461b328f
SUMMARY:MondayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070108T120000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070108T130000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122005Z
LAST-MODIFIED:20070111T122005Z
DTSTAMP:20070111T122005Z
UID:1fccfc24-6ee8-8f42-aa9d-99dca2c55140
SUMMARY:TuesdayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070109T130000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070109T140000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122005Z
LAST-MODIFIED:20070111T122005Z
DTSTAMP:20070111T122005Z
UID:874a1625-c286-2e42-b199-c457306ba9ec
SUMMARY:WednesdayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070110T140000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070110T150000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122005Z
LAST-MODIFIED:20070111T122005Z
DTSTAMP:20070111T122005Z
UID:8df5e82b-347d-aa43-9d9b-f11a35bb2865
SUMMARY:ThursdayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070111T150000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070111T160000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122005Z
LAST-MODIFIED:20070111T122005Z
DTSTAMP:20070111T122005Z
UID:bcd7ba46-e636-e04f-9c6c-ba057719d08a
SUMMARY:FridayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070112T160000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070112T170000
END:VEVENT
BEGIN:VTIMEZONE
TZID:/mozilla.org/20050126_1/America/Detroit
X-LIC-LOCATION:America/Detroit
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700405T020000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=4
END:DAYLIGHT
END:VTIMEZONE
END:VCALENDAR



Same example with this patch:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
BEGIN:VTIMEZONE
TZID:/mozilla.org/20050126_1/America/Detroit
X-LIC-LOCATION:America/Detroit
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700405T020000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=4
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20070111T122722Z
LAST-MODIFIED:20070111T122722Z
DTSTAMP:20070111T122722Z
UID:610af4dd-9ca3-af42-98b6-2ed7461b328f
SUMMARY:MondayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070108T120000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070108T130000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122722Z
LAST-MODIFIED:20070111T122722Z
DTSTAMP:20070111T122722Z
UID:1fccfc24-6ee8-8f42-aa9d-99dca2c55140
SUMMARY:TuesdayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070109T130000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070109T140000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122722Z
LAST-MODIFIED:20070111T122722Z
DTSTAMP:20070111T122722Z
UID:874a1625-c286-2e42-b199-c457306ba9ec
SUMMARY:WednesdayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070110T140000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070110T150000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122722Z
LAST-MODIFIED:20070111T122722Z
DTSTAMP:20070111T122722Z
UID:8df5e82b-347d-aa43-9d9b-f11a35bb2865
SUMMARY:ThursdayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070111T150000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070111T160000
END:VEVENT
BEGIN:VEVENT
CREATED:20070111T122722Z
LAST-MODIFIED:20070111T122722Z
DTSTAMP:20070111T122722Z
UID:bcd7ba46-e636-e04f-9c6c-ba057719d08a
SUMMARY:FridayVent
DTSTART;TZID=/mozilla.org/20050126_1/America/Detroit:20070112T160000
DTEND;TZID=/mozilla.org/20050126_1/America/Detroit:20070112T170000
END:VEVENT
END:VCALENDAR

Comment 22

[cmtalbert]

Comment on attachment 251170 [details] [diff] [review]
Use pvl_unshift for VTIMEZONEs, and pvl_push for all others

Only one minor nit. Would it be possible to correct the indenting for the rest of the else block that is not in this patch (the existing code).  Otherwise, looks good.

Comment 23

[Michiel van Leeuwen (email: mvl+moz@)]

(In reply to comment #20)
> CommuniGate Systems (formerly Stalker Software) does not yet have complete
> webdav implemented for the server side storage area of our server product. I
> wish to vote for a higher priority for this bug report.

So, what is the actual bug? What is not complete about the webdav implementation of communigate? the place of the vtimezone component in the file does not have anything to do with the webdav standard.

Can anybody confirm if moving the vtimezone component does fix the problem?

Comment 24

[Matthew (lilmatt) Willis]

mvl: Check out the blog post from the CommuniGate guy linked to in comment #20.

The patch should fix the problem.

Comment 25

[Michiel van Leeuwen (email: mvl+moz@)]

According to that blogpost, communigate is fixable. Then it would actaulyl comply to the spec. So why should the sunbird code be changed?

Comment 26

[Simon Paquet [:sipaq]]

(In reply to comment #25)
> According to that blogpost, communigate is fixable. Then it would actaulyl
> comply to the spec. So why should the sunbird code be changed?

I think that both parties should fix this on their end. We should try to to interoperate with all available server products (if possible) and communigate should really try to get this fixed, since other products that work on top of libical will also exhibit this behaviour.

Comment 27

[Matthew (lilmatt) Willis]

I've already communicated with folks from CommuniGate to try and get them to fix their parsing, but until that happens, making this change in our code will allow more users to use our apps with their product, and will make our ICS look more similar to ICS generated by iCal, Outlook, Zimbra, etc.

Comment 28

[Bas van den Bosch]

(In reply to comment #27)
Patching the Calendar-code would mean also the old installs (4.x codebase) of Communigate would benefit (assuming this won't make it to a critical update of communigate which is in the 5.x versions now). Also, if the latter is true and there's no side-effects in the functioning of Calendar, it would seem logical to first parse the "meta-data" of the vevent and secondly the event itself. If one has a windows-build with the patch I would happily do some testing with different back-ends (webdav, cgate, local, synchronising with nokia and outlook etc etc) this weekend.

Comment 29

[Michiel van Leeuwen (email: mvl+moz@)]

Comment on attachment 251170 [details] [diff] [review]
Use pvl_unshift for VTIMEZONEs, and pvl_push for all others

I'd like to know if this really fixes the problem before committing it. libical is not our code, and is supposed to be as close to the original as possible. So any change to it is scary and needs a good reason.

Comment 30

[Michiel van Leeuwen (email: mvl+moz@)]

Comment on attachment 251170 [details] [diff] [review]
Use pvl_unshift for VTIMEZONEs, and pvl_push for all others

r2=mvl, so this can be checked in. Then it needs to be tested asap, to see if it does fix the problem. If not, this must be backed out.

Oh, and add a reference to this bug in the code comments.

Comment 31

[Matthew (lilmatt) Willis]

Patch checked in on MOZILLA_1_8_BRANCH and trunk.

Bas,
Would you please test using tomorrow's nightly builds?  They should contain this fix.

I will only mark this FIXED after your confirmation.

Comment 32

[Bas van den Bosch]

Just tested the latest nightly builds of both sunbird and ligtning on a
webdav-server and local: no problems so nothng gets broken.

On Communigate 4.3.12 I got no problems up till now when starting
with blank agenda's. When using pre-existing agenda's there may be some
problems (in my case outlook-mapi calendar, synchronized with my nokia),
don't know yet wether these are caused by sunbird or by communigate.

sunbird: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a2pre)
Gecko/20070115 Calendar/0.6a1
lightning: 0.4a1 - build 2007011505 on Thunderbird 1.5.0.9

It seems like we finally have communigate with sunbird/lightning back...
Going to do some more testing and troubleshooting. Thanks all...

Comment 33

[Bas van den Bosch]

The problem I mentioned is caused by attendees getting doublequoted (see bug 368607 - https://bugzilla.mozilla.org/show_bug.cgi?id=368607). Besides this I haven't been able to find any bugs...

Comment 34

[Martin Schröder [:mschroeder]]

Can this bug be marked FIXED?

Comment 35

[Matthew (lilmatt) Willis]

Given comment 32 and comment 34, -> FIXED

Comment 36

[Bas van den Bosch]

Agree with lilmatt (didn't have rights to set to fixed at 2007-02-04). I will keep a keen eye on this though, doing the litmus-tests with local/file/ftp/http-webdav and http-communigate. 

Comment 37

[Matthias Schlett]

But in which release these changes will appear ? The homepageURL http://www.mozilla.org/projects/calendar/releases/lightning0.4a1.html doesn't exist and the release 0.3.1 contains this bug

Comment 38

[Stefan Sitter [:ssitter]]

(In reply to comment #37)
> But in which release these changes will appear?

It is now fixed in current 0.4a1 nightly test builds and will appear in the next official release (probably 0.5).