mod_caldav: Entry with & in summary fails to load via CalDAV

UNCONFIRMED
Unassigned

Status

UNCONFIRMED
7 years ago
6 years ago

People

(Reporter: m, Unassigned)

Tracking

Lightning 1.1
x86_64
Windows 7

Details

(Whiteboard: [CalDAV server: mod_caldav])

Attachments

(3 attachments)

(Reporter)

Description

7 years ago
Created attachment 587652 [details]
example event

If an '&'-character is present in the SUMMARY field of a .ics file stored on the CalDAV server the event is not displayed. An example .ics file is attached. Importing this file locally works just fine. Hinting on an error in CalDAV parsing.

Further more, adding a "\n" in place of the '&' will lead to an tab character being displayed in the summary field of lightning.

Also an erroneous file breaks the handing of all later handled events.

Comment 1

7 years ago
I tried to reproduce this, but I cannot.

I saved your attachment to hard disk and then used "Events and Tasks" -> "Import ..." within Thunderbird and selected my Apple Calendar Server (trunk).

It appears without a problem.

I am using Lightning 1.1.1 and Thunderbird 9.

Which versions are you using, which calendar server are you using?

It somehow reminds me of bug #704012 ...

Comment 2

7 years ago
reporter please answer the questions in comment #1. For me it works.
Whiteboard: [closeSoon]
(Reporter)

Comment 3

7 years ago
Thunderbird 9.0.1
Lightning 1.1.1
CalDAV server resides on a Synology NAS box (DSM 3.2).

Comment 4

7 years ago
Thanks for the information.

Do you have any information what is running on the Synology?

I found http://www.synology.com/dsm/web_dav.php?lang=enu which shows the menu to enable it, however i cannot find the source of it.

Can you login and see which proces listens on the CalDAV port? Is is apache?
Whiteboard: [closeSoon]
(Reporter)

Comment 5

7 years ago
Accessing the port tells me:
Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/1.0.0d DAV/2 Server at nass Port 5006

Comment 6

7 years ago
So this is something based on apache, but I expect it not to be mod_caldav, then it should be in the header posted in comment #5.

However, it still could be http://code.google.com/p/sabredav/ or http://www.davical.org/. But I would expect mod_php in the header ...

Can you paste apache configuration of this NAS to see how request to https://host:5006/cal/ are handled.
(Reporter)

Comment 7

7 years ago
That will take a bit of time since I myself cannot access the configuration. So I have to wait for my coworker to give me a copy.

Until then, is there a way to make thunderbird/lightning dump the communication with the server. A protocol error on the server side should be visible there.

Comment 8

7 years ago
You can certainly grab the communication with http://www.wireshark.org/, although I am not sure how to get into the SSL content from my head.

Otherwise there are two calendar.debug* options in about:config of thunderbird which produce many debug messages ...
(Reporter)

Comment 9

7 years ago
Created attachment 589470 [details]
Apache Configuration excerpt

(Cal/Web)dav related part of the configuration.
(Reporter)

Comment 10

7 years ago
Intercepting ssl communication with wireshark is to much of a hassle currently

On the other hand I activated calendar.debug* which leads to the following interesting log entry:

===begin===
CalDAV: recv: 
<D:multistatus>
=== snip ===
<D:response>
<D:href>/calendar/mwd/bffbfee5-255e-44f6-a706-1695c5d56699.ics</D:href>
<D:propstat>
<D:prop>
<lp1:getetag>"4b6ccdf226700"</lp1:getetag>
<lp3:calendar-data>BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Europe/Paris
X-LIC-LOCATION:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20120111T104655Z
LAST-MODIFIED:20120111T104659Z
DTSTAMP:20120111T104659Z
UID:bffbfee5-255e-44f6-a706-1695c5d56699
SUMMARY:a
=== end ===

Hence it ends where the '&' character should be. 
Comment #9
Also see

Comment 11

7 years ago
Thanks for the information!

Jaco could you comment on this? I think you have done intensive mod_caldav debugging... ;) Thanks alot!
Summary: Entry with & in summary fails to load via CalDAV → mod_caldav: Entry with & in summary fails to load via CalDAV
(Reporter)

Comment 12

7 years ago
I managed to sniff the HTTP data stream. Here is an excerpt of the http trafic in question:

=== snip ===
<D:response xmlns:lp1="DAV:" xmlns:lp2="http://apache.org/dav/props/" xmlns:lp3="urn:ietf:params:xml:ns:caldav" xmlns:lp4="http://calendarserver.org/ns/">
<D:href>/calendar/mwd/bffbfee5-255e-44f6-a706-1695c5d56699.ics</D:href>
<D:propstat>
<D:prop>
<lp1:getetag>"4b6ccdf226700"</lp1:getetag>
<lp3:calendar-data>BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Europe/Paris
X-LIC-LOCATION:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20120111T104655Z
LAST-MODIFIED:20120111T104659Z
DTSTAMP:20120111T104659Z
UID:bffbfee5-255e-44f6-a706-1695c5d56699
SUMMARY:a & b
DTSTART;TZID=Europe/Paris:20120113T120000
DTEND;TZID=Europe/Paris:20120113T130000
END:VEVENT
END:VCALENDAR
</lp3:calendar-data>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
=== snip ===

As you can see the entry in question is wholy transmitted.

Comment 13

7 years ago
Hi,

It's a bug in mod_caldav.  See http://sourceforge.net/tracker/?func=detail&aid=3468503&group_id=191888&atid=939185 for more details.  Patch for mod_caldav also attached at that location.  The issue is that certain special xml characters should get escaped, so in XML a & starts an entity, however, since mod_caldav doesn't escape it to &amp; it causes some clients to break.  In the case above the SUMMARY:a & b line should be SUMMARY:a &amp; b.
(Reporter)

Comment 14

7 years ago
You are right indeed. The & sign needs to be escaped in an xml stream.

Nonetheless two problems remain

 a) After encountering an ampersand lightning silently aborts. Events that follow the bogus one will not being displayed and since there is no error message the user is not aware that he get's only a part of the calendar. There should be at least an error reported to the user; or better still error recovery mechanisms should kick in and still display the next valid entry.

b) Replacing the ampersand with the string "\n" (that is the literal string, consisting of the two characters '\' and 'n'). Leads to a newline being displayed in the tooltip (hovering the mouse over the event in the calendar view). The error log shows the "\n" however as does the package sniff. I don't know if this is a real problem but since I stumble uppon it I wanted you to be aware of it; especially given the fact that here input data seems to be feed to some routine which interprets character strings which might dangerous.

Comment 15

7 years ago
I agree.  Please see https://bugzilla.mozilla.org/show_bug.cgi?id=641665 which also contains the patch I currently run to avoid the "silent" failures.  Patch seems to be OK, so far.  If you're getting errors in the console, but no visible indication inside of thunderbird then most likely that patch will help you.

Comment 16

7 years ago
(In reply to m from comment #14)
> b) Replacing the ampersand with the string "\n" (that is the literal string,
> consisting of the two characters '\' and 'n'). Leads to a newline being
> displayed in the tooltip (hovering the mouse over the event in the calendar
> view). The error log shows the "\n" however as does the package sniff. I
> don't know if this is a real problem but since I stumble uppon it I wanted
> you to be aware of it; especially given the fact that here input data seems
> to be feed to some routine which interprets character strings which might
> dangerous.
This must again be a mod_caldav issue. I have tested this against my Apple Calendar Server and there the problem does not occur. It should probably be reported upstream at mod_caldav. Can you reproduce this with other clients than Lightning?

Comment 17

7 years ago
(In reply to Felix Möller from comment #16)
> (In reply to m from comment #14)
> > b) Replacing the ampersand with the string "\n" (that is the literal string,
> > consisting of the two characters '\' and 'n'). Leads to a newline being
> > displayed in the tooltip (hovering the mouse over the event in the calendar
> > view). The error log shows the "\n" however as does the package sniff. I
> > don't know if this is a real problem but since I stumble uppon it I wanted
> > you to be aware of it; especially given the fact that here input data seems
> > to be feed to some routine which interprets character strings which might
> > dangerous.
> This must again be a mod_caldav issue. I have tested this against my Apple
> Calendar Server and there the problem does not occur. It should probably be
> reported upstream at mod_caldav. Can you reproduce this with other clients
> than Lightning?

Yes and no.  The root of the problem is still the & entity encoding in mod_caldav (bug already filed as per my previous post) that does not function, however, the error message being generated is by lightning.  Presumably the underlying XML parses pops when encountering the non-entity &, and then proceeds to generate a multi-line error message (IIRC the error message makes PERFECT sense when looking at it in the console), however, when displaying the error message as a tooltip only the first line is being displayed, this is def a thunderbird/lightning issue.  I agree that this portion should be addressed in lightning.  A few more people confirming the mod_caldav bug at http://sourceforge.net/tracker/?func=detail&aid=3468503&group_id=191888&atid=939185 might also be helpful in getting the mod_caldav bug sorted out.
(Reporter)

Comment 18

7 years ago
(In reply to Jaco Kroon from comment #15)
> I agree.  Please see https://bugzilla.mozilla.org/show_bug.cgi?id=641665
> which also contains the patch I currently run to avoid the "silent"
> failures.  Patch seems to be OK, so far.  If you're getting errors in the
> console, but no visible indication inside of thunderbird then most likely
> that patch will help you.

I don't get any recognizable error in the error log (with lightning.debug.* = on).
(Reporter)

Comment 19

7 years ago
Created attachment 591425 [details]
Example ics for the \t error.

Comment 20

7 years ago
Would you please be able to c&p the output that you do get on the console for us please?
(Reporter)

Comment 21

7 years ago
(In reply to Felix Möller from comment #16)
> (In reply to m from comment #14)
> > b) Replacing the ampersand with the string "\n" (that is the literal string,
> > consisting of the two characters '\' and 'n'). Leads to a newline being
> > displayed in the tooltip (hovering the mouse over the event in the calendar
> > view). The error log shows the "\n" however as does the package sniff. I
> > don't know if this is a real problem but since I stumble uppon it I wanted
> > you to be aware of it; especially given the fact that here input data seems
> > to be feed to some routine which interprets character strings which might
> > dangerous.
> This must again be a mod_caldav issue. I have tested this against my Apple
> Calendar Server and there the problem does not occur. It should probably be
> reported upstream at mod_caldav. Can you reproduce this with other clients
> than Lightning?

I just added Attachment 591425 [details] to clarify the behavior. Herein the \n is replaced by a \t.

I have not yet tried to reproduce this with another client a recommendation for a good one is requested. But wireshark tells me that the \t goes over the wire, just like it's supposed to. Additionally the '\' is not a character that must be escaped in XML. Hence this seems not to be a mod_caldav problem.

The \n is parsed by lightning and is transformed to a tab character in the summary.
(Reporter)

Comment 22

7 years ago
This is the output of the error console that is connected to lightning (I left out lines easily identified as mail related).

CalDAV: send: <D:propfind xmlns:D="DAV:" xmlns:CS="http://calendarserver.org/ns/" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <D:resourcetype/>
    <D:owner/>
    <D:supported-report-set/>
    <C:supported-calendar-component-set/>
    <CS:getctag/>
  </D:prop>
</D:propfind>
-
Warning: Use of getAttributeNodeNS() is deprecated. Use getAttributeNS() instead.
Source File: chrome://messenger/content/messenger.xul
Line: 0
-
Interface module loaded.
-
Initializing timer.
-
Timer made.
-
Registering observer for startup completion.
-
CalDAV: Status 207 on initial PROPFIND for calendar mwd@vsr
-
CalDAV: Authentication scheme for mwd@vsr is Basic
-
CalDAV: recv: <?xml version="1.0" encoding="utf-8"?>
<D:multistatus xmlns:D="DAV:" xmlns:ns2="http://calendarserver.org/ns/" xmlns:ns1="urn:ietf:params:xml:ns:caldav" xmlns:ns0="DAV:">
<D:response xmlns:lp1="DAV:" xmlns:lp3="urn:ietf:params:xml:ns:caldav" xmlns:lp4="http://calendarserver.org/ns/" xmlns:g0="DAV:" xmlns:g1="urn:ietf:params:xml:ns:caldav">
<D:href>/calendar/mwd/</D:href>
<D:propstat>
<D:prop>
<lp1:resourcetype><x:calendar xmlns:x="urn:ietf:params:xml:ns:caldav"/><D:collection/></lp1:resourcetype>
<lp1:owner><D:href>/calendar/mwd/</D:href></lp1:owner>
<lp4:getctag>4b759f64c23c0</lp4:getctag>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
<D:propstat>
<D:prop>
<g0:supported-report-set/>
<g1:supported-calendar-component-set/>
</D:prop>
<D:status>HTTP/1.1 404 Not Found</D:status>
</D:propstat>
</D:response>
</D:multistatus>
-
CalDAV: initial ctag 4b759f64c23c0 for calendar mwd@vsr
-
CalDAV: send: OPTIONS https://217.86.228.214:5006/calendar/
-
CalDAV: DAV header: 1,2,access-control,calendar-access, <http://apache.org/dav/propset/fs/1>
-
CalDAV: Server does not support CalDAV scheduling.
-
CalDAV: send(https://217.86.228.214:5006/calendar/mwd/): <?xml version="1.0" encoding="UTF-8"?>
<D:propfind xmlns:D="DAV:">
  <D:prop>
    <D:getcontenttype/>
    <D:resourcetype/>
    <D:getetag/>
  </D:prop>
</D:propfind>
-
CalDAV: recv: 
<D:multistatus>
<D:response>
<D:href>/calendar/mwd/</D:href>
<D:propstat>
<D:prop>
<D:getcontenttype>httpd/unix-directory</D:getcontenttype>
<lp1:resourcetype><x:calendar></x:calendar><D:collection></D:collection></lp1:resourcetype>
<lp1:getetag>"4b759f64c23c0"</lp1:getetag>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
<D:response>
<D:href>/calendar/mwd/bffbfee5-255e-44f6-a706-1695c5d56699.ics</D:href>
<D:propstat>
<D:prop>
<D:getcontenttype>text/calendar</D:getcontenttype>
<lp1:resourcetype></lp1:resourcetype>
<lp1:getetag>"4b759b7796040"</lp1:getetag>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>
-
CalDAV: send(https://217.86.228.214:5006/calendar/mwd/): <calendar-multiget xmlns:D="DAV:" xmlns="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <D:getetag/>
    <calendar-data/>
  </D:prop>
  <D:href>/calendar/mwd/bffbfee5-255e-44f6-a706-1695c5d56699.ics</D:href>
</calendar-multiget>
-
CalDAV: recv: 
<D:multistatus>
<D:response>
<D:href>/calendar/mwd/bffbfee5-255e-44f6-a706-1695c5d56699.ics</D:href>
<D:propstat>
<D:prop>
<lp1:getetag>"4b759b7796040"</lp1:getetag>
<lp3:calendar-data>BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Europe/Paris
X-LIC-LOCATION:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20120111T104655Z
LAST-MODIFIED:20120111T104659Z
DTSTAMP:20120111T104659Z
UID:bffbfee5-255e-44f6-a706-1695c5d56699
SUMMARY:a \t b
DTSTART;TZID=Europe/Paris:20120113T120000
DTEND;TZID=Europe/Paris:20120113T130000
END:VEVENT
END:VCALENDAR
</lp3:calendar-data>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>
-
aChangeLogListener=null
calendarURI=https://217.86.228.214:5006/calendar/mwd/ 
iscached=false
this.mQueuedQueries.length=15

Comment 23

7 years ago
For \t there is bug #686833 -- you may not use them. So \n is something different as it is valid RFC2445.

Good is the iPhone, iCal, and aCal for Android. There is a Windows fat client, but I cannot remember the name currently...

Comment 24

7 years ago
(In reply to Felix Möller from comment #2)
> reporter please answer the questions in comment #1. For me it works.

he answered you... Its a Synology Server.
I have a DS 409+ I can confirm the bug. Thunderbird don't create a entry if there is a & in the summary line.

Comment 25

7 years ago
(In reply to Christian Eichert from comment #24)
> (In reply to Felix Möller from comment #2)
> > reporter please answer the questions in comment #1. For me it works.
> 
> he answered you... Its a Synology Server.
> I have a DS 409+ I can confirm the bug. Thunderbird don't create a entry if
> there is a & in the summary line.

but I must also confirm my android also didnt create the E & K event :))
so it seems a Synology Server issue.
Whiteboard: [CalDAV server: mod_caldav]
You need to log in before you can comment on or make changes to this bug.