Closed Bug 534190 Opened 15 years ago Closed 14 years ago

Acceptance is sent back as "tentative" to outlook users.

Categories

(Calendar :: E-mail based Scheduling (iTIP/iMIP), defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: edavid, Assigned: ssitter)

Details

Attachments

(1 file)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6
Build Identifier: lightning 1.0b1pre, buildID 20091208031515

When I accept an event, here is the kind of answer wich is sent (this
was captured on the wire).

--Boundary_(ID_ryU4ZdJoASiZ+Jo21dCbwA)
Content-type: text/calendar; method=REPLY; charset=UTF-8
Content-transfer-encoding: 8BIT

BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REPLY
BEGIN:VTIMEZONE
TZID:Romance Standard Time
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
LAST-MODIFIED:20091211T110910Z
DTSTAMP:20091211T110910Z
UID:040000008200E00074C5B7101A82E00800000000A09BC86F547ACA0100000000000000
 0010000000A95F57D9BAACB648887D56E5ED9B80D8
SUMMARY:Follow-up PCD plan
PRIORITY:5
STATUS:CONFIRMED
ORGANIZER;***
ATTENDEE;***
DTSTART;TZID=Romance Standard Time:20091215T110000
DTEND;TZID=Romance Standard Time:20091215T113000
DESCRIPTION;LANGUAGE=en-US:****
CLASS:PUBLIC
TRANSP:OPAQUE
SEQUENCE:4
LOCATION;LANGUAGE=en-US:***
X-MICROSOFT-CDO-APPT-SEQUENCE:4
X-MICROSOFT-CDO-OWNERAPPTID:833341401
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
END:VEVENT
END:VCALENDAR


--Boundary_(ID_ryU4ZdJoASiZ+Jo21dCbwA)--

--Boundary_(ID_qyG4ZdjoAsiZ+Jo19dCbWQ)
Content-type: application/ics; name=invite.ics
Content-transfer-encoding: 8BIT
Content-disposition: attachment; filename=invite.ics

BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REPLY
BEGIN:VTIMEZONE
TZID:Romance Standard Time
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
LAST-MODIFIED:20091211T110910Z
DTSTAMP:20091211T110910Z
UID:040000008200E00074C5B7101A82E00800000000A09BC86F547ACA0100000000000000
 0010000000A95F57D9BAACB648887D56E5ED9B80D8
SUMMARY:Follow-up PCD plan
PRIORITY:5
STATUS:CONFIRMED
ORGANIZER;****
ATTENDEE;****
DTSTART;TZID=Romance Standard Time:20091215T110000
DTEND;TZID=Romance Standard Time:20091215T113000
DESCRIPTION;LANGUAGE=en-US:****
CLASS:PUBLIC
TRANSP:OPAQUE
SEQUENCE:4
LOCATION;LANGUAGE=en-US:****
X-MICROSOFT-CDO-APPT-SEQUENCE:4
X-MICROSOFT-CDO-OWNERAPPTID:833341401
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
END:VEVENT
END:VCALENDAR


--Boundary_(ID_qyG4ZdjoAsiZ+Jo19dCbWQ)--

Recipient using outlook 2007 receives a "tentative" event, not
"accept". This might come form the
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
field.

Invitation was sent by outlook 2007.


Reproducible: Always

Steps to Reproduce:
1.send an invitation with outlook 2007
2.accept from thunderbird/lightning
3.
Actual Results:  
Sender sees a "tentative" answer

Expected Results:  
Sender should see an "accept" answer
User agent is really
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.5) Gecko/20091121 Lightning/1.0b1pre Thunderbird/3.0
Reproduced on Lightning 1.0b2pre with the same steps to reproduce.
User experience degraded.
I can confirm that this bug is present in lightning 1.0b2pre though my (incomplete) analysis is different to Erwan David's.

This is an example output (with just names and subject changed) of lightning that is mis-interpreted by Outlook as "tentative":

BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REPLY
BEGIN:VTIMEZONE
TZID:W. Europe Standard Time
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
LAST-MODIFIED:20100512T130146Z
DTSTAMP:20100512T130146Z
UID:040000008200E00074C5B7101A82E0080000000050036217E3F1CA0100000000000000
 0010000000D8A6F9C160D9C64BAEA33E42B1737958
SUMMARY:xyz
PRIORITY:5
STATUS:CONFIRMED
ORGANIZER;CN="Name1":mailto:name1@xyz.com
ATTENDEE;CN="Name2";PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT:mailto:name2@xyz.com
DTSTART;TZID=W. Europe Standard Time:20100520T121500
DTEND;TZID=W. Europe Standard Time:20100520T124500
DESCRIPTION;LANGUAGE=de-DE:Zeit: Donnerstag\, 20. Mai 2010 12:15-12:45 (GM
 T+01:00) Amsterdam\, Berlin\, Bern\, Rome\, Stockholm\, Vienna.\n\n*~*~*~*
 ~*~*~*~*~*~*\n\n\n\n
X-CALENDARSERVER-ACCESS:PUBLIC
CLASS:PUBLIC
TRANSP:OPAQUE
SEQUENCE:0
X-MICROSOFT-CDO-APPT-SEQUENCE:0
X-MICROSOFT-CDO-OWNERAPPTID:734812122
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
X-MICROSOFT-CDO-REPLYTIME:20100512T130143Z
END:VEVENT
END:VCALENDAR

Note that this example contains
ATTENDEE;CN="Name2";PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT:mailto:name2@xyz.com
and does not contain
ATTENDEE;CN="Name2";PARTSTAT=TENTATIVE;ROLE=REQ-PARTICIPANT:mailto:name2@xyz.com
but still it is not interpreted as intended.

And here is an example, produced by Outlook itself, that is correctly interpreted as "accepted":

BEGIN:VCALENDAR
METHOD:REPLY
PRODID:Microsoft Exchange Server 2007
VERSION:2.0
BEGIN:VTIMEZONE
TZID:W. Europe Standard Time
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ORGANIZER;CN="Name1":MAILTO:name1@xyz.com
ATTENDEE;PARTSTAT=ACCEPTED;CN="Name2":MAILTO:name2@xyz.com
COMMENT;LANGUAGE=de-DE:
SUMMARY;LANGUAGE=de-DE:Zugesagt: Aktualisiert: XYZ
DTSTART;TZID=W. Europe Standard Time:20100610T130000
DTEND;TZID=W. Europe Standard Time:20100610T160000
UID:040000008200E00074C5B7101A82E00800000000D06094EC54E1CA01000000000000000
 0100000003FAC07B5D6FFC94BBF1BB0F0230C23E7
CLASS:PUBLIC
PRIORITY:5
DTSTAMP:20100421T152921Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:1
LOCATION;LANGUAGE=de-DE:Raum:tbd
X-MICROSOFT-CDO-APPT-SEQUENCE:1
X-MICROSOFT-CDO-OWNERAPPTID:1536509914
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
END:VEVENT
END:VCALENDAR

Note that this contains the line 
ATTENDEE;PARTSTAT=ACCEPTED;CN="Name2":MAILTO:name2@xyz.com

Maybe lightning should give the "PARTSTAT=ACCEPTED;" part *before* the "CN=..." part? At least, also the examples in http://ietfreport.isoc.org/idref/rfc2446/ seem to suggest that the PARTSTAT=ACCEPTED is given before CN=...
The order of attributes shouldn't make a difference. See http://tools.ietf.org/html/rfc5545#section-3.1.1

Of course it may be that outlook somehow expects an order. We can't really influence the order since this is done in libical.
I too see the same TENTATIVE response when responding to invites from email regardless of whether I accept, decline, or mark tentative. 

When I have checked the sent email bodies for event emails, it appears that some (external) email invites are PARTSTAT=ACCEPTED, though not all.

I can supply email bodies if requested.
Could you try reversing the
CN="Name2";PARTSTAT=ACCEPTED
fields in the next beta, and we can see if that makes the difference?
It shouldn't hurt anything, leastwise.
This is not so easily possible, the ordering is done automatically by an external library.
The problem still exists in lightning-1.0b2
Could anyone please mark this bug as confirmed - and fix it, see below?

I simply manually reversed 
  CN="Name2";PARTSTAT=ACCEPTED
to 
  PARTSTAT=ACCEPTED;CN="Name2"
in a test reply message, but the interpretation was still "tentative".
So the problem had to be elsewhere.

After playing for two hours with all kinds of changes, 
I've finally solved the mystery - it's a very strict 
case-sensitive interpretation of the "mailto:" part!


Quoting from http://tools.ietf.org/html/rfc5545

  All names of properties, property parameters, enumerated property
  values, and property parameter values are case-insensitive.  However,
  all other property values are case-sensitive, unless otherwise stated.

Thus, the syntax element 'cal-address', which by definition is a 'uri', is supposed to be case-sensitive. The email I have in my sender info
happens to be mixed-case (David.von.Oheimb@...), 
but lightning converts this to david.von.oheimb@... in

  ATTENDEE;CN="von Oheimb, David";PARTSTAT=ACCEPTED:mailto:david.von.oheimb@...

And Outlook interprets any case mismatch between the mailto: part and the sender's address as TENTATIVE :-(
Even stronger, at least at some circumstances, if the email addresses do not match at all, Outlook does not even accept the email as a valid iCalendar data.

In a nutshell: this bug can be fixed simply by inserting the sender's email address into the ATTENDEE line *without* any case changes.
P.S. Isn't it kind of ironic that privacy/spam protection concerns prevented us from providing concrete email addresses in the bug reports, which in this case would have been essential for identifying the problem?

BTW, a proof that my above analysis is to the point (and a workaround for users of the existing lightning versions) is to convert the sender's address in the Thunderbird email account settings to lowercase.

Once the problem was identified, it took 10 minutes to pin down and fix the code responsible for it: line 239 of lightning-1.0b2/calendar-js/calItipItem.js
  aAttendeeId = aAttendeeId.toLowerCase();
which should simply be deleted or replaced by a comment, e.g.
    setAttendeeStatus: function ciiSAS(aAttendeeId, aStatus) {
        // Append "mailto:" to the attendee if it is missing it.
        aAttendeeId = aAttendeeId/*.toLowerCase()*/; //DvO MUST be case-sensitive; otherwise Outlook goes wrong.
        if (!aAttendeeId.match(/^mailto:/i)) {
            aAttendeeId = ("mailto:" + aAttendeeId);
        }

Unfortunately, I cannot attach the resuling patched .xpi here, which works fine.
Flags: wanted-calendar1.0?
Flags: blocking-calendar1.0?
Attached patch keep case-sensitive ID — — Splinter Review
Untested patch based on previous investigation by David von Oheimb.
Attachment #485486 - Flags: review?(philipp)
Thanks a lot Stefan for bringing my 'out-of-bound' bug fix into the real process!
Just note that the ".toLowerCase()" you added just before ".match(/^mailto:/i)" 
is entirely redundant/needless, as the match is case-insensitive (see 'i' flag).
How soon can you guys spin a release with this?
Right now I'm having to use another tool to reply to calendar messages, the "Tentative" responses were not well received :-(
If you trust me ;-) you may download the patched version I use from:
http://david.von-oheimb.de/tmp/lightning-1.0b2-DvO.xpi
Comment on attachment 485486 [details] [diff] [review]
keep case-sensitive ID


>-        aAttendeeId = aAttendeeId.toLowerCase();
>-        if (!aAttendeeId.match(/^mailto:/i)) {
>+        if (!aAttendeeId.toLowerCase().match(/^mailto:/i)) {
>             aAttendeeId = ("mailto:" + aAttendeeId);
>         }

I have no way of testing this, but given David reported this fix works, I'm fine with integrating it.
r=philipp with the toLowerCase() removed from the if().
Attachment #485486 - Flags: review?(philipp) → review+
Assignee: nobody → ssitter
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
OS: Linux → All
Hardware: x86 → All
Checked in with nit fixed: http://hg.mozilla.org/comm-central/rev/bbf09f7bab5c
Status: ASSIGNED → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Target Milestone: --- → 1.0b3
Flags: wanted-calendar1.0?
Flags: blocking-calendar1.0?
Has this been included in the 1.0 beta 5 release?
I can confirm that the fix is in 1.0 beta 5. 

I've just checked the resulting notification email and I'm puzzled because now *both* the attendee email entry and the sender email entry have somehow been converted to lower case.
Which also should be fine from the Outlook perspective ;-)
I sent a confirmation from 1.0 beta 5 and it still comes through as Tentative.
So the fix is in this release, but it looks like the fix did not solve the problem.
If you want to test, could you please send me a calendar item and I will respond to it?
Testing with the Exchange server and an Outlook client at our company,
the responses (sent by Lightning 1.0b5) are correctly interpreted.
Yet our Exchange server (not Lightning!) automatically converts the
email addresses of local users (in both the organizer and attendee role)
in *both* the From email header and in the ATTENDEE VCalendar entry to
lower case. For non-local users, the Exchange server converts only the
domain part of the email address in the From header to lower case but
leaves the ATTENDEE VCalendar entry as it is, which is inconsistent and
may contribute to the misinterpretation as "tentative".

So if in certain environments the misinterpretation remains, I am pretty
sure that this is because some mail server inconsistently converts either the
From email header or the ATTENDEE VCalendar entry to lower case. 
So in my view the bug is meanwhile only at the MickeySoft side.
One thing I've observed using Microsoft Outlook Express, is that if I click on "Accept" with "Send Response Now", my response comes through as tentative at the sender's end.
The default setting is "Edit the response before sending", which works okay.
So Outlook Express is formatting the reply differently in these two cases.
P.S. 
Carl asked:
this ATTENDEE VCalendar entry, is it the one attached to the note?
If it's the one attached to the note, could Lightning just reduce it to lowercase along with the other e-mail addresses? Maybe that's why the original Lightning code was reducing the e-mail address, it just didn't do it in enough places.

I answered:
Yes, I meant the entry sent in the body of an invitation/reply email.
Well, this would certainly solve your problem, but would be not conforming to RFC 2821; see for a nice discussion: http://email.about.com/od/emailbehindthescenes/f/email_case_sens.htm
For a clean solution, the bug should be fixed on the Exchange side.
The simplest workaround is to give up capitalizing the email address ;-)
When I send a response from Lightening, it pops up a box that says

Would you like to send a notification E-mail now?
[] Support Outlook 2000 and Outlook 2002/XP

Note that it's already offering a variance in support of Outlook.
It seems like the case change should be part of the Outlook support; if we don't want the case change, then we won't check the box.
(In reply to Carl Ponder from comment #23)
> When I send a response from Lightening, it pops up a box that says
> 
> Would you like to send a notification E-mail now?
> [] Support Outlook 2000 and Outlook 2002/XP
> 
> Note that it's already offering a variance in support of Outlook.
> It seems like the case change should be part of the Outlook support; if we
> don't want the case change, then we won't check the box.


Dialog is for specific outlook versions compatibility.
My bug was against outlook 2003, same thing with outlook 2010 which are clearly not covered by the dialog box choice.
Can we keep this active? I'd hardly call it RESOLVED/FIXED.
(In reply to Carl Ponder from comment #25)
> Can we keep this active? I'd hardly call it RESOLVED/FIXED.

Carl, can you please file a new bug report describing the current situation and how it should work from your point of view. This report is rather old and working from a clear starting point increases the possibility a developer will have a deeper look.
I still have this problem in tb15.0, however I found that changing my email address
in Thunderbird to all lower case made it go away.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: