428392 - Accepting invitation doesn't add event to calender - falsely reporting "not an attendee" if not received in default account

Status: RESOLVED FIXED

(Calendar :: E-mail based Scheduling (iTIP/iMIP), defect)

Description

[Peter Larsen]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.13) Gecko/20080325 Fedora/2.0.0.13-1.fc8 FireFox/2.0
Build Identifier: 0000000 (64bit build) 0.8

With multiple identities, it seems the wrong identify is used when trying to identify if the user is part of the invite as "Accept" is pushed.

In callTipProcessor.js line 166/167: 
  var replyStat = this._getReplyStatus(newItem, transport.defaultIdentity);

If the invite isn't received on the default identify the accept fails.

Reproducible: Always

Steps to Reproduce:
1.Create two identify accounts
2.Send an invite to the non-default identity
3.Try to click ACCEPT for the invite
4.Check error console
Actual Results:  
Error: [Exception... "'Error: _getReplyStatus: You are not on the list of invited attendees, delegation is not supported yet.  See bug 420516 for details.' when calling method: [calIOperationListener::onOperationComplete]"  nsresult: "0x8057001c (NS_ERROR_XPC_JS_THREW_JS_OBJECT)"  location: "JS frame :: file:///home/plarsen/.thunderbird/rb09ypxr.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/js/calStorageCalendar.js :: anonymous :: line 596"  data: no]
Source File: file:///home/plarsen/.thunderbird/rb09ypxr.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/js/calStorageCalendar.js
Line: 596

Expected Results:  
Event added to calendar (does not happen).
Using default setup calendar HOME - not using external calendars etc.

Comment 1

[cmtalbert]

This is an issue we need to deal with.  The code always uses the default identity set in Thunderbird and assumes the invitation was sent to that address.  It is an obviously invalid assumption to make.

--> CONFIRMING

Comment 4

[Balazs Lengyel]

If you receive the invitation as part of a mailgroup/list you will not have that list's address as any of your identities. E.g. In my company we receive 90% of invitations this way. This makes 0.8 for us completely unusable, so I would like this to be a critical bug.

So: you might correctly receive invitations that do not mention any of your identities, as stated in BUG 428274. 

Comment 5

[Bas van den Bosch]

Imho, we should definitely take this for 0.9. I have several mailaccounts which I get collected in one identity in TB. 

Comment 6

[Ziru]

Agreed with Balazs, this is a critical bug.  Lightning 0.8 is totally unusable, and I cannot add events for the past week!  Any idea of when 0.9 can get out?  If we need more than one week to get it out, can we roll out a quick fix, say, 0.8.1?  No lightning for the past week already messed up my daily work a lot.

Comment 7

[Ziru]

Tried to rollback to 0.7, but got a popup warning saying "The calendar data in your profile was updated by a newer version of Lightning, and continuing will probably cause the information to be lost or corrupted.  Lightning will now be disabled and Thunderbird restarted."   It looks scary.  Anybody knows about a quick fix?

Comment 8

[Ziru]

Attachment: patch for the chrome\lightning.jar!\content\lightning\imip-bar.js

Allows users to accept the event using the their email address even when their email does not show up as a calendar mail recipient.

Comment 9

[Ziru]

Attachment: patch for js\calItipItem.js

Allow adding as a new attendee even when the email does not show up as a calendar mail recipient

Comment 10

[Ziru]

Attachment: patch for js\calItipItem.js

Allow adding as a new attendee even when the email does not show up as a calendar mail recipient

Comment 11

[Balazs Lengyel]

I would also need a quick 0.8.1 patch for this !!!

Please everyone, besides commenting put your votes on this bug if you are interested.

Comment 12

[Philipp Kewisch [:Fallen][☁️📆]]

Ziru, thanks for looking into this. Rather than faking the default identity, I'd rather see a fix to correctly find the attendee in the list, or support delegation as in bug 420516. 

Unfortunately creating a 0.8.1 is not a "quick" process. It takes a while to do all the needed branching, tagging, building, QA, etc.

Nothing speaks against using a 0.9pre nightly as soon as this bug is fixed though.


Could one of you attach an email with an invitation that causes this? If you like, you can remove private email addresses, but when doing so, be sure to replace them with something unique, so we know which part to replace with our own emails when importing for testing.

Comment 13

[Craig Longman]

The duplicate bug 427876 has an invitation that I received.  The [e-mail] is the email address associated with the non-default account.  I could not accept that invitation.  By changing my default I am able to accept and schedule things fine, but I'm fortunate in only needing one identity to schedule with, so making it default was OK.

Comment 14

[Philipp Kewisch [:Fallen][☁️📆]]

Attachment: Allow non-default identities also for itip processor - v1

This patch should take care of a couple of issues related to the default identity. It should also make sure bug 423679 is fixed. Even though that bug was marked WFM, I could see why it might possibly have the wrong status.

The default identity should go away at some point, see comment in calItipTransport.

Comment 15

[Peter Larsen]

(In reply to comment #13)
> The duplicate bug 427876 has an invitation that I received.  The [e-mail] is
> the email address associated with the non-default account.  I could not accept
> that invitation.  By changing my default I am able to accept and schedule
> things fine, but I'm fortunate in only needing one identity to schedule with,
> so making it default was OK.
> 

heh - that worked as a work-around for me. Didn't even think of doing that.

Comment 16

[Ziru]

(In reply to comment #12)
> Ziru, thanks for looking into this. Rather than faking the default identity,
> I'd rather see a fix to correctly find the attendee in the list, or support
> delegation as in bug 420516. 

I tried to provide a quick fix, rather than a sound one.  I, as a Lightning user, don't even care whether it will cause other exceptions say removing events later (well, not good for sure, but ok if it will get expired), as long as it allows me to accept events invitations.  Of course, I'd love to see other people working on Lightning can propose a better fix.

Delegation is good for future release, but Lightning 0.8 is almost unusable for enterprise users.  That is why I think this issue is critical and deserves a quick fix on its own.

> Nothing speaks against using a 0.9pre nightly as soon as this bug is fixed
> though.

I don't know how many Lightning users will use nightly release.  I myself am OK with nightly release, but I doubt most enterprise users will be OK.

> Could one of you attach an email with an invitation that causes this? If you
> like, you can remove private email addresses, but when doing so, be sure to
> replace them with something unique, so we know which part to replace with our
> own emails when importing for testing.

I think Craig has already done this.  Here is the scenario.  Most of event invitations emails in my company, ranged from team meetings to all-hands meetings, are sent to mailing-list addresses rather than individual email addresses.  I guess many other companies follow this practice as well.

The membership info is kept by the corp mail server, and I don't think it is possible for Lightning to tell whether my mail identity belongs to such mailing list or not, based on the mail headers it receives on the client end.  

As to the delegation, I don't know what this feature refers to exactly, but this case may not fall neatly under the delegation category based on the naming.

Comment 17

[Ziru]

IMHO, it is better to make the quick fix as small as it can be.  After all, a smaller fix may indicate less chance to introduce new errors + unexpected side effects, and in turn only require little QA efforts, even though it is not perfect.  But if we do decide to fix it in 0.9, which I am sort of against, it doesn't matter too much.

Comment 18

[Philipp Kewisch [:Fallen][☁️📆]]

I'm sorry, but we don't have enough resources to leverage a 0.8.1 release. 0.9 isn't too far away, just wait a few months and the version will be released.

Keep in mind that releases mean less time for new features, since it takes a while to make all the needed steps happen (i.e release candidates, possible l10n, ...).

The patch I have delivered will fix default identitiy issues, and if you want a version now, then you can use a nightly. Enterprise users might not be interested in a nightly, but remember also we are still pre-1.0.

I'd suggest to move discussion into the newsgroups, bugs are reserved for technical discussion.

Comment 19

[Eric Valette]

I see you still do not get it: professional users do not care about fancy new features but would like to have something usable. So far the IPTIP/IMIP implementation is just unusable.

Comment 20

[Peter Larsen]

(In reply to comment #19)
> I see you still do not get it: professional users do not care about fancy new
> features but would like to have something usable. So far the IPTIP/IMIP
> implementation is just unusable.


Correct; however for "most" corporate users, like me, you have ONE email address - not multiple ones. The reason I have more than one, is that I use thunderbird to read private mail too. So the solution for "most" users is simple - make sure the default mail account is your work account. I get "mailling list" or basic group mail too, but they're still addressed to my email account in the "to:" so those kinds of invite _WOULD_ arrive. However - and this is something that I thin is missed here - corporate events are _not_ done as invites where I am. They're simply announced on the corporate calendar.

In other words, you use a different published calendar for corporate events - YOU merge that with yours, and that's how you make things happen. You don't expect 500 "accept" messages when there's an all-hands meeting.

So I would diagree that this issue is _that_ important. There is a simple work-around for "most" users, and while "most" can continue to use the product we can wait a few months for a correct install. The patch simply removes this problem completely, except to set the correct "from" email address when you accept. A small issue - not something that renders the calendar "as-is" useless.

I agree that the accept/reject functionality needs some working. I've had several other issues with the 0.7 version that I hope has been solved in the 0.8 - and if I can reproduce them in 0.8 I'll let the team know.

Comment 21

[Ziru]

(In reply to comment #20)
> Correct; however for "most" corporate users, like me, you have ONE email
> address - not multiple ones. 

Yes, I have only ONE.

> I get "mailling list" or basic group mail too, but they're still addressed to 
> my email account in the "to:" so those kinds of invite _WOULD_ arrive.

For me, no, those event invitation emails only have mailinglist addresses (for teams/depts/orgs) in the "to:" fields, but not mine (they will arrive at my mailbox of course, since i am a member of those lists).  That is where the invitation-accept-failure problems happen for me in 0.8.

Comment 22

[Balazs Lengyel]

I agree with Ziru; receiving but not being able to accept invitation received via mailinglists is what kills the 0.8 release. 
I downgraded to 0.7.  0.8 is unusable!

Comment 23

[Peter Larsen]

(In reply to comment #21)

> > I get "mailling list" or basic group mail too, but they're still addressed to 
> > my email account in the "to:" so those kinds of invite _WOULD_ arrive.
> 
> For me, no, those event invitation emails only have mailinglist addresses (for
> teams/depts/orgs) in the "to:" fields, but not mine (they will arrive at my
> mailbox of course, since i am a member of those lists).  That is where the
> invitation-accept-failure problems happen for me in 0.8.

Well, you took upon yourself to speak for "most" of the rest of us. I simply pointed out, that I for one isn't part of this "most". I also find the idea of mass invites to hundreds of people makes little sense and would indeed be a rare occasion - there are much better ways.

0.7 wasn't good to me either. It wasn't able to respond properly to invites if at all, and would intermittently add or not add events to my calendar. If it did it mostly did it with wrong timestamps. Bottom line to me is, that Lightning isn't ready for prime-time. That doesn't mean I won't keep trying to help making it become so - I like the product and the ideas I see. 

I fail to see the uproar though. If this is that urgent and important, I guess you should give a helping hand solving the actual problem?

Comment 24

[Balazs Lengyel]

Philip, 
Does your patch solve the problem for mailing lists as well or only for secondary addresses? When will your patch become part of the nightly build ?

Bad handling of Mailing lists is the showstopper in this bug! And yes
> mass invites to hundreds of people makes little sense
but inviting 5-20 people via mailing list is the usual way to arrange meetings for fixed groups. 

Comment 25

[Balazs Lengyel]

Philip thinks that the mailing list issue should be a separate problem, so please check bug 
Bug 428274 – Can't accept Invitation when received via a mailgroup since Lightning 0.8 
and vote for it.

Balazs

Comment 26

[Balazs Lengyel]

Please see my comment on BUG 428274.
https://bugzilla.mozilla.org/show_bug.cgi?id=428274#c4

Comment 27

[cmtalbert]

Comment on attachment 316449 [details] [diff] [review]
Allow non-default identities also for itip processor - v1

Philipp, thanks for taking this on.  Your changes look good, just a couple of nits below.

>Index: calendar/base/public/calIItipTransport.idl
>===================================================================
>RCS file: /cvsroot/mozilla/calendar/base/public/calIItipTransport.idl,v
>retrieving revision 1.1.2.1
>diff -u -U 8 -p -r1.1.2.1 calIItipTransport.idl
>--- calendar/base/public/calIItipTransport.idl	7 Mar 2007 20:21:59 -0000	1.1.2.1
>+++ calendar/base/public/calIItipTransport.idl	18 Apr 2008 15:18:14 -0000
>@@ -15,16 +15,17 @@
>  * The Original Code is Simdesk Technologies code.
>  *
>  * The Initial Developer of the Original Code is Simdesk Technologies Inc.
>  * Portions created by the Initial Developer are Copyright (C) 2007
>  * the Initial Developer. All Rights Reserved.
>  *
>  * Contributor(s):
>  *   Clint Talbert <[e-mail]>
>+ *   Philipp Kewisch <[e-mail]>
>  *
>  * Alternatively, the contents of this file may be used under the terms of
>  * either the GNU General Public License Version 2 or later (the "GPL"), or
>  * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
>  * in which case the provisions of the GPL or the LGPL are applicable instead
>  * of those above. If you wish to allow use of your version of this file only
>  * under the terms of either the GPL or the LGPL, and not to allow others to
>  * use your version of this file under the terms of the MPL, indicate your
>@@ -46,16 +47,19 @@ interface calIDateTime;
>  * by transports (eg: email, XMPP, etc.) wishing to send calIItipItems
>  */
> [scriptable, uuid(d9e439da-1014-42f9-be40-aa7e878b8a7d)]
> interface calIItipTransport : nsISupports
> {
>     /**
>      * Default identity for me within this transport. For example, this is
>      * your default email address in Thunderbird.
>+     *
>+     * XXX This should probably go away, using the default identity is far
>+     * from practical. Only user right now is calUtils's sendItipInvitation
>      */
>     readonly attribute AUTF8String defaultIdentity;
Yeah, when sending an iTIP item we should probably ask the user what identity to send the email from.  Note that this only has to be done when we generate a REQUEST or PUBLISH ourselves (i.e. we are the organizer).  This is used as a workaround for the fact that you can't set the organizer email address in the event dialog.  However, there are some smart defaults or configuration settings we could make so that the user could set up their preferred identity to use when sending meeting requests.  Also, as long as our default behavior is to show a compose dialog before sending, the user will have a chance to change the Send From identity via that window.

>Index: calendar/base/src/calItipProcessor.js
>+        // TODO This is email specific -> Move to transport
>         // When replying, the reply must only contain the ORGANIZER and the
>         // status of the ATTENDEE that represents ourselves. Therefore we must
>         // remove all other ATTENDEEs from the itipItem we send back.
>         if (respMethod == "REPLY") {
>             // Get the id that represents me.
>             // XXX Note that this doesn't take into consideration invitations
>             //     sent to email aliases. (ex: lilmatt vs mwillis)
>-            var me;
>-            var idPrefix;
>-            if (transport.type == "email") {
>-                me = transport.defaultIdentity;
>-                idPrefix = "mailto:";
>-            } else {
>-                throw Components.results.NS_ERROR_NOT_IMPLEMENTED;
>-            }
> 
>             var attendees = newItem.getAttendees({});
> 
>             for each (var attendee in attendees) {
>                 // Leave the ORGANIZER alone.
>                 if (!attendee.isOrganizer) {
>                     // example: mailto:[e-mail]
>-                    var meString = idPrefix + me;
>+                    var meString = "mailto:" + aItipItem.identity
Yeah, all of this should definitely be part of the transport.  Otherwise, we assume email.  As I said, please file a follow on bug for your TODO above and note it up in the comment

>     /**
>      * Helper function to get the PARTSTAT value from the given calendar
>      * item for the specified attendee
>      */
>     _getReplyStatus: function cipGRS(aCalItem, aAttendeeId) {
>-        var idPrefix = "mailto:";
>-
>-        // example: mailto:[e-mail]
>-        var idString = idPrefix + aAttendeeId;
>+        var idString = "MAILTO:" + aAttendeeId;
Here is another place where we assume mailto.  That also needs to be retrieved from transport.  Perhaps add an interface on the Transport interface called: getProtocol?  and that would return the URI header for the type of transport we have.  If such a header is not needed, then it could simply return a blank string.

>-        return attendee.participationStatus;
>+        return attendee && attendee.participationStatus;
Nice.

>+     * Centralized location for obtaining the proper transport. If a calendar is
>+     * specified, the transport type is taken from the provider. If the provider
>+     * does not specify a transport, or no calendar is specified, the default
>+     * email transport is returned.
>      */
>-    _getTransport: function cipGT() {
>-        // XXX Support for transports other than email go here.
>-        //     For now we just assume it's email.
>-        var transport = Components.classes["@mozilla.org/calendar/itip-transport;1?type=email"].
>-                        createInstance(Components.interfaces.calIItipTransport);
>+    _getTransport: function cipGT(aCalendar) {
>+        var transportType = (aCalendar && aCalendar.getProperty("itip.transportType")) || "email";
>+
>+        var transport = Components.classes["@mozilla.org/calendar/itip-transport;1?type=" + transportType]
>+                                  .getService(Components.interfaces.calIItipTransport);
>         if (!transport) {
>-            throw new Error("iTipProcessor cannot instantiate transport");
>+            throw new Error("iTipProcessor cannot instantiate transport" + (aCalendar ? "for " + aCalendar.type : ""));
>         }
>         return transport;
>     }
These changes rock.

>Index: calendar/base/src/calUtils.js
>-    organizer.id = "mailto:" + emlSvc.defaultIdentity;
>+    organizer.id = "mailto:" + transport.defaultIdentity;
Here again, we could use that getProtocol idea.  This needs to be in the transport.

So, in summary, file a follow on bug to get all MAILTO strings parametrized into the transport somehow, either through the getProtocol interface idea or some better idea.  And we also should probably file a follow-on bug for the defaultIdentity attribute on the transport interface so that we remove it (as you mentioned) once we have figured out how best to set that field for a new REQUEST.

So, r=ctalbert with those two follow on bugs.  Nice work, Philipp.

Comment 28

[Philipp Kewisch [:Fallen][☁️📆]]

Attachment: Allow non-default identities also for itip processor - v2

Carrying forth r=ctalbert

Followup bug 431126 and bug 431127 filed, "MAILTO" scheme already worked in.

Comment 29

[Philipp Kewisch [:Fallen][☁️📆]]

Checked in on HEAD and MOZILLA_1_8_BRANCH

-> FIXED

Comment 31

[Philipp Kewisch [:Fallen][☁️📆]]

There was a typo in calUtils.js, tranport instead of transport. Fix checked in.

Comment 32

[Ernesto Rey]

(In reply to comment #23)
> (In reply to comment #21)
> 
> 0.7 wasn't good to me either. It wasn't able to respond properly to invites if
> at all, and would intermittently add or not add events to my calendar. If it
> did it mostly did it with wrong timestamps. Bottom line to me is, that
> Lightning isn't ready for prime-time. That doesn't mean I won't keep trying to
> help making it become so - I like the product and the ideas I see. 
> 
> I fail to see the uproar though. If this is that urgent and important, I guess
> you should give a helping hand solving the actual problem?
> 

0.7 at least added invitations to my calendar, now I have to manually type them in. The ability to reply is not as critical, as it could be accomplished by just replying to the e-mail that comes in.

I do consider it critical. ALL my invitations are this way. It is totally unusable !!!

Comment 33

[Daniel Boelzle [:dbo]]

Ernesto, if iTIP invitations don't work anymore for you, please file a new bug and attach a (stripped) sample mail. Thanks!

Comment 34

[Balazs Lengyel]

Ernesto, I already have a BUG for the "mailing list issue": Bug 428274 – Can't accept Invitation when received via a mailgroup since Lightning 0.8. 
Please consider voting on it!

I also think this is absolutely critical and I hope someone will soon start fixing the bug. 

Comment 35

[Bas van den Bosch]

Ernesto and Balazs, if you see the invitation as an attachment, you can drag the attachment to the calendar-button. It creates the invitation. Saves you the trouble of typing in by hand or saving the attachment to the desktop and import it.

Comment 36

[Ernesto Rey]

(In reply to comment #35)
> Ernesto and Balazs, if you see the invitation as an attachment, you can drag
> the attachment to the calendar-button. It creates the invitation. Saves you the
> trouble of typing in by hand or saving the attachment to the desktop and import
> it.

Bas, thanks for the suggestion, I will try it. However the bug with the button should still be fixed as the button reads "accept" but no acceptance gets sent (it never has to MS Outlook invitations since I started using it several releases ago).  

Rgds - Ernesto.

Comment 38

[Stefan Sitter [:ssitter]]

Should this issue be added to the known issues section of the 0.8 release notes?

Comment 39

[Marcel Berteler]

Using 20080519 build on TB2

Invitation send from Lightning/TB2. Not sure which version, but will find out if required.

Can someone please explain:
I receive invitations on my default account, but it still does not add it to my calendar.
What exactly is the code looking for in terms of the default account. Is it only looking for the exact email address? And if so, where should this be found?

The following invitation is not accepted:

BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REQUEST
BEGIN:VTIMEZONE
TZID:/mozilla.org/20071231_1/Africa/Johannesburg
X-LIC-LOCATION:Africa/Johannesburg
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0200
TZNAME:SAST
DTSTART:19700101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20080521T111641Z
LAST-MODIFIED:20080521T112008Z
DTSTAMP:20080521T112008Z
UID:d97b7a00-8dff-41f5-9935-d00bb4a1c169
SUMMARY:Itinerary planning
ORGANIZER;PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT:MAILTO:
 [e-mail]
ATTENDEE;RSVP=TRUE;CN=Marcel Berteler;PARTSTAT=NEEDS-ACTION;
 ROLE=REQ-PARTICIPANT:MAILTO:[e-mail]
DTSTART;TZID=/mozilla.org/20071231_1/Africa/Johannesburg:20080526T171500
DTEND;TZID=/mozilla.org/20071231_1/Africa/Johannesburg:20080526T201500
LOCATION:Greenpoint
CATEGORIES:Projects
X-MOZ-SEND-INVITATIONS:TRUE
SEQUENCE:0
END:VEVENT
END:VCALENDAR

It looks like the CN only contains my name, while the MAILTO contains my default email address. Can this have anything to do with it?

Comment 40

[Marcel Berteler]

Now it becomes even stranger..... I can add this event to my calendar, but I can only add 1 until I restart TB again.

have filed a new bug for this: Bug 435424

Comment 41

[Peter Larsen]

For me, when accepting invitations work, it only works on local calendars. I never get it to put the event on my google calendar when accepting. The local one works now and then. Not consistently. Haven't been able to gather enough info to open a bug - so here's my 5cents worth of input.

Comment 42

[Bas van den Bosch]

@Marcel, this is bug 431536 I guess. Seeing comment 40, this now works for you? Peter, this is bug 430185 I think.

Comment 43

[Marcel Berteler]

@Bas, bug 435424 is indeed a duplicate of bug 431536.