Open Bug 1864415 Opened 2 years ago Updated 5 months ago

Calendar invitation uses wrong organizer email adress (caldav calendar-user-address-set, not the address set for the calendar locally)

Categories

(Calendar :: E-mail based Scheduling (iTIP/iMIP), defect)

Product:
Calendar
Component:
E-mail based Scheduling (iTIP/iMIP)
Version:
Thunderbird 115
Type:
defect

Tracking

(Not tracked)

People

(Reporter: Zentrale-IT, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

I use Thunderbird 115 Calendar with CalDAV (Davical) and Kerberos authentication.
I have several calendars, a personal one and some group calendars, each with its own email address.
I have an appointment/event in a group calendar.
I invite others to this event.

Actual results:

When I open a event in a group calendar and invite other email addresses, the organizer is set to my email address.
All answers of the invited persons return to my email address instead of the email address of the group calendar.
When I change my personal email address in the CalDAV server, the organizer is set to this email address. That means, the organizers email address comes with each invitation from my personal settings in the CalDAV server and not from the connected email address of the calendar in Thunderbird.

Expected results:

The organizer should be the email address connected with the specific calendar in Thunderbird.

For the group calendar, in Thunderbird, check which email it is associated with (right click the calendar in the list of calendars)

Component: Untriaged → E-mail based Scheduling (iTIP/iMIP)
Product: Thunderbird → Calendar

We checked several time on several computers. The group calendars are associated with group emails. Only the personal calendar is associated with my personal email.
The organizer does not come from a setting within Thunderbird. When I change my personal email address in the CalDAV server, the organizer is set to this email address regardless what email is set at the calendar within Thunderbird.
I changed the email address in the CalDAV server to an email, I never used before and is not specified in my Thunderbird settings. The organizer is than set to the new email only set in the CalDAV server.

If Thunderbird doesn't even know about the address, sounds like it's the server that is changing/setting the organizer, not Thunderbird.

Yes, and I think it's a bug.
An invitation created with Thunderbird should use the settings of Thunderbird and not the settings from the CalDAV server.
Even if it is supposed to use the settings from the CalDAV server, than it should use the setting of the group calendar and not my personal calendar. In the CalDAV server each group calendar has it's own group email.
Thunderbird uses the email of my personal calendar regardless which calendar I use for the invitation.
That is a bug.
When I receive an invitation (by mail), Thunderbird uses the calendar associated with the email account. When I send an invitation, Thunderbird always uses my personal email (set in the CalDAV server) not the email associated with the calendar.

Perhaps a server bug, but how could it be a bug in Thunderbird? Thunderbird doesn't even know about that address, so it's not setting it.

If I use Thunderbird to see my calendars and I use Thunderbird to make an invitation, who else than Thunderbird is setting the organizer?
The window "new event" shows the organizer, before I save the invitation. This value is wrong as I described. Thunderbird is setting this field. So it is a bug in Thunderbird. And when I save the new event with invitation Thunderbird uses the wrong email to send the invitations.

hen I change my personal email address in the CalDAV server, the organizer is set to this email address regardless what email is set at the calendar within Thunderbird.

But you say ^^^ you tried changing the email (server side) to an email Thunderbird doesn't know about and then this email was used.

This indicates it's the server that doesn't use the organizer set by Thunderbird.

No, it means that Thunderbird asks the Server, which email to use, instead of using the email associated with the calendar in Thunderbird.
Maybe it is intended that Thunderbird asks the server instead using it's on associated email. But even then, Thunderbird asks the server for my personal calendar instead of the group calendar.
The server itself is not involved in creating the invitation as Thunderbird sends the emails for invitation.
That's why it is a bug in Thunderbird not a bug in the server.

You can check the network traffic of what Thunderbird sends (open the developer tools and see the network tab). I'm pretty sure it never asks the server any such thing.

Yes, it does.
Not when creating an invitation but when connecting the first time the CalDAV server (log in).
After searching through the network traffic, I found, that Thunderbird speaks with the CalDAV server when connecting to the calendar (on startup).

Thunderbird sends a propfind for the calendar (COLLECTION):

<D:propfind xmlns:D='DAV:' xmlns:C='urn:ietf:params:xml:ns:caldav'>
<D:prop>
<C:calendar-home-set/> <C:calendar-user-address-set/> <C:schedule-inbox-URL/> <C:schedule-outbox-URL/>
</D:prop>
</D:propfind>

CalDAV server answers with a propstat:

<propstat>
<prop>
<C:calendar-home-set> <href>/caldav.php/COLLECTION/</href> </C:calendar-home-set>
<C:calendar-user-address-set> <href>mailto:EMAILADDRESS@DOMAIN</href> <href>/caldav.php/COLLECTION/</href> </C:calendar-user-address-set>
<C:schedule-inbox-URL> ... </C:schedule-inbox-URL>
<C:schedule-outbox-URL> ... </C:schedule-outbox-URL>
</prop>
</propstat>

The variable "EMAILADDRESS@DOMAIN" from this respond (<C:calendar-user-address-set>) is used as organizer's email for invitation on the calendar "COLLECTION".

The question is, if this is the intended behavior. If so, then it should be made obvious, that Thunderbird uses the associated email of the calendar ONLY for receiving invitations and is using the CalDAV servers response for sending invitations.

I ran into problems because our new CalDAV server responses with the wrong email addresses in <propstat>. For us, there is no need for changing the behavior of Thunderbird. It is now clear for us. It would be nice, if this behavior is clearly explained in some documentation.

This bug report can be set to solved.

You're right - https://searchfox.org/comm-central/rev/2b5fa337801646b75106595af329cc36943ae37d/calendar/providers/caldav/CalDavCalendar.jsm#1871-1873

See Also: → 1587736, 1766796
Summary: Calendar invitation uses wrong organizer email adress → Calendar invitation uses wrong organizer email adress (caldav calendar-user-address-set, not the address set for the calendar locally)

Honestly I don't think this can possibly be the intended behaviour - after all, you are asked to set the corresponding email address for each calendar. Completely ignoring this cannot be the intended behaviour. That is not a documentation issue, IMO.

After some research I now think, there are two bugs in Thunderbird.
When subscribing to a new calendar resource on a CalDAV server, Thunderbird sends two PROPFIND requests to the calendar server.
The first PROPFIND asks for the OWNER and the CURRENT_USER_PRINCIPAL:

<D:propfind xmlns:D='DAV:' xmlns:C='urn:ietf:params:xml:ns:caldav' xmlns:CS='http://calendarserver.org/ns/'>
<D:prop>
<D:resourcetype/>
<D:owner/>
<D:current-user-principal/>
<D:current-user-privilege-set/>
<D:supported-report-set/>
<C:supported-calendar-component-set/>
<CS:getctag/>
</D:prop>
</D:propfind>

The response is the OWNER which is the requested principal (in other words the principal which owns the collection/resource) and the CURRENT_USER_PRINCIPAL which is the user trying to access the resource (in other words the user who has the rights to access the resource).

In my words: OWNER is the requested calendar (/caldav.php/groupcalendar/), CURRENT_USER_PRINCIPAL is me (/caldav.php/mycalendar/).

The second PROPFIND asks for the CALENDAR-USER-ADDRESS-SET:

<D:propfind xmlns:D='DAV:' xmlns:C='urn:ietf:params:xml:ns:caldav'>
<D:prop>
<C:calendar-home-set/>
<C:calendar-user-address-set/>
<C:schedule-inbox-URL/>
<C:schedule-outbox-URL/>
</D:prop>
</D:propfind>

The response is the CALENDAR-USER-ADDRESS-SET which contains the email of the requested resource.

The first BUG of Thunderbird is, that Thunderbird asks for the CALENDAR-USER-ADDRESS-SET of the CURRENT_USER_PRINCIPAL instead of the OWNER. This has to be changed!

Thunderbird sets the value of the email associated with the calendar to the CALENDAR-USER-ADDRESS-SET when subscribing to an new calendar. This value can be changed manually at the calendar object in Thunderbird. That's pretty fine.

The second BUG of Thunderbird is, that Thunderbird uses the CALENDAR-USER-ADDRESS-SET to determine the organizer of an invitation. It should use the associated email of the calendar, because this value may be changed manually.

Also it's off topic here, I patched our DaviCAL Server Version 1.1.12 to return OWNER instead of CURRENT_USER_PRINCIPAL when asked for CURRENT_USER_PRINCIPAL. The changed code is from DaviCAL Server Version prior to 1.1.4 where DaviCAL had a bug.
This changed the behaviour of Thunderbird and the invitation ist now sent by the email of the group calendar not by the user's email.

davical/inc/DAVResource.php

 case 'DAV::current-user-principal':
    // Original code: return the CURRENT_USER_PRINCIPAL (session)
    // $prop->NewElement('current-user-principal', $reply->href( ConstructURL(DeconstructURL($session->principal->url())) ) );
    // CHANGED CODE: return the OWNER instead (request)
    $prop->NewElement('current-user-principal', $reply->href( ConstructURL(DeconstructURL($request->principal->url())) ) );
    break;
Status: UNCONFIRMED → NEW
Ever confirmed: true
Duplicate of this bug: 1884656

(In reply to Magnus Melin [:mkmelin] from comment #8)

Bug 1587736?

Indeed this looks very similar to me. See my use case there (shared calendars on a NAS), similar problem.

Please try beta, as bug 1587736 is now fixed.

The fix looked promising, and maybe it resolves other cases, but sad to say, for my case I saw no improvement.
I tested Beta 135b02 on Windows:
I created a new event in the calendar, invited one person.
Old problem: For the organizer as well as for the first line in the invitees list (which is identical to the organizer), an invalid string is used for the email address. TB seems to use something derived from my user login name for the caldav server as the email address, instead of the email address of the email account that I have explicitly assigned to my calendar (in the properties* dialog of the calendar).

I am not sure whether this might be an issue with the specific setup of my caldav server. Reading above findings by "Zentrale IT" in comments 11, 14, it might be the case that my server returns a false email address string. Though, I wonder. In the calendar properties* dialog, the selection list (dropdown box) shows the correct email address, in the format "Firstname Lastname <user@domain.com>".
WHY doesn't TB simply use this email address? It is there, and I have selected it.

(In many offices, this bug would be a dealbreaker for using TB at all. I would so much wish that I could donate explicitly for fixing such old and important bugs, and I think my client would be willing to do so. Where is the right place to rant about that?)

We testet Beta 135b02: The problem ist still there.
We recently testet Beta 141b02: The problem ist still there.

This BUG is still not fixed, not even addressed. Our Comment 14 (first Bug) describes the problem an the solution.
"The first BUG of Thunderbird is, that Thunderbird asks for the CALENDAR-USER-ADDRESS-SET of the CURRENT_USER_PRINCIPAL instead of the OWNER." This is done when activating the calendar resource mainly when starting Thunderbird.

Bug 1587736 has the same problem, as here described. But solved was only comment 10 in bug 1587736, which is no solution to the problem but something completely different. In my opinion even bug 1587736 is not solved.

We are also seeing this bug, or something similar, where TB is substituting an account alias for the 'Organiser' instead of the email specified as being associated with a calendar in TB.

The account alias appears when an invitation to a calendar event is sent to a third party email recipient.

Viewing network traffic on TB when sending the invitation, there is a <propstat> response from the calendar server (in this case iCloud) linking the email associated with the ORGANISER (and the organiser as 'ATTENDEE'), which is the email specified in the TB calendar, with a long string of text.

This long string of text is then used by Thunderbird as the email address for the 'Organiser' and also the name of as the the initially-specified attendee (aka the organiser) at the top of the the invitation which is emailed out to the third party email recipient.

This appears to be similar to the findings of "Zentrale IT" in comments 11 & 14, above.

The following is part of what is sent by the calendar server (with text removed in brackets):

ORGANIZER;PARTSTAT=NEEDS-ACTION;CUTYPE=INDIVIDUAL;EMAIL=(username)@(domain);CN=(name of organiser associated with email address):/(XXXXXXXXXXxxx etc - long string of text)/principal/
ATTENDEE;PARTSTAT=ACCEPTED;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;SCHEDUL
E-AGENT=CLIENT;EMAIL=(username)@(domain):/(XXXXXXXXXXxxx etc - long string of text)/principal/

TB then shows the the 'Organiser' of the event, when it tries to send an invitation email, as:
(username)@(domain) </(XXXXXXXXXXxxx etc - long string of text)/principal/>

and the first listed attendee (the organiser) as:
/XXXXXXXXXXxxx etc - long string of text)/principal/

It is then unable to send the email out, until the organiser is removed as a recipient.
Then it is possible to send the email out, however the organiser shows as as the string of text as listed above.

I am not an expert at all in this, and am only tracking through what Zentrale IT has so usefully set out above, however it appears to me that Thunderbird should just use the email address of the email account that that has been explicitly assigned to it in the calendar (in the properties* dialog of the calendar), as set out by Philippp above.

To add some more information to comment 22, above:

The purpose has been to sync an iCloud / iCal calendar (on an iphone) with a Thunderbird calendar on a PC.

Checking at the recipient end, it appears that even if an invitation to the event is not sent from Thunderbird, as the calendars are synced, the iCloud calendar is sending out an email invitation separately. This occurs whether the event is set up in TB or in the synced icalendar.

This invitation can then be accepted by the recipient. The acceptance message replies directly to the caldav.icloud.com server, and the acceptance to the invitation can be viewed on the calendar viewed on an iphone (or presumably any icalendar on an apple device logged in to the same account).

The acceptance of the invitation is not synced back to Thunderbird however, so you can't see on the TB calendar whether an invitation has been accepted (and in fact you cannot see that it has been sent, unlike how you can when using a TB only calendar, where the sent invitation shows up the in the 'sent' mailbox).

The invitation that appears in the recipient's inbox names the sender, but with the email listed as <noreply@email.apple.com>

In this case, one additional problem that arises is that a recipient doesn't see the sender's email address, to be able to enter into any further correspondence.

In an ideal world, we would like to see the invitation come from the senders email, and be visible as a 'sent' item in the TB sent mailbox, so that reply acceptance emails from recipients could also be seen in the TB inbox.

In that case, there would also not be a double-up of invitations that arrive in a recipient's inbox (one from the organiser's email send via the TB client, and one from apple - only the organiser's email via the TB client (or Imap linked webmail via browser) would be sent).

I realise I've now discovered another problem we need to solve, but thought I'd put it out there for any others considering similar issues.

We would still also like to see the original issue solved, as set out in the original post.

Looking at calendar/providers/caldav/CalDavCalendar.sys.mjs, the relevant code appears to be the handling of C:calendar-user-address-set.

The previous logic effectively used the last mailto: entry, because this.mCalendarUserAddress was overwritten in a loop. The patch in D218507 changes this to use the first mailto: entry. This is an improvement, but I do not think it fully solves the issue.

The first address is still dependent on the order returned by the CalDAV server. The user has no direct control over this order.

In my opinion, Thunderbird should select the organizer address in this order:

  1. If the calendar has an explicitly selected Thunderbird email identity and that address is present in calendar-user-address-set, use that address.
  2. Otherwise, if the CalDAV server marks one mailto: entry as preferred="1", use that address.
  3. Otherwise, fall back to the first mailto: entry.

This would avoid relying on arbitrary server order and would respect the user’s explicit calendar identity setting.

A possible approach could be:

const addrSet = response.firstProps["C:calendar-user-address-set"] || [];
const mailtoAddresses = addrSet.filter(addr => /^mailto:/i.test(addr));

let selectedAddress = null;

// 1. Prefer the Thunderbird calendar email identity, if it is one of the
// server-provided CalDAV user addresses.
// Pseudo-code: the exact property/API may differ.
const identity = this.getProperty("imip.identity");
const identityEmail = identity?.email;
if (identityEmail) {
  const identityMailto = `mailto:${identityEmail}`.toLowerCase();
  selectedAddress = mailtoAddresses.find(
    addr => addr.toLowerCase() == identityMailto
  );
}

// 2. Otherwise use the address marked preferred by the server.
// This requires that the XML parser preserves the `preferred="1"` attribute
// for entries in C:calendar-user-address-set.
if (!selectedAddress) {
  selectedAddress = findPreferredMailtoAddressFromCalendarUserAddressSet(response);
}

// 3. Otherwise fall back to the first mailto address.
if (!selectedAddress) {
  selectedAddress = mailtoAddresses[0];
}

if (selectedAddress) {
  this.mCalendarUserAddress = selectedAddress;
}

If the current representation of response.firstProps["C:calendar-user-address-set"] only contains strings and loses attributes such as preferred="1", then Thunderbird would also need to preserve those attributes when parsing this property.

The important part is the selection order:

calendar identity selected by the user > server preferred address > first mailto fallback.

This would solve cases where a provider exposes multiple valid addresses, such as a custom-domain address plus provider-internal aliases.

You need to log in before you can comment on or make changes to this bug.