505024 - .ICS text/calendar (VCal) in multipart/alternative is not visible, unless Lightning is installed

Status: RESOLVED WORKSFORME

(MailNews Core :: MIME, defect)

Description

[Michael Baffoni]

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.1pre) Gecko/20090718 Shredder/3.0b4pre

I have received a .ICS calendar meeting attachment, but no visual evidence of the invitation exists; no attachment item, nor text display within the message.  I used to receive these as just text within the message.

Here is the MIME type and beginning of attachment (from message source):
Content-class: urn:content-classes:calendarmessage
Content-Type: text/calendar;
	name="meeting.ics";
	method=REQUEST
Content-Transfer-Encoding: quoted-printable

BEGIN:VCALENDAR
METHOD:REQUEST
PRODID:Microsoft CDO for Microsoft Exchange
VERSION:2.0
BEGIN:VEVENT
DTSTAMP:20090717T002452Z
DTSTART:20090723T163000Z

Comment 1

[Michael Baffoni]

Is the change to component "Lightning" correct?  I don't have lighting installed, I'm using MeetingMaker to handle my calendar info.  Also, this issue is evident in Tbird safe mode so I'm sure it is not being affected by add-ins.

Comment 2

[Mark Banner (:standard8)]

Hmm maybe I read that wrong then.

Comment 3

[Magnus Melin [:mkmelin]]

Do you have View | Display Attachments inline checked?
Can you attach a sample .eml?

Comment 4

[Michael Baffoni]

Attachment: .eml of message with issue.

Calendar attachment does not appear even when display attachments inline is turned off.

Comment 5

[Michael Baffoni]

BTW, I tried to do some cleanup of the data within the calendar invite; if it doesn't work, let me know of an individual working on this issue and I can send the original to them so I don't post potentially sensitive information to the ticket.  However, I did open the email with the altered data and it still appears the same - no attachment and no text, with or without display attachments turned on using Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.1pre) Gecko/20090720 Shredder/3.0b4pre.

Comment 6

[Magnus Melin [:mkmelin]]

The sample shows just fine for me. (As an invitation text without lightning, an invitation with lightning.)

Comment 7

[Michael Baffoni]

Interesting.  I have two different machines (one XP 64bit, one Xp 32bit) and both don't show any attachment, even with attachment not displayed inline (they have completely different profiles, one is not a copy of the other).  

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2pre) Gecko/20090727 Shredder/3.0b4pre.  

I don't know if it matters, but I've never had lightning installed in my profiles, but they both are upgrades from 2.0 profiles.

Comment 8

[Magnus Melin [:mkmelin]]

You tried it as .eml file?

Comment 9

[Michael Baffoni]

Correct, I saved the file from the bug and double-clicked on it to replicate your experience.

Comment 10

[:Gijs (he/him)]

I just experienced this issue with TB 3.0b2 and then b4 (tried the upgrade to see if it'd help). I'm on Mac OS X 10.4. The profile used to be in use on TB2... so it looks like it is related to that. I don't recall having lightning installed - it certainly is not installed now. Are there any steps I could take to narrow down what the issue is? It seems like something one would want fixed before 3.0 is released --> requesting blocking3.0 for that reason -- attachments are just invisible, and it'd be better if it was at least an attachment or even visible as plaintext inside the email...

Comment 11

[Phil Ringnalda (:philor)]

The steps I'd start with would be:

* reduce the testcase to something more obvious, like just the text "I'm the text/plain part" "I'm the text/html part" "I'm the attachment description" to be sure that everyone will know what they are seeing

* make sure it's a regression - when I open the .eml in 2.0, I see the same thing I see in 3.0 - and if it is find the one-day window, which will do more good than anything else possibly could

* make sure whether it's associated with having used a profile in 2.0, or with having updated from 2.0, or both, or neither: does a new profile on 3.0 do the same? does a fresh install of 3.0b4 do the same? does a new profile created by the fresh install do the same? does it actually require having installed Lightning (in 2.0, or in 3.0? a particular version of Lightning?)?

* see whether it's Tb-only, or the same in a comparable build of SeaMonkey

Comment 12

[Dan Mosedale (:dmosedale, :dmose)]

Given reproducibility issues, we can't block on this bug as it stands today.  Please follow the suggestions in comment 11, and, once those issues are sorted out, if it still seems like it should be a blocker, please renominate.  Thanks!

Comment 13

[Bryan Laird]

Possibly related but perhaps not.

Even in the sample email that was submitted the message can be opened viewed and all the invite information is present.  It's not "lost".  Question I have is more that the invite correctly interpreted and displayed in the message offers no interface (that I see) where you could save or run the invite against an external calendar.

For example here on OSX I expected to see a .ics or calendar attachment that I could save to my desktop.  But that option doesn't exist instead if I wanted to enter this information into iCal I would have to manually create the invite.

This seen via TB3B4, fresh install no addons or anything.  I do see the same behavior in TB2 so (OSX again) I'm wondering if this is by design?  Is this a ''bug'' or an interface design decision?  Should I be able to save/view/ download the Calendar information from a message?

Assumption is that the same logic that interprets 'text and html' and doesn't provide and interface for "saving the Text Version" of a the original email and doesn't display a multipart message with text and html versions as having attachments is what 'hides' you from being able to save a calendar invite?

(mind you if this is seen entirely off topic from this posted bug let me know and I'll take my thoughts elsewhere)

Comment 14

[:Gijs (he/him)]

(In reply to comment #13)
> (mind you if this is seen entirely off topic from this posted bug let me know
> and I'll take my thoughts elsewhere)

I believe your problem is described in bug 242937 - this bug is about not seeing anything (so not even seeing the ICS data in the email text itself).

Comment 15

[Michael Baffoni]

Attachment: Reduced test case of email containing VCAL

Attaching reduced testcase.

Per comment 11, confirmed that this behavior is not a regression, the same behavior exists in Seamonkey, Tbird 2, and Tbird3; it exists in clean profiles or upgraded profiles.  

Lightning is not installed, nor has it been installed in prior versions either.

I played around with the .eml quite a bit, and essentially the VCAL doesn't show as text as long as the attachment type is text/calendar.  Messing with that at least allows the text to show, but clobbers the actual message text.

Comment 16

[Michael Baffoni]

Removing regression.  Can we mark this dataloss since there is no practical way in the standard UI interface to retrieve the data in the message short of saving the email as a file and cutting out the VCAL information manually (or is that considered a reasonable workaround)?  Half the time, I don't know the VCAL is there to go retrieving it from the message source....

I'm surprised that if Tbird doesn't understand the application type it doesn't at least just display it as an (unknown) attachment.  It would be reasonable to at least just save the meeting.ics as a standard text file....

Comment 17

[:Dossy Shiobara]

First, I'm glad I'm not the only one seeing this bug.

Without looking at the code (I'll be doing that shortly), my hypothesis is that Thunderbird only displays one text/* multipart MIME part, and Outlook invitations are sent with text/plain, text/html and text/calendar.  The code needs to be changed to "teach" Outlook that text/calendar should be presented as a file attachment, not "ignored" when displaying the text/html or text/plain.

Now, to see if I can come up with a fix ...

Comment 18

[:Dossy Shiobara]

Just thinking out loud ... it looks like src/mailnews/mime/src/mimemalt.cpp:MimeMultipartAlternative_display_part_p might be a starting point for the fix for this bug.

As an aside, won't line 255 return in a memory leak as *ct isn't PR_FREEIF'ed?

Comment 20

[John Peterson]

I just wanted to mention that I'm having the same issue with Thunderbird 9.1 on Mac OS X 10.7.

Comment 21

[John Peterson]

Correction Thunderbird 9.0.1.

Comment 22

[WADA:World Anti-bad-Duping Agency]

All of attached mails is multipart/alternative and text/plain, text/html, text/calender part are contained in it. Because of multipart/alternative, any part in it is ALTERNATIVE each other. Where can we see ATTACHMENT in multipart/alternative?

To see this special part in malformed mail as if attachment, two ways are currently available.
(i)  Lightning extension shows this text/calender as if attachment.
     (broken by Tb 8, but will be fixed by Tb 10. see bug 713380)
(ii) View/Message Body As/All Body Parts,
     with mailnews.display.show_all_body_parts_menu = true
     (implemented by Bug 602718. available since Tb 8)
     In order to see any part, any multipart/xxx is treated as multipart/mixed,
     and any part is shown as if attachment at attachment pane.    

Similar malformation is seen in multipart/related which was perhaps originally born by MS. Bug 674473 is for such malformed multipart/related case.
By that bug, "non-referred part and non-displayable part in malformed multipart/related" will be shown as if attachment.

Similar enhancement will be needed in case of wrong use of multipart/alternative by some mailers and some mail systems of some companies who don't respect mail RFC.
  - Limit application of "alternative" to mime part which Tb knows only.
    Apply to text/plain, text/html, and some predefined text/xxx only.
  - Show any other part as if attachment at attachment pane.
I believe this is natural enhancement if mail like next.
  multipart/alternative
    text/plain, text/html, application/pdf, audio/wav, video/x-mpeg
  In this case, any part can be actual/valid ALTERNATIVE, because PDF version
  of mail, voice version of mail, video version of mail is possible.
  Even if Tb can render text/plain or text/html only, I believe audio/wav etc.
  is better shown in attachment pane for user's convenience.
And, "doesn't choose single part only" and "doesn't ignore other parts than part of mail sender's highest preference order" is never RFC violation by Tb. 

This enhancement can do nothing for "reversed order in multipart/alternative" case. I think reversed order case is far rare than "text/calender in multipart/alternative" case and is relieved by View/Message Body As/All Body Parts. However, enhancement like next, partially ignore "mail sender's preference order", is better for multipart/alternative.
  - When View/Message Body As/Original/Simple HTML, search text/html part only.
    If only text/plain is contained or if text/html part is null/blank,
    use text/plain part.
  - When View/Message Body As/Plain Text, search text/plain part only.
    If only text/html is contained, or if text/plain part is null/blank,
    use text/html part and convert it to Text.

Comment 23

[Jim Porter (:squib)]

Taking. I have a fix in the works.

Comment 24

[Jim Porter (:squib)]

Attachment: WIP patch

This should fix the issue. Bienvenu, what do you think of this solution? Among other things, it special-cases text/calendar parts and provides a name for them so that they don't get skipped. As an alternative, we could add a displayableInline property to nsMsgAttachmentData and show all non-displayable-inline parts in the attachment list.

Comment 25

[Jim Porter (:squib)]

(In reply to Jim Porter (:squib) from comment #24)
> As an alternative, we could add a displayableInline property to nsMsgAttachmentData
> and show all non-displayable-inline parts in the attachment list.

Hmm, maybe we should do this, since apparently some MUAs actually send attachments as multipart/alternative subparts: bug 636253.

Comment 28

[Jim Porter (:squib)]

(In reply to Jim Porter (:squib) from comment #25)
> (In reply to Jim Porter (:squib) from comment #24)
> > As an alternative, we could add a displayableInline property to nsMsgAttachmentData
> > and show all non-displayable-inline parts in the attachment list.
> 
> Hmm, maybe we should do this, since apparently some MUAs actually send
> attachments as multipart/alternative subparts: bug 636253.

This actually works with the current patch.

One limitation, however, is that text/whatever parts will get shown inline after the "proper" body (so you'd have, e.g. text/html and text/whatever shown inline). This is because we try to find a handler for the *exact* MIME type when determining if we're at the last inlinable part, but then when we try to show it, we fudge things, and treat unknown text/* parts as text/plain. Maybe we should add an option to mime_create that forwards exact_match_p onto mime_find_class? This would also let me remove the "text/calendar" handling in mime_find_class. Ideas?

Comment 29

[David :Bienvenu]

Comment on attachment 613852 [details] [diff] [review]
WIP patch

I haven't tried the patch, but it seems like the pragmatic thing to do.

Comment 30

[Eliot Lear]

Hey Jim,

Thanks for taking this on.  I was the opener of Bug 301441 (I think), and while I don't have time to do a full build, if you have an image for MacOSX built, I'll be happy to test.  Just drop me a pointer (mozilla-bugs "the round A symbol" ofcourseimright.com).

Eliot

Comment 31

[Ian Neal]

I presume Lightning still picks things up correctly with this patch applied?

Comment 32

[Jim Porter (:squib)]

(In reply to Ian Neal from comment #31)
> I presume Lightning still picks things up correctly with this patch applied?

Well, that depends a lot on how strict you're being with "correctly". Lightning will almost certainly require a patch to work with this code, specifically to hide .ics attachments when it's handling them through a notification bar.

Comment 33

[Michael Baffoni]

I hope it doesn't.  I use a different meeting calendar, but I really appreciate the HTML markup view within Tbird that lightning does on the .ics, and still allow me to associate a different helper app to .ics instead of lighting "gobbling" it up.

Comment 34

[Philipp Kewisch [:Fallen][📱📆]]

(In reply to Jim Porter (:squib) from comment #32)
> (In reply to Ian Neal from comment #31)
> > I presume Lightning still picks things up correctly with this patch applied?
> 
> Well, that depends a lot on how strict you're being with "correctly".
> Lightning will almost certainly require a patch to work with this code,
> specifically to hide .ics attachments when it's handling them through a
> notification bar.

Could you elaborate wat kind of code changes are needed? I don't plan to remove the HTML representation of the ics attachment, I think this would go in the wrong direction. If the email message has its own html representation, then I could imagine providing a way to show alternative representations, but I would prefer showing the lightning HTML template first.

Comment 35

[Eric Davis]

This issue has really annoyed me for a while now.  I used to use mutt to handle the multipart/alternative calendar attachment until I came across this thunderbird addon:

https://addons.mozilla.org/en-US/thunderbird/addon/show-all-body-parts

It does the trick but requires an extra mouse/menu/click when viewing a meeting invite email.

FWIW, I really like how Claws Mail handles these multipart/alternative parts. It has a couple action icons on the right of the message window (one icon per multipart and others for real attachments).

Comment 37

[Rob Janssen]

What is the current status on this problem?

With SeaMonkey 2.10 a multipart/alternative message with text, html and test/calendar parts (as produced by Outlook and Google Calendar) shows as the text part in the mail pane and a "meeting.ics" file in the attachment pane which cannot be opened or saved.   When the "View -> Message Body as -> All body parts" trick is used, all parts are shown in the attachment pane and the meeting.ics can be opened or saved.
While cumbersome, it is usable as a workaround to access the meeting.ics (which we can import into the calendar app we use - not lightning)

However, in SeaMonkey 2.11 and 2.12 the attachment pane no longer shows up when a message like this is opened.   I'm sure I have read a bug (for Thunderbird) that discussed this change, but now I can no longer find it.

Unfortunately this means that our users no longer see the trigger to try the "all body parts" workaround when they receive a message (the nonfunctional meeting.ics attachment).  So we have been unable to upgrade past 2.10.
Today I installed 2.13beta and unfortunately it has not been fixed.

We need either the old behaviour which we can use with the workaround, or better the solution depicted above (which I understand will mean that we see the text part in the message pane and the text/calendar part as an attachment in the attachement pane where it can be saved without using "View -> Message Body as -> All body parts")

Any idea in what version we can expect this fix?  (e.g. the one in comment #25)

Comment 38

[Jim Porter (:squib)]

(In reply to Rob Janssen from comment #37)
> What is the current status on this problem?

My patch fails for all but the simplest test cases, and I haven't had time to fix it up. I'm unassigning myself from this for now, since I'm unlikely to work on any MIME parsing bugs until the new Javascript MIME parser is in place and turned on. If anyone else would like to take this on, I'll be happy to help provide guidance, though.

Comment 40

[Tony Hansen]

This is still a problem in version 15. I just filed Bug 792434 because I didn't find bug 505024 at first.

Comment 41

[Christopher Schultz]

I'd just like to put in a vote for this bug: I have messages that come from colleagues using WebEx and they are all multipart/alternative with the last alternative being text/calendar. Stupid. The icing on the cake is that the last part is base64 encoded, so even looking at the message source (which I'm totally willing to do) won't reveal the source of the message invite. Did I mention that WebEx doesn't put the date and time of the meeting into the text portion of the message? So I get these meeting invites all the time that say "come to my meeting" and I have absolutely no idea when the meeting is going to be. Stupid. Not tb's fault - to be sure - but insanely frustrating nonetheless.

I read the long and nasty saga of bug 674473 and I certainly wouldn't want to repeat that here. Jim, thanks for taking on the responsibility for both bug 674473 and for taking a swing at this one.

Jim: if your fix only works 5% of the time, would you feel comfortable releasing it so at least 5% of the time I can read these messages? Or is the other 95% such a horrible failure that it's worse than the current behavior?

I'm happy to provide raw material for test case email messages if that would be helpful.

Thanks to everyone on the tb team for years of great work: I've been using tb since before it had a name and have always been very happy with it. Keep up the great work!

Comment 42

[Jim Porter (:squib)]

(In reply to Christopher Schultz from comment #41)
> Jim: if your fix only works 5% of the time, would you feel comfortable
> releasing it so at least 5% of the time I can read these messages? Or is the
> other 95% such a horrible failure that it's worse than the current behavior?

The problem is that it breaks most messages that use multipart/alternative and emits a bunch of bogus "attachments".

Comment 43

[Christopher Schultz]

(In reply to Jim Porter (:squib) from comment #42)
> The problem is that it breaks most messages that use multipart/alternative
> and emits a bunch of bogus "attachments".

Fantastic. Thanks for the attempt.

I went ahead and enabled mailnews.display.show_all_body_parts_menu=true so now at least i can get tb to show me the data for those messages (decoded from base64, which is an enormous help as my base64 is a little rusty). I just need to be smart enough now to turn it on when I see a meeting invite with no meeting data evident in it.

Comment 44

[Sabuj Pattanayek]

I don't know about needing lightning or not, but this happens with 17.0.2 on Linux when .ics files are sent from exchange/outlook clients. Also see bug #713380

Comment 46

[joshfeingold]

Any idea when this bug might be fixed?

Comment 47

[rik belew]

Following Wayne,  i posted an example message over at https://bugzilla.mozilla.org/show_bug.cgi?id=783603

Comment 48

[Jonathan Kamens]

I just got bitten by this -- I got an email invite that I didn't even realize was an invite, and therefore I didn't respond to it in a timely fashion.

One can legitimately argue whether Outlook should be mixing up text/plain and text/html parts with a text/calendar part that has info in it that isn't displayed in the other two, but it is what it is, and Thunderbird really needs to cope with it better than it is.

Comment 49

[Sabuj Pattanayek]

arg hit me again today. The .ics part of the email doesn't show up, just the text part I guess which contains call in information for the meeting

Comment 50

[Patrick]

Hate to say me too, but me too.  Have missed a couple of meeting requests (sent by Exchange/Outlook users) since switching to Thunderbird and it's starting to be embarrassing.  In the short term, it would be great to just have an indication that there's attachments and/or a plain text view.  Happy to give examples / test cases / further detail if needed.
Thanks!

Comment 51

[luckyboy2000]

I'm having this same problem ever since I can remember using Thunderbird, years ago. Look guys, unfortunately MS/Outlook might not follow the industry standards or best practices, but is still what most people use out there. So more people will adopt Thunderbird ONLY when they can receive emails from people that don't use Thunderbird. I'd love to see this issue solved and be able to ditch the OSX Mail client (which by the way properly opens Calendar and other email attachments, even coming from Outlook).
Thanks in advance!

Comment 52

[Shula Amokshim]

i pinpointed the bug to a single line

Content-Type: multipart/mixed;
vs.
Content-Type: multipart/alternative;

the email is sent from a python code, and the attachment is visible both cases on gmail webclient.


Content-Type: multipart/alternative;      --> not showing attachment
vs.
Content-Type: multipart/mixed;         --> does show attachment

example eml file:
not showing attachment:   http://pastebin.com/33cvj9Pi
showing attachment:    http://pastebin.com/yjbgVK6w

Comment 53

[mhfrey]

I also have this problem.  I hate to be a pain, but any idea when there there will be a solution it has been 5 years and it is still marked new?

Comment 55

[Mitra Ardron]

Added my vote to this - this should be treated as a serious data-losing bug,  calendar invites used to be rare (between people in different organisations) since cross system compatibilities were very common, but they are very common these days as they work on every platform EXCEPT Thunderbird. Since the symptom is a blank message with no indication at all that it is a calender invite it means people lose data every day because of it. 

Note - I've had two invites, from people in different organisations this week, if I wasn't looking for this bug I'd have lost the invitation thinking they just randomly sent me a blank email, instead of going to Apple Mail to open the invitation.

Comment 56

[Brian Candler]

FYI I have the same issue with Thunderbird 31.2.0 under OSX 10.9.5. If someone sends me an Outlook / Office365 calendar invitation, there is no indication that an appointment is included.

I can confirm that when I view the message source, it has
Content-Type: multipart/alternative

Workaround is to set View > Message Body As > All Body Parts, but I still have to scan through the Outlook rubbish like

...
DTSTART:16010101T020000
...
DTSTART:16010101T020000
...

until I eventually get to the real appointment time:

DTSTART;TZID=GMT Standard Time:20141119T100000
DTEND;TZID=GMT Standard Time:20141119T160000

So the ideal would be (a) to have a built-in renderer for text/calendar, (b) invoke it *even* if the content type is multipart/alternative

Comment 57

[Mitra Ardron]

I really don't believe we need a renderer, that puts the solution in the too-hard-wont-get-done-for-another-five-years bucket.   Just a quick look at how these messages are handled by EVERY other mail client I've tried shows how to do this ... treat it as an attachment, i.e. from Outlook it should be treated as multipart/mixed, not multipart/alternative.

Comment 58

[Ian]

Ideally we need an attachment so we can open it with the calendar. A renderer would be a nice addition but you still need to be able to see the ics file. It's worth pointing out that the Lightning code already has a renderer.

Comment 59

[Ian]

Sorry, when I say "the calendar" I mean "the calendar of our choice". That may or may not be lightning.

Comment 60

[Deshke]

still not fixed in TB 33/34 :/

Comment 61

[Mitra Ardron]

Please .. lets not turn this into a major task of building a renderer, its an attachment it just needs to be visible as one so it can be opened, not silently, and invisibly ignored as currently.

Comment 62

[sicaine]

I have the same issue on Thunderbird 31.2.0. 'Fixed' it by installing Lightning :(

Took me a while and a few meetings to figure the issue itself out.

Comment 63

[Mitra Ardron]

yes - it really shouldn't take installing a Add-on that will actually confuse people who don't need another calender program in order to fix a DATA LOOSING BUG

Comment 64

[Ian]

Can the renderer simply be copied from Lightning?

Comment 65

[K Moser]

One more vote for adding the ability to save the attachment, please.

Comment 66

[Chris]

One more vote to fix this bug. This is extremely irritating given it affects us in a business environment.

Comment 67

[Thomas Peters]

+1 vote to get this fixed. No fix within almost six years is pretty embarrassing.

Comment 68

[:Gijs (he/him)]

I've just marked a *lot* of comments as me-too and/or advocacy. Please don't comment with "I'm seeing this too" or "when are you fixing it" or "This is a terrible bug and it is shameful it's not fixed yet".

We know all of that. Telling us again won't help - it just makes the bug far harder to scan and understand if people *are* actually interested in fixing it.



Now... Andrew/Mike, is the JS MIME parser thing something that ever happened? I can't find it from a quick search in bugzilla and MXR, but maybe I missed it (Andrew, poking you because of bug 447842 which looked related).

Irrespective of that, AIUI the easiest way to avoid the lack of clarity / visibility of the ics attachment is to ignore and/or replace-with-multipart/mixed any use of multipart/alternative if and only if there is a part that with mimetype text/calendar . Is that right, and if not, can you suggest an alternative (minimal) way to address this

If that sounds acceptable, where does that need to be addressed? Still mailnews/mime/src/mimemalt.cpp and friends, or elsewhere?

Comment 69

[Mike Conley (:mconley) (:⚙️)]

Yes, I believe the JSMIME code is now in the tree. jcranmer, can you confirm?

Comment 70

[Joshua Cranmer [:jcranmer]]

(In reply to :Gijs Kruitbosch from comment #68)
> Now... Andrew/Mike, is the JS MIME parser thing something that ever
> happened? I can't find it from a quick search in bugzilla and MXR, but maybe
> I missed it (Andrew, poking you because of bug 447842 which looked related).

JSMime has, in effect partially landed. It's already in place for most header parsing and emission (as of TB 32 or so).  It's *not* in place for parsing the body and message display--libmime is *very* featureful, so there is a very good deal of work that needs to get done to get it even in a state where it could be enabled by preference.

> Irrespective of that, AIUI the easiest way to avoid the lack of clarity /
> visibility of the ics attachment is to ignore and/or
> replace-with-multipart/mixed any use of multipart/alternative if and only if
> there is a part that with mimetype text/calendar . Is that right, and if
> not, can you suggest an alternative (minimal) way to address this

If you have Lightning installed, you probably want to treat text/calendar properly in a multipart/alternative. Otherwise, you'd want to treat it as a basic attachment.

> If that sounds acceptable, where does that need to be addressed? Still
> mailnews/mime/src/mimemalt.cpp and friends, or elsewhere?

Quite frankly, the multipart/alternative handling of libmime is a giant black hole, and the attachment handling is near as bad.

Comment 71

[Andrew Sutherland [:asuth] (he/him)]

I think Joshua addressed the JSMime stuff.  The API added for bug 447842 is something that could be used to work-around the problem (it gets to hear about all the MIME parts, not just what libmime emits for specific display), but it would require a second streaming of the message and hooking the result up to the message reader.

Pre-JSMime-conversion, the simplest 95% fix is probably to always treat non-displayable multipart/alternative leaf node subparts as attachments rather than trying to re-interpret multipart/alternative as multipart/mixed.  (Basically, if MimeMultipartAlternative_display_part_p returns false and it's not multipart/*, emit that it's an attachment).  It seems like this would avoid screwing up lightning, since lightning would indicate that the part is displayable.

Patch-size-wise, the fix is probably fairly small, but it's still probably a bit of effort to come up to speed on the libmime bits.  (So, not impossible, but a bit of a slog.  Although one might get really lucky with just cargo culting some of the attachment emitting bits.)

(I do need to very explicitly disclaim that I'm not able to help with writing the patch or reviewing because I'm swamped with the Gaia email app and think it's most productive for me to work on that instead.)

Comment 72

[Rohit]

Just confirming that it still exists and that its a real pain.  Enough to abandon thunderbird (I have been using it what feels like over 10 years).

I have tried all the related add-ons on the site (msg/eml/ics/winmail.dat) and others besides.  I have lightning installed and I have tried external applications.  Nothing works.  The problem is that most of the time I dont even know that someone has sent me an invite because it is not obvious that something is missing.

I do not want to import it into lightning, I just need the info visiible inside the email.  How hard can it be.,  Even Gamil works fine.  I can send sample emials if required

Comment 73

[Mitra Ardron]

I'm confirming what Rohit says that not only it exists, but the Add-Ons (like "Show All Body Parts") don't solve the problem.  Mac's email, and iPhone's have no problem handling these attachments IF you know they are there - usually the symptom is a blank message from someone, but i missed a meeting earlier this week because the message had a considerable amount of text, so I didn't notice something was strange. 

This is a Data-Loss-Bug and should be treated with the urgency that implies, but its been there five years now.

Comment 74

[Ben Bucksch (:BenB)]

Implementation suggestions:
Step 1: Let "text/calendar" be decoded as text/plain. That should be a 1-line change in libmime.
Step 2: Make a display feature similar to vcard. vcard is a libmime plugin (!). See mailnews/mime/cthandlers/vcard/mimevcrd.cpp/h for the code. Make sure that it still also shows as attachment that I can save as file.

Comment 75

[Ben Bucksch (:BenB)]

mimei.cpp line 479 has specific code for text/calendar. Apparently it's that what messes us up here. (I was already wondering, because unknown attachment types should normally show as file attachment. text/calendar leaves no trace whatsoever, which is the bug here. Apparently, it's caused by this code.)

Comment 76

[Ben Bucksch (:BenB)]

-> Neil. See the last 2 comments.

Comment 77

[Ben Bucksch (:BenB)]

Attachment: Hack to display it as plain text

The last WIP patch doesn't work for me, I still don't see the ICS attachment.
Also, reading the code, it also seems to do something that is wrong or at least debatable according to RFC 2046 5.1.4. <https://www.ietf.org/rfc/rfc2046.txt>:
"What is most critical, however, is that the user not automatically be shown multiple versions of the same data."

At least not normally, text/calendar is an exception due to a bug in Exchange. Exchange uses multipart/alternative, and does not put all information in the plaintext and HTML parts, it only puts the description there and not the date/time and place, so the plaintext part is useless.
As WADA already diagnosed in comment 22, this is a clear violation of Internet standards, which explicitly state that the "content of the various parts are interchangeable."

So, here is a HACK that implements step 1 above: display text/calendar as plain text. It's really ugly, because it's a data format for machines in Field: value form, not for humans. So, I don't consider it very useful for end users, but at least much better than status quo, because it does contain and display all data.

Comment 78

[Ben Bucksch (:BenB)]

Neil, please take a look whether you can migrate the Lightning text/calendar libmime plugin from Lightning to mailnews, so that text/calendar will be displayed even when Lightning is not enabled (addon disabled).

Comment 79

[Ben Bucksch (:BenB)]

Comment on attachment 8600315 [details] [diff] [review]
Hack to display it as plain text

This is a band-aid, but still better than before. Unpretty, but at least no information loss.
Tested without Lightning. Still needs testing with Lightning enabled. It should not change anything in this case, because plugins have priority over built-in types, so the Lightning plugin should still be used even with this patch.

Comment 80

[[e-mail]]

Attachment: Alternative approach

When I was looking into this last week (i.e. before the last few comments above) I noticed that we simply display the last alternative that we can, and ignore subsequent alternatives, even though strictly speaking they should be a better format. I therefore came up with this approach which shows those alternatives as attachments (and they show up in the attachment pane and also in the message list as having attachments). In this case this then allows you to launch the invitation using an external calendaring application.

Comment 81

[[e-mail]]

So, there are at least three cases:
1. Invitation sent as a text/calendar message
   In this case you see the invitation as plain text.
2. Invitation sent as multipart/mixed
   In this case you see the invitation as an attachment.
3. Invitation sent as multipart/alternate
   In this case the invitation is ignored.

Also, try toggling mailnews.display.show_all_body_parts_menu in config editor and then using View - Message Body As - All Body Parts. (Caution! This will start marking all your multipart messages as having attachments. The marking will go away when you switch display option and reload the message.)

Comment 82

[[e-mail]]

Attachment: Yet another idea

For completeness, this is a variant of attachment 8602813 [details] [diff] [review] whereby the additional multiparts are only attached if they have a file name (which applies in the attached vcal example).

Comment 83

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 84

[Ian]

Far too much time has passed with this and other calendar bugs. It's simply unusable in the current release state. I'm afraid we have now decided to drop Thunderbird and move our department onto eMClient on Windows and Apple Mail on Mac. So long and thanks for all the delays.

Comment 85

[Mitra Ardron]

Agreed Ian - six years, and a major data loss bug - still marked "New", it just shows that noone really cares about TB. 

Just missed another meeting this week because of a hidden calendar invite that I didn't even know was there. 

As calendar invites become really common, and TB is the only significant mail client that just ignores them I can see how it becomes untenable to keep using them, I'm about to jump ship myself.

Comment 86

[Eliot Lear]

While I share frustration over the length of time this bug is open, as I myself am not in a position to fix it, I have refrained from complaining.  And so to those who do, I would ask if they would volunteer some time to come up with a fix.  Several attempts have been made, and I would imagine since none of them were taken, that the problem is tricky and involved.  Anyway, thanks to those who ahve tried.

Comment 87

[Thomas D. (:thomas8)]

Neil, could you explain why this important bug is currently stalled?
What is needed/missing to get it moving again?

This has been elevated to "major" by rkent in 2014, 7 duplicates, 24 votes, and obviously biting people badly when they miss their invitations because Thunderbird doesn't show them (regardless of the fact the respective messages might be malformed by other widespread mailers, we should safeguard against that).

Comment 88

[Jim Porter (:squib)]

(In reply to Eliot Lear from comment #86)
> While I share frustration over the length of time this bug is open, as I
> myself am not in a position to fix it, I have refrained from complaining. 
> And so to those who do, I would ask if they would volunteer some time to
> come up with a fix.

Now would be a good time to remind folks: I'm happy to mentor anyone who'd like to take this bug on. Unfortunately, I don't have time to work on it myself, nor do I expect I will in the near future, but I should have time to at least point people in the right direction.

Comment 89

[markham breitbach]

I am also seeing this problem on OSX Yosemite.

Comment 90

[Mitra Ardron]

I can confirm that adding Lightning lets you view the invite in TB. Lightning itself unfortunately has plenty of bugs, so you can't download the invite, export it to iCal or do any of the other things which you would expect to do with a calender invite.  (See Bug #1217237 )

Comment 91

[jimimaseye]

V38.2 and still a problem.  I have users sending emails back to clients saying "your email is empty. Can you tell me the meeting date please?" and effectively making us look stupid because we couldnt see their .ICS invitation that they had actually sent.  And adding Lightning just to see these isnt the correct solution (its an unnecessary workround, and an unrequired addon for our business).

Comment 92

[Nick Hilliard]

After nearly 7 years of eating ICS attachments, I've given up on TB and have moved to Postbox. Losing ICS attachments was one of the main reasons (lightning is basically unusable).

Comment 93

[Mitra Ardron]

Note that in Lightning bug #1243407 the Lightning team confirm that there is no way to export single events in Lightning so I'm not sure why the Thunderbird team suggest that Lightning is the reason not to solve this 7 year old data-losing bug.

Comment 94

[Wayne Mery (:wsmwk)]

Neil won't be replying in the next several months. So to move this foward we have proposals in Neil's comment 80, 81 and comment 82.  Ben, what is your opinion of Neil's proposals?

Comment 95

[Rohit]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #94)
> Neil won't be replying in the next several months. So to move this foward we
> have proposals in Neil's comment 80, 81 and comment 82.  Ben, what is your
> opinion of Neil's proposals?

I know it wasn't addressed to me, but I think it sounds like a good proposal.

Comment 96

[Tony Hansen]

I would prefer a variation on comment 77, but instead of displaying as plain text, display the keyword-value pairs as an HTML table.

I found using mailnews.display.show_all_body_parts_menu (comment 81) to be totally unacceptable. I started using lightning as the workaround just to get the displays right.

But the solution expressed in comment 80 - comment 82 is certainly preferable to nothing being done.

Comment 98

[jimimaseye]

ITs all very well closing other bugs as duplicates of this bug, but if this bug is being progressed then they all might as well be closed....including this one.  Otherwise effectively its saying "Save wasting your time.  Yeah, we know about this problem.  We are not going to deal with it and dont need you telling us about it loads of times."

Comment 80 and 81 dated 7th May 2015 (1 year ago) and no progress since.  And the whole bug report approaching its SEVENTH birthday!

Yes, Im frustrated.  Sorry.

Comment 99

[jimimaseye]

...if this bug ISNT being progressed*

Comment 100

[William Pietri]

In case somebody needs a simple workaround just to deal with the very occasional Outlook user, here's what I did:

1. View message source; scroll down to "Content-Type: text/calendar" section
2. Copy the block of base64 text and decode it. (On Linux, I use base64 -d; there are also online decoders like https://www.base64decode.org/ ).
3. Save the resulting text (should start with "BEGIN:VCALENDAR" and end with "END:VCALENDAR") into a file. E.g., event.vcs.
4. Open up Google Calendar. In the left margin, click on the "Other calendars" menu and pick "Import calendar."
5. Select your file and click "Import". It should say, "Processed one event. Successfully imported one event."

After that, you should have the event on your calendar. This definitely isn't optimal, but it seems better than having to email back a client and ask them to send you details in plain text.

Comment 101

[jimimaseye]

"After that, you should have the event on your calendar. This definitely isn't optimal, but it seems better than having to email back a client and ask them to send you details in plain text."

Is it??!

Comment 102

[William Pietri]

As I thought was obvious, I mean that it seems better to me. If it isn't better for you, I will gladly offer you a full refund on your purchase price.

Comment 103

[jimimaseye]

Im concerned that offers of 'workrounds' (however short or long-winded) being posted on this bug report page only serves to further delay any potential real fix to the problem that MIGHT just be in the offing.  (Sort of a "oh look, it doesnt stop the show because there is something they can do without it so we will lower the priority even more" sort of thing). I know it shouldnt, but given we are approaching 7 years old with no solution on the horizon, it is easy to understand why such a concern should be had.  (Even proposals - comment 79 and onwards - just have "yes, I agree"'s yet no one willing to commit or progress).

(BTW, Im afraid your offer of the *simple* workround or even the virtual refund doesnt apply to me - Ive had to find a different solution due to lack of this functionality.  I would have really struggled to try to explain to 'Mary, in Accounts' and all the other users about "content-type's" and "base64 decoding") :-)

Comment 104

[William Pietri]

I'm exactly none of the developers, but I find your sense of entitlement offputting here. If you'd like things to worry about, you might try worrying that being demanding toward people who you are not paying and who are often volunteers might delay them working on this. In favor of, say, something where they get to deal with pleasant people, or just something they find personally rewarding.

On my part, all I did was try to help, and I've gotten two grumbly emails from some internet rando. So you've certainly convinced me that trying to help here is a bad idea.

Comment 105

[Jim Porter (:squib)]

As a developer who's worked on this area of the code (but unfortunately has no time to delve back into libmime at the moment), I can assure you that workarounds posted here have no bearing on the schedule for fixing this.

I may look into this when we're further along the process of migrating from libmime to jsmime, but probably not before then. Any significant improvements to libmime would be lost once we switch to jsmime for everything, and I'd prefer to spend my limited time on improvements that will last.

Comment 106

[Terje Bråten]

Hi Jim Porter.

I have recently done some other bug-fixes to libmime and the handling of multipart/alternative massages, and I was pondering if I should try to fix this bug.

You said in an earlier comment that you could be willing to mentor and point in the right direction.
So, please, any pointers would be welcome. Where should I start to look in order to fix this bug?

Terje B.

Comment 107

[Jim Porter (:squib)]

If I were you, I'd take a look at the existing patches, (especially the last two by Neil) and see how they work. They might already do what we need, but some tests would be nice.

Comment 114

[Jorg K (CEST = GMT+2)]

Terje, are you still interested, we could fix this ancient old bug. Without looking I'd say that Neil's patches don't apply any more since you've made changes in mimemalt.cpp.

BTW, I got here via Neil's review queue.

Comment 115

[Jorg K (CEST = GMT+2)]

Comment on attachment 8600315 [details] [diff] [review]
Hack to display it as plain text

I don't think you're going to get this review :-(

Comment 116

[Jorg K (CEST = GMT+2)]

In *HTML* view in a TB 53 Daily attachment 405560 [details] looks very nice, the last part is displayed: Content-Type: text/calendar. For attachment 389696 [details] also the last part Content-Type: text/calendar is displayed.

In both case one clearly sees that there is an attachment. This was most likely already fixed by Terje in bug 574989. I remember he added text/calendar here:
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.140

In plaintext view there is no attachment shown.

Terje, can we improve this further?

Comment 117

[Terje Bråten]

Hi Jorg.

I am sorry, but I am busy with work related stuff now.
It might be some time before I will look at this again.

I highly recommend that some one else looks at it, because I might never get around to it.

Kind regards

Terje B.

Comment 118

[Jorg K (CEST = GMT+2)]

I'm coming back to this bug after fixing bug 1334937.

Looking at attachment 389696 [details] and attachment 405560 [details].
With Lightning installed, both show the invitation. 
With Lightning disabled, both show the HTML part.

Looking at the messages they have three parts:
text/plain
text/html
text/calendar.

With Lightning not installed, the text/calendar part is not displayable, see bug 1334937 comment #39.

So I guess you can't have it both ways: Bug 1334937 wanted to see the HTML part and this bug here wants so see an attachment where there isn't one.

The test message from bug 1334937, attachment 8832838 [details], is different. It has those three parts *and* it has a .ics attachment, and that shows, with or without Lightning.

Comment 119

[Richard Shuford]

The previous comments show that Lightning is distrusted.

Comment 120

[Eliot Lear]

There are a few issues here.  The first is that when using mulitpart/alternative, one uses the preference order of the sender in the order that the receiver understands.  Thus, in your example, you act on text/calendar if you understand text/calendar, and otherwise move up the list to text/html.

The challenge here is that I don't believe the standard matches the user's expectations, particularly when lightning isn't installed.  In particular, the user may wish to view the appointment *and* "double click" on it to hand it off to a handler for Outlook or iCal.

Comment 121

[Jorg K (CEST = GMT+2)]

Right.

Bug 1334937 comment #46 explains that when you have three parts
text/plain
text/html
text/calendar
and *no* ICS attachment, the difficulty is to display the third usually hidden part additionally or as attachment. RFC 1521 says:
  Receiving user agents should pick and display the last format
  they are capable of displaying.  In the case where one of the
  alternatives is itself of type "multipart" and contains unrecognized
  sub-parts, the user agent may choose either to show that alternative,
  an earlier alternative, or both.

Comment 122

[Markus Kuhn]

This seems really a bug in Microsoft Exchange, not a bug in Thunderbird. However, since so many people send out event invitations via Exchange, it seems other MUAs have added special-case algorithms to deliberately violate the MIME specification if the sender was MS-Exchange and show multiple multipart/alternative parts simultaneously if one of them is an ICS calendar and the others are text/plain or text/html.

The problem is that Exchange sends out calendar invitations as

Example A:

- multipart/alternative
  * text/plain
  * text/html
  * text/calendar

even in cases where the text/plain and text/html parts are empty, or otherwise are no legitimate representation of the same information as the calendar file (e.g., just a human written cover message).

In an ideal world, where Microsoft respected Internet standards, Exchange would instead have sent out

Example B:

- multipart/mixed
  * multipart/alternative
    o text/plain
    o text/html
  * text/calendar

This way, the client would always display either the plain-text or HTML cover letter (which presumably are just alternative renderings of exactly the same information), plus show below that the ICS file as a proper attachment.

But since Exchange sends out so many inappropriate Example A messages, it may be worthwhile to add circumvention code. Thunderbird could detect the structure of Example A and convert it into the structure of Example B before displaying it. That transform can be made conditional to the presence of header fields (such as
regular expression /^x-ms-exchange-/i), such no messages sent my non-Microsoft products are affected.

I can forward example messages.

Comment 123

[Eliot Lear]

Regarding Comment 122, one of the use cases here is really quite simple: we want to be able to link a MIME handler to a text/calenar message.  That is what Bug 301441 was requesting lo those many year ago (marked a duplicate of this one).  And so it's not just a matter of whatever exchange bug might exist.  At the time, and I don't know if it has changed, it was not possible to register a MIME handler in the UI.

Comment 124

[Santiago Zarate]

Could it be that this bug is also related o that https://bugzilla.mozilla.org/show_bug.cgi?id=1396754 is a duplicate of this but with groupwise instead of exchange?.

Comment 125

[Gordon P. Hemsley [:GPHemsley]]

The Calendly.com scheduling service is affected by this problem.

It's not helpful to decry that there exist implementations that are not spec-compliant. If the reality is that this issue is so widely deployed as to be defacto standard, then it needs to be treated as such. (That said, I'm not entirely convinced that it's a violation of the standard—the e-mail is essentially saying that text/calendar is the preferred format of the message, and text/html and text/plain are provided as the fallback. The bug in the sender, then, is that these other formats don't have all the details that the text/calendar message has.)

Thunderbird should be updated, if not to support the calendar item wholesale (since Thunderbird doesn't have a built-in calendar), at least know enough to convert it to an ICNS attachment that can be downloaded and used by an actual calendar app.

Comment 126

[Eliot Lear]

Regarding Comment 125, This is a very old bug indeed.  For it to close, someone would actually have to want to work on it.  If you are willing to work on it, great.

Comment 127

[Sebastian]

Hi,

during the last year I have been faced with the same problem that Outlook appointments are not shown in my Thunderbird. As I did not want to use Lighting I spent some spare time during the Christmas holidays to hack together a small addon which shows a button in the message window when an outlook calendar part is being detected.

I have no experience in creating Thunderbird addons nor did I dive deep into the documentation. I learned from existing addons and put together something that works. I would be happy if someone could review and improve / rewrite the code.

The addon works for me, but I also did not that much testing. Before I wanted to publish this addon it would be nice if some more people could test it with more than my five test messages and only my Thunderbird installation.

You find my addon here: https://github.com/sebastianha/sfoa

Let me know your thoughts, create comments and merge requests.

Regards, Sebastian

Comment 128

[Jorg K (CEST = GMT+2)]

I've visited this bug far too many times :-( - So let's try a summary.

Attachment 389696 [details] has a multipart/alternative structure
multipart/alternative
  text/plain
  text/html
  text/calendar

The so-called "reduced" (not!) test case in attachment 405560 [details] has:
multipart/mixed
  multipart/alternative
    text/plain
    text/html
    text/calendar
but there is no further "mixed" part :-(

TB with Lightning shows the invitation, TB without Lightning shows the HTML part, no indication of any attachment since for an attachment we need multipart/mixed with the attachment in a "mixed" part.

Nothing new here, already explained in comment #122 (!!).

Now the add-on, I got this version:
https://github.com/sebastianha/sfoa/blob/master/dist/sfoa-v0.1.4.xpi
I briefly browsed the code (being an add-on reviewer as well) and it all looks good at first glance. It overlays the expandedHeadersBottomBox to display a "Download ICS" button. Somehow it doesn't work, I don't see the button using the test messages here.

If you can make it work, great, submit it at addons.thunderbird.net. Drop a comment here and I'll review/approve it. Thanks for your contribution ;-)

Comment 129

[Sebastian]

Hi,

thanks for your comments, I have to admit that I did not test with the attached mails here as I thought these kinds of appointment invitations are obsolet. I did only test with a recent one I got from a colleague.

However, please check v0.2.1 which should now support all three types of known outlook invitations (my sample and the two 
 attached here). All sample I used for testing are in the "samples" folder.

There might be encoding issues in the VCAL-file itself in some fields but at least it should be able to import into a calendar.

Regards, Sebastian

Comment 130

[Jorg K (CEST = GMT+2)]

Yes, that works fine. You need to know that you have to use Customise to place the button. You can submit this at ATN and let me know when it's ready so I can approve it. PM is OK. In general I'm a Lightning user, so it's not to terribly useful for me. That said, we have a long-standing issue that Lightning can only handle one attached event, bug 547754. Can your add-on save more than one? Or does Outlook only ever send one. For multipart/alternative, multiple text/calendar parts are imaginable.

Comment 131

[Sebastian]

Good to hear! I will submit the addon soon and let you know.

The hint for the button is quite hidden but inside the Readme. I added another when within the "Installing" section. I could not figure out a way to add a button which is not fixed but visible right after installing. Can anyone help me with this?

Currently it will only handle only one (the first found) calendar event. I do not know if its possible to send multiple appointments in one mail, I will ask a colleague which is using Outlook.

Comment 133

[c.o.hopkins]

Installed the addon but still no joy. I'll look out for further updates on this bug.

Comment 134

[Sebastian]

Hi,
if you like send me an email or open a bug on github: https://github.com/sebastianha/sfoa
Perhaps we can find a solution, until now I only a small sample of test messages. It could be that the message is not being detected correctly.
Regards, Sebastian

Comment 136

[Daniel AJ Sokolov]

The work-around with an add-on mentioned here is not compatible with TB 68.
Can someone with the power to do so please set a prority for this bug? It has been over a decade. Thank you.

Comment 137

[Worcester12345]

This actually used to work for quite a while. I was able to see a calendar invite, and click it to add to my calendar. This stopped working maybe a year ago? I don't know if it has to do with Thunderbird, or the sending application. Too bad there is no standard that all programs can follow.

Comment 138

[Daniel AJ Sokolov]

It is not correct that this is only for "Windows XP". In my case, it is Linux. So this bug likely affects all platforms. I couldn't fiugure out how to add Linux as an affected platform for this bug.

Comment 139

[Stéphane Gourichon]

Same as Daniel here. I experienced this bug on Linux.
Can reproduce it now by displaying sample e-mail from comment 4 ( https://bugzilla.mozilla.org/show_bug.cgi?id=505024#c4 ) on 68.10.0 (64 bits).

In my case I received automatic invitation e-mails which did not have any text, just the calendar entry. They appeared as totally blank e-mail, only showing the e-mail source code allowed me to realize there was a calendar entry.

Comment 140

[Markus Kuhn]

Dear newcomers to this issue. As explained above in comment 122, this issue is really a bug (violation of the MIME standard) in the calendar function of Microsoft Exchange that sends these messages. Regarding Thunderbird, this issue is a feature request for a workaround for a violation of the MIME standard by a Microsoft product.

For history buffs: The underlying issue is that Microsoft Exchange was originally designed to be an in-house X.400 mail server that added an Internet (RFC822+MIME) email gateway only later as an after-thought. X.400 only allows a flat list of attachments, but MIME allows (and requires in this case) a hierarchical tree of a multipart/alternative inside a multipart/mixed attachment. Microsoft still has not yet fully resolved all the ancient early-1990s legacy of the in-house X.400 mail product that eventually became Microsoft Exchange, and probably as a result their Exchange Calendar function keeps sending out malformed MIME. (To better understand the complex ways of how Microsoft Exchange internally maps RFC822+MIME messages onto their own X.400-style message objects, see [MS-OXCMAIL]: RFC 2822 and MIME to Email Object Conversion Algorithm. Warning, this is not for the faint hearted. Don't use Microsoft Exchange if what you really want is proper RFC822+MIME email. It never was. Spread the word.)

Repeat: this is a feature request for a workaround of a long-standing Microsoft Exchange bug. This is not a Thunderbird bug.

Comment 141

[Worcester12345]

It seems that the defacto is that this should work. Maybe it is time to update the MIME standard to agree with what is reality.

Meanwhile, Thunderbird users are left holding the short straw.

Comment 142

[Stéphane Gourichon]

Thanks @MarkusKuhn for taking care of explaining.

I understand that it is vexing when users complain about a problem caused by a bug in a third party program.

Still, thunderbird not displaying any hint feels wrong.

I'm also aware that https://en.wikipedia.org/wiki/Robustness_principle can sometimes hurt. I guess the question can only be resolved pragmatically, like "Do we have the power to force Microsoft Exchange to fix that?", "Does implementing the wish cause costs, like extra code complexity?" or "What happen if we refuse to do anything about this problem experiences by users?".

Anyway, thanks again Markus for enlightening us.

Don't use Microsoft Exchange if what you really want is proper RFC822+MIME email. It never was. Spread the word.

I'm already self-hosted and since the hosting company of my freelancing activity switched to MS Exchange I'm considering to self-host that, too, for what it's worth.

Repeat: this is a feature request for a workaround of a long-standing Microsoft Exchange bug. This is not a Thunderbird bug.

I'm willing to change the field "Type: defect" to " Type: feature request" but Bugzilla does not let me do that.

Comment 143

[Ben Bucksch (:BenB)]

the ancient early-1990s legacy ... Exchange

I bet even today's Microsoft engineers fear that code.

violation of the MIME standard in the calendar function of Microsoft Exchange that sends these messages

Right. "Microsoft doesn't respect Internet standards" isn't new. Thunderbird always had the policy to treat this as bug and fix it.

The reality is that people working in companies send these invitations, to people inside and outside the company, and if you are unfortunate enough to use Thunderbird without Lightning, you don't even see the meeting time or location. Guess who the meeting organizer will blame? Try arguing with him that Exchange has a bug.

---

That said, I know (comment 118, and from my own testing), the bug appeared only if you use Thunderbird without Lighting. Since Thunderbird 78, Lightning is integrated into Thunderbird core. Can somebody reproduce this bug in Thunderbird 78 with Lightning enabled?

Comment 144

[Daniel AJ Sokolov]

Yes, this is malformed MIME. Thunderbird should be able to handle it. It does not, so it's a bug.

Iff I where 2 introduse malformation s IN my writting, u wud still understend. As you should.

Comment 146

[José M. Muñoz]

Since TB78 Lightning is always "installed" (integrated into TB), so this bug here can be closed. There is also bug 760412 which is about only showing the last text/calendar part of multipart/alternative.
To make it clear:
This bug here: Invite not showing when no Lightning installed.
Bug 760412: text/html part not shown in multipart/alternative if text/calendar comes last.

Comment 147

[José M. Muñoz]

FIXED in TB 78 be integrating Lightning, so it is never "not installed".

Comment 148

[Magnus Melin [:mkmelin]]

->WTF (easier for tracking), and basically true as well.

Comment 149

[Eliot Lear]

One question: is it possible to call another MIME handler instead of Lightning? That was my original bug back in the dark ages, back when there were dinosaurs.

Comment 150

[Mitra Ardron]

@Eliot - I think for most users that would be a preferable solution - for those who want to avoid Lightning entirely.

For my own purposes I finally navigated the complex steps of integrating it so Lightning uses iCal - no mere mortal will ever succeed with those steps - but until that point one of the worst features of TB was failure to integrate with the same calendar used for all other functions on OSX, and given how much calendars are integrated into people's workflow these days must be a significant driver away from TB and towards Apple's Mail.

Comment 151

[Eliot Lear]

Ok, should there be a separate bug (again) for that?

Comment 152

[Worcester12345]

(In reply to Michael Baffoni from comment #0)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.1pre)
Gecko/20090718 Shredder/3.0b4pre

I have received a .ICS calendar meeting attachment, but no visual evidence
of the invitation exists; no attachment item, nor text display within the
message. I used to receive these as just text within the message.

Here is the MIME type and beginning of attachment (from message source):
Content-class: urn:content-classes:calendarmessage
Content-Type: text/calendar;
name="meeting.ics";
method=REQUEST
Content-Transfer-Encoding: quoted-printable

BEGIN:VCALENDAR
METHOD:REQUEST
PRODID:Microsoft CDO for Microsoft Exchange
VERSION:2.0
BEGIN:VEVENT
DTSTAMP:20090717T002452Z
DTSTART:20090723T163000Z

Michael Baffoni, is this REALLY "working for you"?

Comment 153

[Miguel J]

(In reply to Worcester12345 from comment #152)

Michael Baffoni, is this REALLY "working for you"?

Not working for me, since about one week:

• problem with message in MIME multipart, with invitation in .ICS
• I see a header saying "this message contains an invitation", I can accept it or not, but I cannot see its contents
• but the invitation is displayed in human-readable form if I select Display / Message body as / Plain text
  (maybe not exactly these messages, my Thunderbird is not in English)

Version: release, 91.3.0 (64 bits), Windows, French.

Comment 154

[Miguel J]

(In reply to Miguel Julier from comment #153)

(In reply to Worcester12345 from comment #152)

Michael Baffoni, is this REALLY "working for you"?

Not working for me, since about one week:

• problem with message in MIME multipart, with invitation in .ICS
• I see a header saying "this message contains an invitation", I can accept it or not, but I cannot see its contents
• but the invitation is displayed in human-readable form if I select Display / Message body as / Plain text
  (maybe not exactly these messages, my Thunderbird is not in English)

Version: release, 91.3.0 (64 bits), Windows, French.

Well, it actually works, but by default the event description is folded in (I just see the title), and I must click on the small arrow to unfold it.
This is not convenient at all since, when an invitation is sent, the receiver most likely wants to see the description.

Comment 155

[Gaetano Giunta]

I confirm the last comment from Miguel Julier: the event description is always folded in when the message is displayed, and having to click on the small arrow can get tedious and frustrating.
Still, I dare say this is working better now than in previous versions, as at least there seems to be some certainty that the invitation data will be shown within the message without having to go to extreme measures such as "display message body as plain text"

Comment 156

[Brian Candler]

I didn't realise until now that the event details were folded away, and became visible if the arrow is clicked. Thank you, this is awesome! It would be more awesome if it was open by default - but I don't know if this is Thunderbird's rendering of an ics file, or something controlled by Microsoft's HTML/CSS.

Getting invites from Office365: "View Message Body > All Body Parts" didn't show the event details, and "View Message Source" just showed a bunch of base64-encoded junk. So I wasn't able to see the event details at all, until I learned the click-to-expand trick.

Comment 157

[Bob Dingman]

(Using 91.4.1)
I agree with comments 154 through 156. Until I read these comments, I had no idea this arrow was clickable and am fairly certain I had tried clicking on it previously to no avail. (I likely missed it because it's so small.)
As the the other commenters said, hopefully we can get this description to show automatically.

Comment 158

[Ben Bucksch (:BenB)]

REOPENing. In all fairness, while it might work for developers who know what's happening, it doesn't work for end users as-is.
Even strictly speaking, the original problem "calendar (VCal) [content] is not visible" is not fixed.

Comment 159

[Magnus Melin [:mkmelin]]

We can track that in bug 1742101.

Comment 160

[Ben Bucksch (:BenB)]

Makes sense.

Comment 161

[brendel oliver]

Sorry, but I can't see where the arrow is to see the date and time of an invitation. I get buttons to accept / refuse or do other things (in the screenshot, if I find a way to add it, I have cliqued on "Plus" and then used the option "save a copy". However this will only create an event in the calendar, but as I have not idea as to the date ....

Comment 162

[brendel oliver]

Attachment: shows the top of an email with an invitation and the proposed options

I do not see in the proposed options of this email with an invitation, where I can see the actual date and time of the invitation.

Comment 163

[Gaetano Giunta]

Attachment: the small triangle which can be used to make the invitation data display

Comment 164

[Gaetano Giunta]

Comment on attachment 9257452 [details]
the small triangle which can be used to make the invitation data display

@oliver see the screenshot. The small arrow is highlighted in a red circle

Comment 165

[Brian Candler]

It's at the top of the body of the mail, just below the screenshot you showed:

|> (Person Name) has invited you to (Meeting Name)

There is a small black triangle next to "Person Name". The usability bug is that it shouldn't be necessary to point this out :-)

Comment 166

[brendel oliver]

thanks for the explanations, I would have never thought that the small black triangle was something clickable.
Perhaps this should e made more evident, usually clickable things are blue ... or underlined ... or have a downward arrow that indicates the direction in which something will be happening. But that's for others, I now know ;-)
On the other hand, once one has accepted the invitation, there is a "Details" button in the top blue frame and one also gets the details in this way

Comment 167

[Brian Candler]

thanks for the explanations, I would have never thought that the small black triangle was something clickable.

Me neither. Bug 1742101 is now dealing with this usability issue.