The default bug view has changed. See this FAQ.

text/calendar MIME part wrongly displayed when Lightning is not installed

VERIFIED FIXED in Thunderbird 54.0

Status

MailNews Core
MIME
VERIFIED FIXED
2 months ago
9 hours ago

People

(Reporter: Mike, Assigned: Jorg K (GMT+1))

Tracking

({regression, reproducible})

Thunderbird 54.0
Unspecified
All
regression, reproducible

SeaMonkey Tracking Flags

(thunderbird52+ fixed, thunderbird53 fixed, thunderbird54 fixed, seamonkey2.47 wontfix, seamonkey2.48 fixed, seamonkey2.49 fixed, seamonkey2.50 fixed, seamonkey2.51 fixed)

Details

User Story

Facts:
-------
a) Reproducible for Google Calendar Invitations
c) No problem in older SM because those versions did not 
   wrongly show .ics contents inline.
d) Valid regression, SM /B may not show .ics without having an
   integrated .ics viewer. 
e) Also TB Problem, with 'wessage body as full html" and Lightning 
   disabled TB Daily 53.0a1 (2017-01-09) (64-bit) also shows that
   .ics source code
b) Appeared with SM 2.46

More investigation required
---------------------------
./.

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

2 months ago
User Agent: Mozilla/5.0 (Windows NT 5.1; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46
Build ID: 20161213183751

Steps to reproduce:

I always received attachments .ics(with calendar invitations) in version 2.39 and was listed (rendered & previewed correctly). After upgrade to 2.46 it is not interpreted any more (no frame of calendar meeting and invited persons - pure not parsed at all message only).


Actual results:

I can't see email messages with .ics (calendar files) properly in 2.46.
I had to rollback to v2.39 where everythink still works ok with .ics calendar preview.


Expected results:

Please repair preview and rendering of ics files in 2.46 like it was working in 2.39 and earlier.
(Reporter)

Updated

2 months ago
Severity: normal → critical
Component: MailNews: General → MailNews: Message Display

Comment 1

2 months ago
Dupe of Bug 505024 ?

Comment 2

2 months ago
Not sure if it might be related or a dupe of bug 1326271. If yes should be fixed in the next version.
(Reporter)

Comment 3

2 months ago
I don't use any plugins in seamonkey and OS (Lightning or sth.)

Comment 4

2 months ago
(In reply to Mike from comment #3)
> I don't use any plugins in seamonkey and OS (Lightning or sth.)

I really really doubt that that is true.  2.39 showed something different to a simple text listing? I doubt. 
Please forward to me by email such an email which was rendered as a Lightning like calendar invitation box.
Flags: needinfo?(t50)
(Reporter)

Comment 5

2 months ago
Created attachment 8832492 [details]
Screenshots for people, who don't trust to the Dark Side of their bugs;-)

I attached the screenshots which ilustrates bugged rendering of invite.ics between 2.39(or older was OK) and 2.46(which is buggy).
Screenshots was made on the same e-mail when was previewed and opened in both versions - build version also included.
Flags: needinfo?(t50)
Comment hidden (off-topic)

Comment 7

2 months ago
I only ever saw such raw calandar entry mails when Lightning was not installed. There were a lot of changes in the mailnews code between 2.39 and 2.46. They might just be processed differently now. If would be good if you can attach a sample anonymized eml here.

Please refrain from further personal accusations.

Comment 8

2 months ago
More or less REPRODUCIBLE with official  German SeaMonkey 2.46 FINAL  (Windows NT 6.1; WOW64; rv:49.0)  Gecko/20100101 Firefox/49.0 Build 20161213183751  (Default Classic Theme)  on German WIN7 64bit.

Reporter's problem is not related to standard .ics attachments, which AFAIK  never were visible with SeaMonkey.

But if I create an invitation for a guest from my Google Calendar, it will be shown similar to reporter's screenshots with SM 2.39, but 2.46 only will show  source code.

I will attach a sample file for further investigation
Severity: critical → normal
User Story: (updated)
Keywords: reproducible
Summary: No preview and no reading of email attachment files .ics in ver 2.46 - in 2.39 all worked OK → No preview and no reading of email attachment files .ics from Google Calendar Invitation

Comment 9

2 months ago
Created attachment 8832838 [details]
Invitation From Google Calendar (.eml)

Steps how to reproduce with SM 2.46:

0. Download Attachment
1. Launch SM email → Change 'View Message body' to simpe of full HTML if 
   necessary
2. Menu 'File → Open → Browse for Attachment → [open]
   » .eml will open in separate Window and show some .ics source

With 2.39 I see an invitation similar to reporter's screenshots

Comment 10

2 months ago
(In reply to Rainer Bielefeld from comment #9)
I forgot to mention in step 1: Disable Lightning, if necessary!

I think the core of the problem is that SM 2.46 shows the attached .ics (inline) without having an integrated .ics viewer.
SM must not show inline attachments which it can not render!

After some more investigation I will forward this one to MailNews CORE
User Story: (updated)

Comment 11

2 months ago
Also REPRODUCIBLE with attachment "Invitation From Google Calendar (.eml)" and  Server-Installation of  official  en-US SeaMonkey 2.50a1 (X11; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0 Build 20170118003001  (Default Classic Theme) on VirtualBox Ubuntu 14.04.1:

.ics wrongly shown inline.
So OS=All
OS: Unspecified → All
Summary: No preview and no reading of email attachment files .ics from Google Calendar Invitation → .ics attachment wrongly shown inline although no viewer available

Comment 12

2 months ago
b1): With steps from comment 9
     Already REPRODUCIBLE with  English SeaMonkey 2.46a2  (Windows NT 6.1; WOW64;
     rv:49.0)  Gecko/20100101 Firefox/49.0 Build 20160723030242  
    (Default Classic Theme)  on German WIN7 64bit
b2): Still ok with  English SeaMonkey 2.45  (Windows NT 6.1; WOW64; rv:48.0) 
    Gecko/20100101 Firefox/48.0 Build 20160710233343  
    (Default Classic Theme)  on German WIN7 64bit 
b): So probably appeared with MailNews Core 49
User Story: (updated)
Component: MailNews: Message Display → Attachments
Product: SeaMonkey → MailNews Core
Version: SeaMonkey 2.46 Branch → 49

Comment 13

2 months ago
f) No obvious DUP found with <https://bugzilla.mozilla.org/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=DUPs1334937&sharer_id=41036>
Status: UNCONFIRMED → NEW
Ever confirmed: true

Updated

2 months ago
User Story: (updated)
(Reporter)

Comment 14

2 months ago
Thank You Rainer for Your professional analysis. 
I can't attach You genuine ics's becouse I preview "restricted to me invitations" from Google Mail System.
I could try anonymize it in source maybe, but not now.
All invitations "rendering" worked in Seamonkey versions 2.39 & earlier with coloured borders and was OK.
After direct upgrade to 2.46 rendering doesn't work.
I didn't know if I need any plugin or something to render this or something has broken.
I enetered this as a Bug becouse rollback to 2.39 helped me to recover .ics rendering functionality.
If You have any solution for me to this or safe plugin to install for viewing in 2.46 - it is welcomed.
Thank's.
(Assignee)

Comment 15

2 months ago
This is basically bug 505024. However, the behaviour of the system changed a bit in bug 574989 which landed on TB 49. So TB 49 = SM 2.46 behaves differently to prior versions.

The problem is the MIME structure of the message. The top level is multipart/mixed so the first part is the message, and the second part the ICS attachment.

The message itself is multipart/alternative with three (!!) parts instead of two:
text/plain
text/html
text/calendar

For multipart/alternative the last (!) part should be displayed to get the richest display according to RFC 1521. Sadly, the last part is not the HTML but some funny text part.

Anyway, I'll attach a patch in a minute.
(Assignee)

Updated

2 months ago
Component: Attachments → MIME
(Assignee)

Comment 16

2 months ago
For people interested in history:
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.136
(Assignee)

Updated

2 months ago
Blocks: 574989
Keywords: regression
(Assignee)

Comment 17

2 months ago
Created attachment 8832864 [details] [diff] [review]
1334937-remove-text-calendar.patch

OK, here's the patch. In order to understand it, you need to understand the prioritising we have implemented.

When the display is set to HTML (TB: View > Message Body As > Original HTML), we're looking for a rich part in the alternative section, regardless of the order.

text/plain is prioritised as "plain text" and text/html, text/enriched, text/richtext and sadly also text/calendar were prioritised as "rich text". The last rich part won, so in this case text/calendar over text/html.

Obviously this is wrong, since text/calendar is "plain text" MIME type and not a rich one. So just removing that from the group fixes the problem.

Voilà.

Now the bigger problem is getting this reviewed. FRG, can you do reviews?
Assignee: nobody → jorgk
Status: NEW → ASSIGNED
Attachment #8832864 - Flags: review?(iann_bugzilla)
(Assignee)

Updated

2 months ago
status-thunderbird52: --- → affected
status-thunderbird53: --- → affected
status-thunderbird54: --- → affected
tracking-thunderbird52: --- → +

Comment 18

2 months ago
>> Now the bigger problem is getting this reviewed. FRG, can you do reviews?

Occasionally for small things. I think this one qualifies and IanN will not shoot me. I can't approve any uplifts down to comm-release.
status-seamonkey2.47: --- → wontfix
status-seamonkey2.48: --- → affected
status-seamonkey2.49: --- → affected
status-seamonkey2.50: --- → affected
status-seamonkey2.51: --- → affected

Comment 19

2 months ago
Comment on attachment 8832864 [details] [diff] [review]
1334937-remove-text-calendar.patch

r=frg

Don't forget to change the patch heading to reflect this.

Thanks
Attachment #8832864 - Flags: review?(iann_bugzilla) → review+
(Assignee)

Comment 20

2 months ago
https://hg.mozilla.org/comm-central/rev/a2e039a852d7c66b373d7020847f30bc15783e8a

Thanks for the review. I changed the commit message and even the patch itself to clarify this in a comment ;-)
Status: ASSIGNED → RESOLVED
Last Resolved: 2 months ago
status-seamonkey2.51: affected → fixed
status-thunderbird54: affected → fixed
Resolution: --- → FIXED
(Assignee)

Comment 21

2 months ago
Comment on attachment 8832864 [details] [diff] [review]
1334937-remove-text-calendar.patch

Clear regression from bug 574989, and, might I add, the only one although we did quite some changes in that bug ;-)

This needs to go into all branches, especially for TB 52 ESR. I'll handle the C-A and C-B uplifts.

I forgot to say in the previous comment: My new landing policy is to trigger a build after every M-I to M-C merge. One had just happened after the last Daily build, so this push/build was necessary.
Attachment #8832864 - Flags: approval-comm-release+
Attachment #8832864 - Flags: approval-comm-beta+
Attachment #8832864 - Flags: approval-comm-aurora+
(Assignee)

Updated

2 months ago
Target Milestone: --- → Thunderbird 54.0
(Assignee)

Updated

2 months ago
Summary: .ics attachment wrongly shown inline although no viewer available → text/calendar MIME part wrongly prioritised over text/html in multipart/alternative

Comment 22

2 months ago
Thanks. I will wait till its in c-b and then push it to c-r.

Comment 23

2 months ago
Had a Lotus Notes message in my drafts folder which I copied there some time ago to test something. It showed the same behaviour after disabling Lighting. With the fix here a proper preview and message is shown.

User agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0 SeaMonkey/2.51a1
Build identifier: 20170202161617
Status: RESOLVED → VERIFIED

Comment 24

2 months ago
Jorg,

I just noticed that with Lightning enabled my Notes message is no longer recognized as an invitation and I am unable to add it to a calendar. This might need to be backed out.
Status: VERIFIED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 25

2 months ago
https://hg.mozilla.org/comm-central/rev/a2e039a852d7c66b373d7020847f30bc15783e8a (comment #20)
https://hg.mozilla.org/comm-central/rev/3cbd2f30207c05929b94af61fea9b9f4184aa8eb (bustage fix)

Oops, Linux and Mac didn't like |text/*| in a comment.

But now I see that there is a complaint in comment #24. Can you attach the message in question or send it to me in a PM.
(Assignee)

Comment 26

2 months ago
I can see it now. The HTML part displays fine, but Lightning doesn't pick up the invitation.
(Assignee)

Comment 27

2 months ago
Hmm, I did an experiment an removed the attachment. Lightning gets the invitation from the third mime part, the text/calendar, not the invitation.

I think the patch is right as far as the display is concerned, so not to display the text/calendar part instead of the text/html part, but then Lightning doesn't see that part. Let's take it out for now.

Backout:
https://hg.mozilla.org/comm-central/rev/dca11f96a228ccdcc9ec5a0123f82d6f8348139e
https://hg.mozilla.org/comm-central/rev/ca5f57cbe502fa561f667282abfd0914f522eb3f
status-seamonkey2.51: fixed → affected
status-thunderbird54: fixed → affected

Comment 28

2 months ago
Just tried to anonymize my mail and remove the company information but failed so sorry can't send you my testcase right now.

I agree. The display part is right. Looks like it just needs a patch somewhere else too.
(Assignee)

Comment 29

2 months ago
MMD, if you have a multipart/alternative message with three MIME parts:
text/plain
text/html
text/calendar
how does Lightning grab the last part as invitation?

If I change the MIME processing to always select the html part for display, then Lightning doesn't see the invitation. So does Lightning just grab the displayed part and extract an invitation if it's text/calendar?

Also I'm wondering how this worked before bug 574989, for example in TB 45.x. With Lightning, we get the invitation and without Lightning I get the html part displayed, although it's not the last part. Hmm.
Change was this: https://hg.mozilla.org/comm-central/rev/ab68d375e7e4.

Terje, do you have an idea?
Flags: needinfo?(makemyday)
Flags: needinfo?(bugzilla)
(Assignee)

Comment 30

2 months ago
MMD, there doesn't appear to be a test assuring that with Lightning installed see that text/calendar part as invitation. Here
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=a2e039a852d7c66b373d7020847f30bc15783e8a
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=3cbd2f30207c05929b94af61fea9b9f4184aa8eb
the patch from this bug was in place prioritising the text/html part over the text/calendar part for display, yet, no test failed.
(Assignee)

Comment 31

2 months ago
Confirming the regression. After backing out revision ab68d375e7e4 locally the behaviour is this:
With Lightning: Invitation is displayed.
Without Lightning: HTML part is displayed.

https://hg.mozilla.org/comm-central/rev/ab68d375e7e4 isn't huge change, so it shouldn't be to hard to fix, well, it's MIME ;-(

I noticed a MIME calendar hook here:
https://dxr.mozilla.org/comm-central/rev/756fa9e390ab08f995382c9a9dfc6bf4b72421ca/mailnews/mime/src/mimei.cpp#466
in mime_find_class().

Looks like a parameter to the call of that function got changed here:
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.61
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.75
(Assignee)

Comment 32

2 months ago
Created attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

FRG, you can try this.
Attachment #8832864 - Attachment is obsolete: true
Flags: needinfo?(makemyday)
Attachment #8833038 - Flags: feedback?(frgrahl)
Attachment #8833038 - Flags: feedback?(bugzilla)
(Assignee)

Comment 33

2 months ago
(Oops, I hit <enter> while the caret wasn't in the text area and the comment got submitted).

Terje, can you explain why you changed 'true' to 'false'
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.61
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.75
and what consequences it might have to turn it back.
(Assignee)

Comment 34

2 months ago
The test from bug 574989 and bug 253830 still passes:
mozmake SOLO_TEST=message-window/test-view-plaintext.js mozmill-one

Comment 35

2 months ago
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

Quick check looks ok to me. With Lightning disabled I get the html view. 

Enabled the invite buttons etc. work.
Attachment #8833038 - Flags: feedback?(frgrahl) → feedback+
(Assignee)

Comment 36

2 months ago
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

You can stick an r+ onto it then. Your review is as good as anyone's (no offence intended) since absolutely nobody understands this stuff, unless maybe Terje. I'll wait for his feedback.
Attachment #8833038 - Flags: review?(frgrahl)

Comment 37

2 months ago
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

Ok then

r=me
Attachment #8833038 - Flags: review?(frgrahl) → review+
(Assignee)

Comment 38

2 months ago
Green try here:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=6803b6951dafb88103de119f8a49934de880add4
(Assignee)

Updated

2 months ago
Attachment #8832864 - Flags: approval-comm-release+
Attachment #8832864 - Flags: approval-comm-beta+
Attachment #8832864 - Flags: approval-comm-aurora+
(Assignee)

Comment 39

2 months ago
I did some more debugging on this.

With the current state
  mime_find_class (ct, sub_hdrs, self->options, false);
  if (clazz && clazz->displayable_inline_p(clazz, sub_hdrs)) {
all parts text/plain, text/html and text/calendar are considered displayable, regardless of whether Lightning is installed or not.
With Lightning installed, the text/calendar part is grabbed by Lightning and displayed as invitation. Without Lightning, the text/calendar part is displayed instead of the HTML part. Lowering, as I did yesterday, the priority of the text/calendar part has as a consequence that it was never prioritised and Lightning didn't find it.

With the previous and new planned state
  mime_find_class (ct, sub_hdrs, self->options, true);
  if (clazz && clazz->displayable_inline_p(clazz, sub_hdrs)) {
text/calendar is only considered displayable if Lightning is installed. So with Lightning installed, the text/calendar part passes into the prioritisation, has the same priority as HTML, since it's last, it gets selected and Lightning turns it into an invitation. Without Lightning, text/calendar is not considered displayable so the HTML part is displayed.

So going back to 'true' is the right fix.
Flags: needinfo?(bugzilla)
(Assignee)

Comment 40

2 months ago
https://hg.mozilla.org/comm-central/rev/84718664672cc6451b3fe7d136f4788235af8107
Second time lucky ;-)
Status: REOPENED → RESOLVED
Last Resolved: 2 months ago2 months ago
status-seamonkey2.51: affected → fixed
status-thunderbird54: affected → fixed
Resolution: --- → FIXED
(Assignee)

Updated

2 months ago
Summary: text/calendar MIME part wrongly prioritised over text/html in multipart/alternative → text/calendar MIME part wrongly displayed when Lightning is not installed
(Assignee)

Comment 41

2 months ago
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

I'll uplift this once I've convinced myself in a Daily that it's really working this time ;-)
Attachment #8833038 - Flags: feedback?(bugzilla)
Attachment #8833038 - Flags: approval-comm-beta+
Attachment #8833038 - Flags: approval-comm-aurora+
(Assignee)

Comment 42

2 months ago
Aurora (TB 53, SM 2.50):
https://hg.mozilla.org/releases/comm-aurora/rev/97aa0b3426f4ff9a8fa4c6ae61e1386ea079c873
status-seamonkey2.50: affected → fixed
status-thunderbird53: affected → fixed
(Assignee)

Updated

2 months ago
Attachment #8833038 - Flags: approval-comm-release+
(Assignee)

Comment 43

2 months ago
Beta (TB 52, SM 2.49):
https://hg.mozilla.org/releases/comm-beta/rev/501c44488ee83d93c9e67cdf7e6eaff4f010ffea
status-seamonkey2.49: affected → fixed
status-thunderbird52: affected → fixed

Comment 44

2 months ago
Release (SM 2.48):
https://hg.mozilla.org/releases/comm-release/rev/025a17fc08a02f40efff11502a465be159ace252
status-seamonkey2.48: affected → fixed

Comment 45

2 months ago
I did some tests with with attachment 2017 [details]-02-02 12:12 CET, Rainer Bielefeld and it's source and
Server-Installation of  unofficial (by FRG)  en-US SeaMonkey 2.51a1 (NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0 Build 20170204194856  (Default Classic Theme) on German WIN7 64bit 
and 
TB Daily 54.0a1 (2017-02-04) (64-bit):

- With Lightning active
  -- Lightning view of .ics shown inline
  -- Drag and drop of .ics from Mail Heading attachment pane to Lighning works fine
- With Lightning INactive
  -- HTML (or text) view of .ics shown

With SeaMonkey I additionally gambled around a little with calendar and .ics attachments: Everything seems to work fine.
Status: RESOLVED → VERIFIED
(Assignee)

Comment 46

2 months ago
Thanks for verifying, but there is still some confusion.

ICS attachments are *never* shown inline. The message has three MIME parts:
text/plain
text/html
text/calendar

Without Lightning, the HTML part is displayed since the text/calendar part is classified non-displayable. With Lightning, the text/calendar part is displayed, but Lightning grabs that and displays its fancily formatted invitation.

None of this has anything to do with the ICS attachment, see attachment 389696 [details] from bug 505024 where you have those three message parts, but *without* the ICS attachment. The complaint in that bug is that without Lightning, the HTML part is displayed, but the user has no idea that there is an invitation in an non-displayable third MIME part.

BTW, without fixing this bug here, the text/calendar part *was* displayed even without Lightning. In one way that was good, since it helped bug 505024 a bit since users *did* see the invitation. In another way it wasn't good, since users lost the HTML display.

To fix bug 505024 one would have to pretend that the third non-displayable part was indeed an attachment. Since this is hard, that bug from 2009 is still not solved. As you can see, changing one 'false' to 'true' in MIME can have some surprising effects ;-(

Comment 47

a month ago
1. Is this bug the reason I cannot add calendar invites to my google calendar any more?

2. Isn't Lightning installed and turned on by default in all Thunderbird and Seamonkey installations? Does this mean people are turning off that functionality, and then seeing that once turned off that it doesn't work?
(Reporter)

Comment 48

9 hours ago
When will You release new official build with this bug fixed on Seamonkey homepage ?(Windows installer counrty localized version)
You need to log in before you can comment on or make changes to this bug.