text/calendar MIME part wrongly displayed when Lightning is not installed

VERIFIED FIXED in Thunderbird 54.0

Status

defect
VERIFIED FIXED
2 years ago
2 years ago

People

(Reporter: t50, Assigned: jorgk)

Tracking

({regression, reproducible})

Thunderbird 54.0
Unspecified
All
Dependency tree / graph

Thunderbird Tracking Flags

(thunderbird52+ fixed, thunderbird53 fixed, thunderbird54 fixed, seamonkey2.47 wontfix, seamonkey2.48 fixed, seamonkey2.49esr fixed, seamonkey2.50 fixed, seamonkey2.51 fixed)

Details

User Story

Facts:
-------
a) Reproducible for Google Calendar Invitations
c) No problem in older SM because those versions did not 
   wrongly show .ics contents inline.
d) Valid regression, SM /B may not show .ics without having an
   integrated .ics viewer. 
e) Also TB Problem, with 'wessage body as full html" and Lightning 
   disabled TB Daily 53.0a1 (2017-01-09) (64-bit) also shows that
   .ics source code
b) Appeared with SM 2.46

More investigation required
---------------------------
./.

Attachments

(3 attachments, 1 obsolete attachment)

Reporter

Description

2 years ago
User Agent: Mozilla/5.0 (Windows NT 5.1; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46
Build ID: 20161213183751

Steps to reproduce:

I always received attachments .ics(with calendar invitations) in version 2.39 and was listed (rendered & previewed correctly). After upgrade to 2.46 it is not interpreted any more (no frame of calendar meeting and invited persons - pure not parsed at all message only).


Actual results:

I can't see email messages with .ics (calendar files) properly in 2.46.
I had to rollback to v2.39 where everythink still works ok with .ics calendar preview.


Expected results:

Please repair preview and rendering of ics files in 2.46 like it was working in 2.39 and earlier.
Reporter

Updated

2 years ago
Severity: normal → critical
Component: MailNews: General → MailNews: Message Display

Comment 1

2 years ago
Dupe of Bug 505024 ?
Not sure if it might be related or a dupe of bug 1326271. If yes should be fixed in the next version.
Reporter

Comment 3

2 years ago
I don't use any plugins in seamonkey and OS (Lightning or sth.)

Comment 4

2 years ago
(In reply to Mike from comment #3)
> I don't use any plugins in seamonkey and OS (Lightning or sth.)

I really really doubt that that is true.  2.39 showed something different to a simple text listing? I doubt. 
Please forward to me by email such an email which was rendered as a Lightning like calendar invitation box.
Flags: needinfo?(t50)
Reporter

Comment 5

2 years ago
I attached the screenshots which ilustrates bugged rendering of invite.ics between 2.39(or older was OK) and 2.46(which is buggy).
Screenshots was made on the same e-mail when was previewed and opened in both versions - build version also included.
Flags: needinfo?(t50)
Comment hidden (off-topic)
I only ever saw such raw calandar entry mails when Lightning was not installed. There were a lot of changes in the mailnews code between 2.39 and 2.46. They might just be processed differently now. If would be good if you can attach a sample anonymized eml here.

Please refrain from further personal accusations.

Comment 8

2 years ago
More or less REPRODUCIBLE with official  German SeaMonkey 2.46 FINAL  (Windows NT 6.1; WOW64; rv:49.0)  Gecko/20100101 Firefox/49.0 Build 20161213183751  (Default Classic Theme)  on German WIN7 64bit.

Reporter's problem is not related to standard .ics attachments, which AFAIK  never were visible with SeaMonkey.

But if I create an invitation for a guest from my Google Calendar, it will be shown similar to reporter's screenshots with SM 2.39, but 2.46 only will show  source code.

I will attach a sample file for further investigation
Severity: critical → normal
User Story: (updated)
Keywords: reproducible
Summary: No preview and no reading of email attachment files .ics in ver 2.46 - in 2.39 all worked OK → No preview and no reading of email attachment files .ics from Google Calendar Invitation

Comment 9

2 years ago
Steps how to reproduce with SM 2.46:

0. Download Attachment
1. Launch SM email → Change 'View Message body' to simpe of full HTML if 
   necessary
2. Menu 'File → Open → Browse for Attachment → [open]
   » .eml will open in separate Window and show some .ics source

With 2.39 I see an invitation similar to reporter's screenshots

Comment 10

2 years ago
(In reply to Rainer Bielefeld from comment #9)
I forgot to mention in step 1: Disable Lightning, if necessary!

I think the core of the problem is that SM 2.46 shows the attached .ics (inline) without having an integrated .ics viewer.
SM must not show inline attachments which it can not render!

After some more investigation I will forward this one to MailNews CORE
User Story: (updated)

Comment 11

2 years ago
Also REPRODUCIBLE with attachment "Invitation From Google Calendar (.eml)" and  Server-Installation of  official  en-US SeaMonkey 2.50a1 (X11; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0 Build 20170118003001  (Default Classic Theme) on VirtualBox Ubuntu 14.04.1:

.ics wrongly shown inline.
So OS=All
OS: Unspecified → All
Summary: No preview and no reading of email attachment files .ics from Google Calendar Invitation → .ics attachment wrongly shown inline although no viewer available

Comment 12

2 years ago
b1): With steps from comment 9
     Already REPRODUCIBLE with  English SeaMonkey 2.46a2  (Windows NT 6.1; WOW64;
     rv:49.0)  Gecko/20100101 Firefox/49.0 Build 20160723030242  
    (Default Classic Theme)  on German WIN7 64bit
b2): Still ok with  English SeaMonkey 2.45  (Windows NT 6.1; WOW64; rv:48.0) 
    Gecko/20100101 Firefox/48.0 Build 20160710233343  
    (Default Classic Theme)  on German WIN7 64bit 
b): So probably appeared with MailNews Core 49
User Story: (updated)
Component: MailNews: Message Display → Attachments
Product: SeaMonkey → MailNews Core
Version: SeaMonkey 2.46 Branch → 49

Comment 13

2 years ago
f) No obvious DUP found with <https://bugzilla.mozilla.org/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=DUPs1334937&sharer_id=41036>
Status: UNCONFIRMED → NEW
Ever confirmed: true

Updated

2 years ago
User Story: (updated)
Reporter

Comment 14

2 years ago
Thank You Rainer for Your professional analysis. 
I can't attach You genuine ics's becouse I preview "restricted to me invitations" from Google Mail System.
I could try anonymize it in source maybe, but not now.
All invitations "rendering" worked in Seamonkey versions 2.39 & earlier with coloured borders and was OK.
After direct upgrade to 2.46 rendering doesn't work.
I didn't know if I need any plugin or something to render this or something has broken.
I enetered this as a Bug becouse rollback to 2.39 helped me to recover .ics rendering functionality.
If You have any solution for me to this or safe plugin to install for viewing in 2.46 - it is welcomed.
Thank's.
Assignee

Comment 15

2 years ago
This is basically bug 505024. However, the behaviour of the system changed a bit in bug 574989 which landed on TB 49. So TB 49 = SM 2.46 behaves differently to prior versions.

The problem is the MIME structure of the message. The top level is multipart/mixed so the first part is the message, and the second part the ICS attachment.

The message itself is multipart/alternative with three (!!) parts instead of two:
text/plain
text/html
text/calendar

For multipart/alternative the last (!) part should be displayed to get the richest display according to RFC 1521. Sadly, the last part is not the HTML but some funny text part.

Anyway, I'll attach a patch in a minute.
Assignee

Updated

2 years ago
Component: Attachments → MIME
Assignee

Comment 16

2 years ago
For people interested in history:
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.136
Assignee

Updated

2 years ago
Blocks: 574989
Keywords: regression
Assignee

Comment 17

2 years ago
OK, here's the patch. In order to understand it, you need to understand the prioritising we have implemented.

When the display is set to HTML (TB: View > Message Body As > Original HTML), we're looking for a rich part in the alternative section, regardless of the order.

text/plain is prioritised as "plain text" and text/html, text/enriched, text/richtext and sadly also text/calendar were prioritised as "rich text". The last rich part won, so in this case text/calendar over text/html.

Obviously this is wrong, since text/calendar is "plain text" MIME type and not a rich one. So just removing that from the group fixes the problem.

Voilà.

Now the bigger problem is getting this reviewed. FRG, can you do reviews?
Assignee: nobody → jorgk
Status: NEW → ASSIGNED
Attachment #8832864 - Flags: review?(iann_bugzilla)
>> Now the bigger problem is getting this reviewed. FRG, can you do reviews?

Occasionally for small things. I think this one qualifies and IanN will not shoot me. I can't approve any uplifts down to comm-release.
Comment on attachment 8832864 [details] [diff] [review]
1334937-remove-text-calendar.patch

r=frg

Don't forget to change the patch heading to reflect this.

Thanks
Attachment #8832864 - Flags: review?(iann_bugzilla) → review+
Assignee

Comment 20

2 years ago
https://hg.mozilla.org/comm-central/rev/a2e039a852d7c66b373d7020847f30bc15783e8a

Thanks for the review. I changed the commit message and even the patch itself to clarify this in a comment ;-)
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Assignee

Comment 21

2 years ago
Comment on attachment 8832864 [details] [diff] [review]
1334937-remove-text-calendar.patch

Clear regression from bug 574989, and, might I add, the only one although we did quite some changes in that bug ;-)

This needs to go into all branches, especially for TB 52 ESR. I'll handle the C-A and C-B uplifts.

I forgot to say in the previous comment: My new landing policy is to trigger a build after every M-I to M-C merge. One had just happened after the last Daily build, so this push/build was necessary.
Attachment #8832864 - Flags: approval-comm-release+
Attachment #8832864 - Flags: approval-comm-beta+
Attachment #8832864 - Flags: approval-comm-aurora+
Assignee

Updated

2 years ago
Target Milestone: --- → Thunderbird 54.0
Assignee

Updated

2 years ago
Summary: .ics attachment wrongly shown inline although no viewer available → text/calendar MIME part wrongly prioritised over text/html in multipart/alternative
Thanks. I will wait till its in c-b and then push it to c-r.
Had a Lotus Notes message in my drafts folder which I copied there some time ago to test something. It showed the same behaviour after disabling Lighting. With the fix here a proper preview and message is shown.

User agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0 SeaMonkey/2.51a1
Build identifier: 20170202161617
Status: RESOLVED → VERIFIED
Jorg,

I just noticed that with Lightning enabled my Notes message is no longer recognized as an invitation and I am unable to add it to a calendar. This might need to be backed out.
Status: VERIFIED → REOPENED
Resolution: FIXED → ---
Assignee

Comment 25

2 years ago
https://hg.mozilla.org/comm-central/rev/a2e039a852d7c66b373d7020847f30bc15783e8a (comment #20)
https://hg.mozilla.org/comm-central/rev/3cbd2f30207c05929b94af61fea9b9f4184aa8eb (bustage fix)

Oops, Linux and Mac didn't like |text/*| in a comment.

But now I see that there is a complaint in comment #24. Can you attach the message in question or send it to me in a PM.
Assignee

Comment 26

2 years ago
I can see it now. The HTML part displays fine, but Lightning doesn't pick up the invitation.
Assignee

Comment 27

2 years ago
Hmm, I did an experiment an removed the attachment. Lightning gets the invitation from the third mime part, the text/calendar, not the invitation.

I think the patch is right as far as the display is concerned, so not to display the text/calendar part instead of the text/html part, but then Lightning doesn't see that part. Let's take it out for now.

Backout:
https://hg.mozilla.org/comm-central/rev/dca11f96a228ccdcc9ec5a0123f82d6f8348139e
https://hg.mozilla.org/comm-central/rev/ca5f57cbe502fa561f667282abfd0914f522eb3f
Just tried to anonymize my mail and remove the company information but failed so sorry can't send you my testcase right now.

I agree. The display part is right. Looks like it just needs a patch somewhere else too.
Assignee

Comment 29

2 years ago
MMD, if you have a multipart/alternative message with three MIME parts:
text/plain
text/html
text/calendar
how does Lightning grab the last part as invitation?

If I change the MIME processing to always select the html part for display, then Lightning doesn't see the invitation. So does Lightning just grab the displayed part and extract an invitation if it's text/calendar?

Also I'm wondering how this worked before bug 574989, for example in TB 45.x. With Lightning, we get the invitation and without Lightning I get the html part displayed, although it's not the last part. Hmm.
Change was this: https://hg.mozilla.org/comm-central/rev/ab68d375e7e4.

Terje, do you have an idea?
Flags: needinfo?(makemyday)
Flags: needinfo?(bugzilla)
Assignee

Comment 30

2 years ago
MMD, there doesn't appear to be a test assuring that with Lightning installed see that text/calendar part as invitation. Here
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=a2e039a852d7c66b373d7020847f30bc15783e8a
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=3cbd2f30207c05929b94af61fea9b9f4184aa8eb
the patch from this bug was in place prioritising the text/html part over the text/calendar part for display, yet, no test failed.
Assignee

Comment 31

2 years ago
Confirming the regression. After backing out revision ab68d375e7e4 locally the behaviour is this:
With Lightning: Invitation is displayed.
Without Lightning: HTML part is displayed.

https://hg.mozilla.org/comm-central/rev/ab68d375e7e4 isn't huge change, so it shouldn't be to hard to fix, well, it's MIME ;-(

I noticed a MIME calendar hook here:
https://dxr.mozilla.org/comm-central/rev/756fa9e390ab08f995382c9a9dfc6bf4b72421ca/mailnews/mime/src/mimei.cpp#466
in mime_find_class().

Looks like a parameter to the call of that function got changed here:
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.61
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.75
Assignee

Comment 32

2 years ago
FRG, you can try this.
Attachment #8832864 - Attachment is obsolete: true
Flags: needinfo?(makemyday)
Attachment #8833038 - Flags: feedback?(frgrahl)
Attachment #8833038 - Flags: feedback?(bugzilla)
Assignee

Comment 33

2 years ago
(Oops, I hit <enter> while the caret wasn't in the text area and the comment got submitted).

Terje, can you explain why you changed 'true' to 'false'
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.61
https://hg.mozilla.org/comm-central/rev/ab68d375e7e4#l4.75
and what consequences it might have to turn it back.
Assignee

Comment 34

2 years ago
The test from bug 574989 and bug 253830 still passes:
mozmake SOLO_TEST=message-window/test-view-plaintext.js mozmill-one
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

Quick check looks ok to me. With Lightning disabled I get the html view. 

Enabled the invite buttons etc. work.
Attachment #8833038 - Flags: feedback?(frgrahl) → feedback+
Assignee

Comment 36

2 years ago
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

You can stick an r+ onto it then. Your review is as good as anyone's (no offence intended) since absolutely nobody understands this stuff, unless maybe Terje. I'll wait for his feedback.
Attachment #8833038 - Flags: review?(frgrahl)
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

Ok then

r=me
Attachment #8833038 - Flags: review?(frgrahl) → review+
Assignee

Updated

2 years ago
Attachment #8832864 - Flags: approval-comm-release+
Attachment #8832864 - Flags: approval-comm-beta+
Attachment #8832864 - Flags: approval-comm-aurora+
Assignee

Comment 39

2 years ago
I did some more debugging on this.

With the current state
  mime_find_class (ct, sub_hdrs, self->options, false);
  if (clazz && clazz->displayable_inline_p(clazz, sub_hdrs)) {
all parts text/plain, text/html and text/calendar are considered displayable, regardless of whether Lightning is installed or not.
With Lightning installed, the text/calendar part is grabbed by Lightning and displayed as invitation. Without Lightning, the text/calendar part is displayed instead of the HTML part. Lowering, as I did yesterday, the priority of the text/calendar part has as a consequence that it was never prioritised and Lightning didn't find it.

With the previous and new planned state
  mime_find_class (ct, sub_hdrs, self->options, true);
  if (clazz && clazz->displayable_inline_p(clazz, sub_hdrs)) {
text/calendar is only considered displayable if Lightning is installed. So with Lightning installed, the text/calendar part passes into the prioritisation, has the same priority as HTML, since it's last, it gets selected and Lightning turns it into an invitation. Without Lightning, text/calendar is not considered displayable so the HTML part is displayed.

So going back to 'true' is the right fix.
Flags: needinfo?(bugzilla)
Assignee

Comment 40

2 years ago
https://hg.mozilla.org/comm-central/rev/84718664672cc6451b3fe7d136f4788235af8107
Second time lucky ;-)
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
Assignee

Updated

2 years ago
Summary: text/calendar MIME part wrongly prioritised over text/html in multipart/alternative → text/calendar MIME part wrongly displayed when Lightning is not installed
Assignee

Comment 41

2 years ago
Comment on attachment 8833038 [details] [diff] [review]
1334937-call-mime_find_class-true.patch

I'll uplift this once I've convinced myself in a Daily that it's really working this time ;-)
Attachment #8833038 - Flags: feedback?(bugzilla)
Attachment #8833038 - Flags: approval-comm-beta+
Attachment #8833038 - Flags: approval-comm-aurora+
Assignee

Updated

2 years ago
Attachment #8833038 - Flags: approval-comm-release+

Comment 45

2 years ago
I did some tests with with attachment 2017 [details]-02-02 12:12 CET, Rainer Bielefeld and it's source and
Server-Installation of  unofficial (by FRG)  en-US SeaMonkey 2.51a1 (NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0 Build 20170204194856  (Default Classic Theme) on German WIN7 64bit 
and 
TB Daily 54.0a1 (2017-02-04) (64-bit):

- With Lightning active
  -- Lightning view of .ics shown inline
  -- Drag and drop of .ics from Mail Heading attachment pane to Lighning works fine
- With Lightning INactive
  -- HTML (or text) view of .ics shown

With SeaMonkey I additionally gambled around a little with calendar and .ics attachments: Everything seems to work fine.
Status: RESOLVED → VERIFIED
Assignee

Comment 46

2 years ago
Thanks for verifying, but there is still some confusion.

ICS attachments are *never* shown inline. The message has three MIME parts:
text/plain
text/html
text/calendar

Without Lightning, the HTML part is displayed since the text/calendar part is classified non-displayable. With Lightning, the text/calendar part is displayed, but Lightning grabs that and displays its fancily formatted invitation.

None of this has anything to do with the ICS attachment, see attachment 389696 [details] from bug 505024 where you have those three message parts, but *without* the ICS attachment. The complaint in that bug is that without Lightning, the HTML part is displayed, but the user has no idea that there is an invitation in an non-displayable third MIME part.

BTW, without fixing this bug here, the text/calendar part *was* displayed even without Lightning. In one way that was good, since it helped bug 505024 a bit since users *did* see the invitation. In another way it wasn't good, since users lost the HTML display.

To fix bug 505024 one would have to pretend that the third non-displayable part was indeed an attachment. Since this is hard, that bug from 2009 is still not solved. As you can see, changing one 'false' to 'true' in MIME can have some surprising effects ;-(

Comment 47

2 years ago
1. Is this bug the reason I cannot add calendar invites to my google calendar any more?

2. Isn't Lightning installed and turned on by default in all Thunderbird and Seamonkey installations? Does this mean people are turning off that functionality, and then seeing that once turned off that it doesn't work?
Reporter

Comment 48

2 years ago
When will You release new official build with this bug fixed on Seamonkey homepage ?(Windows installer counrty localized version)
You need to log in before you can comment on or make changes to this bug.