Closed Bug 760412 Opened 12 years ago Closed 3 years ago

Multipart/alternative (with Lightning): Out of 3 alternative parts, TB does not display nor offer as attachment any of (1) text/plain or (2) text/html, but only (3) text/calendar (mail with invite from google calendar)

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Version:
12 Branch
Platform:
x86_64
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(thunderbird_esr91+ fixed, thunderbird92 affected)

Status:
RESOLVED FIXED
Milestone:
93 Branch
Tracking Flags:
Tracking Status
thunderbird_esr91 + fixed
thunderbird92 --- affected

People

(Reporter: giunta.gaetano, Assigned: mkmelin)

References

(Blocks 1 open bug)

Details

(Keywords: testcase)

Attachments

(4 files)

the email in question
12.10 KB, message/rfc822
Details
Screenshot of tb displaying the mail
47.52 KB, image/png
Details
Bug 760412 - Always display text/calendar part along with chosen text/html part. r=darktrojan
48 bytes, text/x-phabricator-request
wsmwk
: approval-comm-esr91+
Details | Review
Bug 760412 - followup to remove now unused css selector. r=henry
48 bytes, text/x-phabricator-request
Details | Review
Attached file the email in question — 
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
Build ID: 20120420145725

Steps to reproduce:

[filing under general, though it might be Lightning-related]

receive an email. Email was sent using the "send msg to guests" button in goggle calendar's event page


Actual results:

TB only showed the event info, not the plaintext part of the mail (the "message" in the google calendar form)

.ics file was correctly shown as atachment as well, and could be saved

The funny thing is: the popup-notification window when mail was received showed instead the plaintext message


Expected results:

tb should have displayed the plaintext part of the mail above the event invite
(In reply to Gaetano Giunta from comment #0)
checked your attachment and the text part appears for me (which shows the following):

Hello,
I am so sorry, but someone had scheduled a meeting exactly at the same time  
as ours. Lets try gotowebinar:  
https://www3.gotomeeting.com/register/128585726
Thanks for your patience.
Melissa
Attachment #629143 - Attachment mime type: application/octet-stream → message/rfc822
Gaetano did you try in Thunderbird's safe-mode ?

Hashem do you have lightning installed ?
Keywords: testcase
(In reply to Gaetano Giunta from comment #0)
> TB only showed the event info, not the plaintext part of the mail (the
> "message" in the google calendar form)

Where can we see "ATTACHMENT part of text/calender" in the mail?

Mail structure.
  Content-Type: multipart/mixed; boundary=0016e6d976262e053a04c1649135
  --0016e6d976262e053a04c1649135
  Content-Type: multipart/alternative; boundary=0016e6d976262e053304c1649133
    --0016e6d976262e053304c1649133
    Content-Type: text/plain;
    --0016e6d976262e053304c1649133
    Content-Type: text/html;
    --0016e6d976262e053304c1649133
    Content-Type: text/calendar;
    --0016e6d976262e053304c1649133--
  --0016e6d976262e053a04c1649135
  Content-Type: application/ics; name="invite.ics"
  --0016e6d976262e053a04c1649135--

Because multipart/alternative and Tb has capability to show text/plain or text/html as message body, Tb chooses one of text/plain or text/html part in multipart/alternative, and because Tb is not a mailer who shows text/calender as message body, Tb ignores text/calander in multipart/alternative according to definition of multipart/altenative.
What's wrong?
(In reply to Ludovic Hirlimann [:Usul] from comment #2)
> Gaetano did you try in Thunderbird's safe-mode ?
> 
> Hashem do you have lightning installed ?

No, I don't.
Attachment #629143 - Attachment mime type: message/rfc822 → text/plain
(In reply to Gaetano Giunta from comment #0)
> the email in question
> Sender: Google Calendar <calendar-notification@google.com>

Does Google intentionally ignore difference among RFCs for multipart/mixed, multipart/alternative, multipart/related?

As for text/calender in multipart/related which is not pointed by text/html.
(i)  Lightning shows it as attachment
(ii) Tb already has quirks for it and shows it as if attachment,
     if one of name(Content-Type:), attachment(Content-Disposition:),
     filename(Content-Disposition:) is specified.

As for text/calender which is placed in multipart/alternative.
(iii) Tb 12 already shows it as if attachment too,
      if one of name(Content-Type:), attachment(Content-Disposition:),
      filename(Content-Disposition:) is specified. 
      However, even if this part is shown as if attachment by Tb,
      following problem occurs in Tb 12.0.1.
      - The text/calander part can't be opened due to following error.  
        ! This attachment appears to be empty.
          Please check with the person who sent this.
          Often company firewalls or antivirus programs will destroy attachments.
      - If save is requested, nothing is saved. Saved file is not created.
      This is already reported phenomenon to other bug.

In any case of malformed text/calender under multipart/related and malformed text/calender under multipart/related, 
(iv) text/calender part is not shown as if attachment by Tb, if no name(Content-Type:), no attachment(Content-Disposition:), no filename(Content-Disposition:). This is also known issue.

To see/save malformed text/calender under multipart/alternative, following workaround is already available.
(1) mailnews.display.show_all_body_parts_menu = true
(2) View/Message Body As/All Body Parts.
    Because any multipart/xxx is treated as multipart/mixed in this mode,
    any part can be opened by an application and can be saved as a file.
(3) Change back to mailnews.display.show_all_body_parts_menu=false for daily use.
@wada

> Does Google intentionally ignore difference among RFCs for multipart/mixed, multipart/alternative, multipart/related?

No clue. Ask them! :-)

Thanks for the detailed explanation you have given so far, but I think you might have misunderstood the problem.

When opening the mail, I DO see the calendar invite, both inline in the mail body and as attachment that can be saved as .ics.

The problem is that I DO NOT see the textual part which reads "I am so sorry, but someone ...".
This text is sent as parts 1 (text) and 2 (html) of the 3 which are sent as alternatives within the 1st part of the top level "mixed" multipart
PS: of all recipients of this mail, I was the only one having this problem, so other mail readers act differently (I'd expect gmail's own and apple mail to have been used here)

    
    

    
  
Lat note: when enabling mailnews.display.show_all_body_parts_menu = true, I see the missing text displayed twice: once in a "part" of its own, once just above the invite box

    
    

    
  
(In reply to Gaetano Giunta from comment #6)
> The problem is that I DO NOT see the textual part which reads "I am so
> sorry, but someone ...".
> This text is sent as parts 1 (text) and 2 (html) of the 3 which are sent as
> alternatives within the 1st part of the top level "mixed" multipart

Oh, sorry for my misunderstanding.

When did your problem started to occur?
No problem with Tb 12 and started to occur after upgrade to Tb 12.0.0 or 12.0.1?

Do you see your problem with any combination of of View/Message Body As and View/Display Attachments Inline ?
(Original HTML/Simple HTML/Plain Text, Display Attachments Inline=Checked/Unchecked)

Do you see your problem with new profile, with no add-on?
(1) thunderbird.exe -ProfileManager, create a new profile, start with it.
    When account Wizard is shown, cancel it,
    and manually create a dummy News or POP3 account(via Manual Setup).
    (any non-existent server such as news.news.news or pop3.pop3.pop3).
    "Local Folders" account is created.
(2) Create a folder(call FolderX) under Local Folders.
    Drag&Drop .eml file of the mail to thread pane of FolderX.
    (import of .eml by Drag&Drop)
What is shown by Tb 12?

As invitation is shown, you probably install Lightning.
Does your problem occur with Lightning installed to the new profile?

    
  

        Attachment #629143 -
        Attachment mime type: text/plain → message/rfc822

    
  
Summary: thunderbird does not display text multipart when mail contains an invite from google calendar → Multipart/alternative (with Lightning): Out of 3 parts, TB does not display nor offer as attachment any of (1) text/plain or (2) text/html, but only (3) text/calendar (mail with invite from google calendar)

    
    

    
  
(In reply to WADA from comment #10)
> (In reply to Gaetano Giunta from comment #6)
> > The problem is that I DO NOT see the textual part which reads "I am so
> > sorry, but someone ...".
> > This text is sent as parts 1 (text) and 2 (html) of the 3 which are sent as
> > alternatives within the 1st part of the top level "mixed" multipart
> When did your problem started to occur?
> No problem with Tb 12 and started to occur after upgrade to Tb 12.0.0 or
> 12.0.1?

didn't test for regression range

> Do you see your problem with any combination of of View/Message Body As and
> View/Display Attachments Inline ?
> (Original HTML/Simple HTML/Plain Text,
>  Display Attachments Inline=Checked/Unchecked)

With all of these combinations (exactly as above), result is same:
(TB12.0.1, new profile, no addons, WinXP)

a) with Lightning:
- only text/calender (alternative part 3) is shown
- text/plain and text/html (alternative part 1 and 2) are missing entirely (from display, and attachment pane)
- only .ics is shown as real attachment

b) without Lightning:
- correctly toggles between: View msg body as > Plain text vs. HTML (alternative parts 1 or 2 are shown)
- text/calendar (alternative part 3) is entirely missing (never displayed, not in attachment pane)
- .ics is always shown as real attachment

> Do you see your problem with new profile, with no add-on?
> What is shown by Tb 12?

Exactly same as above (tried new profile, no addons, with and without Lightning).

> As invitation is shown, you probably install Lightning.
> Does your problem occur with Lightning installed to the new profile?

Yes, as above.
Status: UNCONFIRMED → NEW
Ever confirmed: true

    
  
Summary: Multipart/alternative (with Lightning): Out of 3 parts, TB does not display nor offer as attachment any of (1) text/plain or (2) text/html, but only (3) text/calendar (mail with invite from google calendar) → Multipart/alternative (with Lightning): Out of 3 alternative parts, TB does not display nor offer as attachment any of (1) text/plain or (2) text/html, but only (3) text/calendar (mail with invite from google calendar)

    
    

    
  
This is the reduced problem in a nutshell:

msg with multipart/alternative part containing 3(!) alternative parts:
(1) text/plain
(2) text/html
(3) text/calendar

a) with Lightning:
- we never show any of (1) text/plain or (2) text/html
- we only show (3) text/calendar

b) without Lightning:
- we correctly toggle between (1) text/plain or (2) text/html
- we never show (3) text/calendar

Even if that triple alternative structure produced by google might be unfortunate or malformed, we need to handle this case better (robustness principle).

    
    

    
  
"Show text/calender under multipart/alternative as if attachment when name/filename/attachment is specified" is a new functionality of recent Tb.
This may affect on Lightning's subpart selection/handling.

Do you see similar problem in Tb 12/Lightning on (a) "text/calender under multipart/related" or (b) "text/calender mail"?

(a) Edit .eml file of "the email in question" by text editor, and change multipart/alernative to multipart/related, then Drag&Drop .eml file to thread pane of a local mail folder.

(b) Mail's Content-Type: is "Content-Type: text/calender", and message body of the mail is text/calendar data.

    
    

    
  
Issue in "text/calender under multipart/alternative" without Lightning is currently processed by Bug 505024. Setting dependency for ease of problem analysis, tracking, and search.
Depends on: 505024

    
  
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE

    
    

    
  
Let's get this straight:

Bug 505024: In an multipart/alternative message, the text/calendar part isn't shown unless Lightning is installed.

Bug 760412: In an multipart/alternative message, the text/html part isn't shown if Lightning is installed. Different issue.

Since TB78 Lightning is always "installed" (integrated into TB), so bug 505024 has been resolved and bug 760412 permanently becomes an issue.


Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---

    
  
Status: REOPENED → NEW

    
    

    
  
There used to be this add-on to always show the text/calendar part:

https://addons.thunderbird.net/en-US/thunderbird/addon/sfoa/



    
    

    
  
I think this is the correct issue for us to continue on.  Interestingly, I saved the .eml for two different effected emails, removed the entire text/calendar section and opened it again and they now display the text/html version as well as the attached .ics file.  Seems like if there is a way to prefer the text/html OR ignore the text/calendar version, it would resolve this bug.



    
    

    
  
Please read the RFC: https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html

The multipart/alternative type is syntactically identical to multipart/mixed, but the semantics are different. In particular, each of the parts is an "alternative" version of the same information. User agents should recognize that the content of the various parts are interchangeable. The user agent should either choose the "best" type based on the user's environment and preferences, or offer the user the available alternatives. In general, choosing the best type means displaying only the LAST part that can be displayed.


So if the message contains text/html followed by text/calendar, only text/calendar part will be displayed. If the event is also included as a .ics attachment, there is no point to have a text/calendar part with the same information. Microsoft OL clearly infringes the RFC here. That is not to say that TB shouldn't handle the case better.



    
    

    
  
To be honest, I'm not a web-person to this detail, just an avid user for many years who wants Thunderbird to keep going strongly.  The emails are from Google Calendar, which is also a large user base, so I wouldn't want this to be the cause of lost users.  Happy to continue to test/report back/etc. to help with this issue.



    
    

    
  
Sorry, I re-read bug 1725690 comment #0 now, it's not Microsoft, but Google. I thought that Google didn't put any content into the HTML part, but for updated events, that's apparently not the case. The bug has been on file since 2012. You'll have to see whether the TB team prioritizes it. Sadly, only showing one MIME part is deeply buried in 20+ y/o MIME code that no one wants to touch.



    
    

    
  
Hopefully it can be addressed at some point soon.  It is certainly a bother for those imporant event update emails.



    
    

    
  
Does this mean it will get into Thunderbird?



    
    

    
  
I can confirm that the fix noted for Betterbird in comment 27 works.  Betterbird displays the text/HTML and then the ics context (assuming due to inline attachments).



    
    

    
  


  
This makes us always show the generated invite html together with the normal html part.

Invites usually have an ok display in there generated by the server or sending client, and I think we don't want to normally show both parts at the same time. This patch changes the imip invite to just show an expandable summary on top of the html from the email. By clicking the arrow the details (like we had them) will be displayed, so that it still is accessible for the cases where the html representation is not there in the email, or it's bad.


Displaying the normal html content of the email solves problems expecially for "cancelled with a note" updates where you otherwise can't see the note because it's only inside the mail content.


Based on https://github.com/Betterbird/thunderbird-patches/blob/main/91/bugs/760412-always-show-text-calendar.patch


Assignee: nobody → mkmelin+mozilla
Status: NEW → ASSIGNED
No longer blocks: 1406868

        Attachment #9237555 -
        Attachment description: Bug 760412 - Always display text/calendar part along with chosen test/plain or text/html. r=darktrojan → Bug 760412 - Always display text/calendar part along with chosen text/html part. r=darktrojan

    
    

    
  
I'm happy to test out this patch if there is an easy way to.



    
    

    
  
I'd be happy to test this too - Happy to see this getting fixed [I have been waiting for this fix for years now! https://bugzilla.mozilla.org/show_bug.cgi?id=1581070]



    
    

    
  
I took the latest nightly build [https://ftp.mozilla.org/pub/thunderbird/nightly/2021/08/2021-08-30-10-47-50-comm-central/thunderbird-93.0a1.en-US.mac.dmg]


I still see the issue that I reported in the original bug I filed [1581070 ].


I am not sure if the fix is supposed to be present in 30Aug nightly this build?



    
    

    
  
It's not yet fixed. I need to adjust some tests before getting it landed.



    
    

    
  
Pushed by mkmelin@iki.fi:

https://hg.mozilla.org/comm-central/rev/565fc4b596ea

Always display text/calendar part along with chosen text/html part. r=darktrojan


Status: ASSIGNED → RESOLVED
Closed: 8 years ago → 3 years ago
Resolution: --- → FIXED

    
    

    
  
I think we eventually want to uplift to 91, since there's so cases where it's needed to get the information someone sent you.



          status-thunderbird92:
          --- → affected

          status-thunderbird_esr91:
          --- → affected

          tracking-thunderbird_esr91:
          --- → +
Target Milestone: --- → 93 Branch

    
    

    
  
I confirm this is fixed in the nightly build: 93.0a1 (2021-09-01) (64-bit)


@Magnus - thank you so much for getting this resolved.



    
    

    
  
After this change, this CSS selector is unused https://searchfox.org/comm-central/rev/88d88aa4086176def5c63925b9d727b593ec6b99/calendar/base/themes/common/imip.css#28-34 . It used to be used for the imipHtml-header, which was the table <caption class="header">, but is now a <summary>. Can you clean it up?



    
    

    
  
Confirming that 93.0a1 is working properly for these emails from Google Calendar this morning. Very excited for this to be pushed to the regular release.  Thank you all for your hard work and attention to this long existing bug!



    
    

    
  
Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/4ff85c5a8693
followup to remove now unused css selector. r=henry

    
    

    
  
Comment on attachment 9237555 [details]

Bug 760412 - Always display text/calendar part along with chosen text/html part. r=darktrojan


[Approval Request Comment]

To show missing information from invites. Landed in 93 milestone.



        Attachment #9237555 -
        Flags: approval-comm-esr91?

    
    

    
  
Comment on attachment 9237555 [details]

Bug 760412 - Always display text/calendar part along with chosen text/html part. r=darktrojan


[Triage Comment]

Approved for esr91


(it also has tests)



        Attachment #9237555 -
        Flags: approval-comm-esr91? → approval-comm-esr91+

    
    

    
  
It's expected yes.



    
  
Regressions: 1742101

    
    

    
  
I've run into that as well, and I agree with Richard that this UI is unfortunate. I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1742101



          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: