667019 - eventual MIME type parsing misinterpretation causing missing attachment with multipart/related

Status: RESOLVED DUPLICATE of bug 674473

(MailNews Core :: MIME, defect)

Description

[Yavor Stefanov]

User-Agent:       Mozilla/5.0 (Windows NT 5.2; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
Build Identifier: B_ID: 20110616145146 || V: 1.9.2.4184  || Mozilla/5.0 (Windows; U; Windows NT 5.2; en-GB; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11

Received e-mail message doesn't show the attached file, while showing inline images inserted into html. with a webclient (RoundCube) and Outlook 6 it's showing up. The actual data of the file is in the source, but it's not displayed in the UI rendering it useless. 

in TB 3.1.11 only the modified _MOD_message.eml shows the attachment.
in TB 5.0b2  the atachment in both samples is missing.
in TB 7.0a1  the atachment in both samples is missing.

Note: check the attached email samples.

Reproducible: Always

Steps to Reproduce:
open the attached original_message.eml


Actual Results:  
no attachment is showing up, while the images are embedded in the message.

Expected Results:  
a file must be showing up as attachment.

Comment 1

[Yavor Stefanov]

a link with the two eml file samples. http://heth.ludost.net/tmp/~yavor/tb/

Comment 2

[Yavor Stefanov]

Attachment: Original

original message causing the fault

Comment 3

[Yavor Stefanov]

Attachment: modified message

modified the message removing a mime encasing ( I'm not sure what(if that) is causing the fault)

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> in TB 3.1.11 only the modified _MOD_message.eml shows the attachment.
> in TB 5.0b2  the atachment in both samples is missing.
> in TB 7.0a1  the atachment in both samples is missing.

Where can we see "attachment" in the mails?
Can multipart/related mail contain attachment part like multipart/mixed mail?

"Non referred part in multipart/related by html via cid" is not rendered in inline.
In your case, (Case-A) it's pdf part in multipart/mixed in multipart/related(which is malformed mail), if View/Message Body As/HTML.
If View/Message Body As/Plain Text,(Case-B) two jpeg parts in multipart/related are also "Non referred part in multipart/related".

In this case, the "Non referred part in multipart/related" was shown as if attachment until Tb 3.x, but the parts are ignored afte6 Tb 5.
See Bug 602718 for Case-B issue.

By enhancement of Bug 602718, "a way to access such parts as if attachment" is implemented, and it's also effective for malformed Case-A.
However, your original case involves another/severe malform : duplicated boundary definitions.
  boundary="---Mail-Builder-Boundary-8380-1308737881-1"
  is used by both multipart/alternative and multipart/mixed.
I think this is reason why next happens, because such "duplicated boundary definitions" doesn't exist in modified case(multipart/mixed is removed by you).
> in TB 3.1.11 only the modified _MOD_message.eml shows the attachment.
 
Your original mail structure:
> Content-type: multipart/related; boundary="__MailScanner_found_Cyrus_boundary_substring_problem__"
>
>   --__MailScanner_found_Cyrus_boundary_substring_problem__
>   Content-Type: multipart/alternative; boundary="---Mail-Builder-Boundary-8380-1308737881-1"
>
>     -----Mail-Builder-Boundary-8380-1308737881-1
>     Content-Type: text/plain; charset=windows-1251
>     Content-Transfer-Encoding: 8bit
>
>     -----Mail-Builder-Boundary-8380-1308737881-1
>     Content-Type: text/html; charset=windows-1251
>     Content-Transfer-Encoding: 8bit
>
>     <img src="cid:inline001.jpg" />
>     <img src="cid:inline005.jpg" />
>
>     -----Mail-Builder-Boundary-8380-1308737881-1--
>
>   --__MailScanner_found_Cyrus_boundary_substring_problem__
>   Content-Type: multipart/mixed; boundary="---Mail-Builder-Boundary-8380-1308737881-1"
>
>     -----Mail-Builder-Boundary-8380-1308737881-1
>     Content-Type: application/octet-stream; name="210016413139_0087872468_20110531.pdf"
>
>     -----Mail-Builder-Boundary-8380-1308737881-1--
>
>   --__MailScanner_found_Cyrus_boundary_substring_problem__
>   Content-Type: image/jpeg; name="inline001.jpg"
>   Content-ID: <inline001.jpg>
>
>   --__MailScanner_found_Cyrus_boundary_substring_problem__
>   Content-Type: image/jpeg; name="inline005.jpg"
>   Content-ID: <inline005.jpg>
>
>   --__MailScanner_found_Cyrus_boundary_substring_problem__--

Can you see pdf part(and jpeg part with Plain Text mode) of original case as if attachment with trunk nightly build?

Have you requested mail sender to send well-formed mail instead of malformed mail?

Comment 5

[Yavor Stefanov]

(In reply to WADA from comment #4)
> (In reply to comment #0)
> > in TB 3.1.11 only the modified _MOD_message.eml shows the attachment.
> > in TB 5.0b2  the atachment in both samples is missing.
> > in TB 7.0a1  the atachment in both samples is missing.
> 
> Where can we see "attachment" in the mails?

I don't understand your question. I refer to attachment as no inline file showing up in TB interface.

> Can multipart/related mail contain attachment part like multipart/mixed mail?
> 
> "Non referred part in multipart/related by html via cid" is not rendered in
> inline.
> In your case, (Case-A) it's pdf part in multipart/mixed in
> multipart/related(which is malformed mail), if View/Message Body As/HTML.
> If View/Message Body As/Plain Text,(Case-B) two jpeg parts in
> multipart/related are also "Non referred part in multipart/related".
> 
> In this case, the "Non referred part in multipart/related" was shown as if
> attachment until Tb 3.x, but the parts are ignored afte6 Tb 5.
> See Bug 602718 for Case-B issue.
> 
> By enhancement of Bug 602718, "a way to access such parts as if attachment"
> is implemented, and it's also effective for malformed Case-A.
> However, your original case involves another/severe malform : duplicated
> boundary definitions.
>   boundary="---Mail-Builder-Boundary-8380-1308737881-1"
>   is used by both multipart/alternative and multipart/mixed.
> I think this is reason why next happens, because such "duplicated boundary
> definitions" doesn't exist in modified case(multipart/mixed is removed by
> you).
> > in TB 3.1.11 only the modified _MOD_message.eml shows the attachment.
>  
> Your original mail structure:
> > Content-type: multipart/related; boundary="__MailScanner_found_Cyrus_boundary_substring_problem__"
> >
> >   --__MailScanner_found_Cyrus_boundary_substring_problem__
> >   Content-Type: multipart/alternative; boundary="---Mail-Builder-Boundary-8380-1308737881-1"
> >
> >     -----Mail-Builder-Boundary-8380-1308737881-1
> >     Content-Type: text/plain; charset=windows-1251
> >     Content-Transfer-Encoding: 8bit
> >
> >     -----Mail-Builder-Boundary-8380-1308737881-1
> >     Content-Type: text/html; charset=windows-1251
> >     Content-Transfer-Encoding: 8bit
> >
> >     <img src="cid:inline001.jpg" />
> >     <img src="cid:inline005.jpg" />
> >
> >     -----Mail-Builder-Boundary-8380-1308737881-1--
> >
> >   --__MailScanner_found_Cyrus_boundary_substring_problem__
> >   Content-Type: multipart/mixed; boundary="---Mail-Builder-Boundary-8380-1308737881-1"
> >
> >     -----Mail-Builder-Boundary-8380-1308737881-1
> >     Content-Type: application/octet-stream; name="210016413139_0087872468_20110531.pdf"
> >
> >     -----Mail-Builder-Boundary-8380-1308737881-1--
> >
> >   --__MailScanner_found_Cyrus_boundary_substring_problem__
> >   Content-Type: image/jpeg; name="inline001.jpg"
> >   Content-ID: <inline001.jpg>
> >
> >   --__MailScanner_found_Cyrus_boundary_substring_problem__
> >   Content-Type: image/jpeg; name="inline005.jpg"
> >   Content-ID: <inline005.jpg>
> >
> >   --__MailScanner_found_Cyrus_boundary_substring_problem__--
> 
I have no knowledge of the MIME type syntax, I've just played around to make the file show so I can extract the data.

> Can you see pdf part(and jpeg part with Plain Text mode) of original case as
> if attachment with trunk nightly build?

I will try that and post later the results.

> Have you requested mail sender to send well-formed mail instead of malformed
> mail?
The mail sender is an automated system of the local power supply company, attaching invoices. I've tried to contact them and waiting response, althou none yet, at the moment of writing.

As a user I expect Thunderbird to try to extract the attachment if there is any and at least show it as a file somewhere in the interface, so it is not 'lost' as in the above case.
As a programmer I know this is malformation of an email and is not high priority since somebody is not keeping the syntax in readable form, which is not a TB problem at least on first look.
As a system administrator I see this as a problem since my users can't use Thunderbird as a mail application, because all other popular mail clients are showing the attachment, while TB does not, which ties my hands in promoting free software.

As a sidenote I'm thinking if there is a way to detect an attached file in any form and just show it in the attachments box in any case, no matter of the mime syntax, or some feature like that that is not dependent on the mime malformations inside the email, this could solve this and further issues like that in the future, althou this is only an assumption.

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to Yavor Stefanov from comment #5)
> > Have you requested mail sender to send well-formed mail instead of malformed
> > mail?
> The mail sender is an automated system of the local power supply company,
> attaching invoices. I've tried to contact them and waiting response, althou
> none yet, at the moment of writing.

Aha. Malformed multiprt/related was not generated by mailers(developers of mailers know mail RFCs well). It was genelated by application program used by the local power supply company. Application developers of many companies look that they don't respect mail related RFCs.

> As a user I expect Thunderbird to try to extract the attachment if there is
> any and at least show it as a file somewhere in the interface, so it is not
> 'lost' as in the above case.
> As a programmer I know this is malformation of an email and is not high
> priority since somebody is not keeping the syntax in readable form, which is
> not a TB problem at least on first look.
> As a system administrator I see this as a problem since my users can't use
> Thunderbird as a mail application, because all other popular mail clients
> are showing the attachment, while TB does not, which ties my hands in
> promoting free software.

As for multipart/related(malformed or not), enhancement is requested.
See Bug 674473.
> Bug 674473 Make MIME parts in a multipart/related context that are not
> referred to or cannot be displayed inline available as attachments again 

> As a sidenote I'm thinking if there is a way to detect an attached file in
> any form and just show it in the attachments box in any case, no matter of
> the mime syntax, or some feature like that that is not dependent on the mime
> malformations inside the email, this could solve this and further issues
> like that in the future, althou this is only an assumption.

Read thru Bug 602718, which I already pointed, please.
Enhancement by Bug 674473 is initially for "non rendered images in multiparet/related with View/Message Body As/Plain Text". However, the enhancement is applicable to any malformed mail, because new "View all parts as attachments" feature shows mail as if "any multipart/xxx is multipart/mixed".

Comment 7

[WADA:World Anti-bad-Duping Agency]

Duping to Bug 674473