580971 - Thunderbird gets an incorrect mime part when opening an attachment of a received message with malformed content type (Content-Type: =?windows-1252?q?application/pdf;)

Status: RESOLVED DUPLICATE of bug 659355

(MailNews Core :: MIME, defect)

Description

[Luis Martin-Santos]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; es-es) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.4) Gecko/20100608 Thunderbird/3.1

Certain messages I receive with multiple parts , some of them attachments, are incorrectly handled. When I try to Open an attachment, another part (the HTML one) gets downloaded with the same file name as the attachment I want to open to the temporary folder.


Reproducible: Sometimes

Steps to Reproduce:

For example. A message with 5 parts:

Part 1: Plain text body
Part 2: HTML body
Part 3: GIF Signature
Part 4: PDF File
Part 5: Another PDF FIle.


Actual Results:  
I try to open Part 4 and Part 2 gets saved instead of part 4

Expected Results:  
Open Part 4.

This happens with messages received from multiple sources.

Comment 1

[Luis Martin-Santos]

Please, mark this as confirmed since on Mozilla's GetSatisfaction, several people have the very same problem.

http://getsatisfaction.com/mozilla_messaging/topics/problem_opening_pdf_attachments_in_tb

Comment 2

[Ludovic Hirlimann [:Usul]]

can you attach an simple message here please (see add an attachment link) ?

Comment 3

[Luis Martin-Santos]

Attachment: A test mail message whose attachment cannot be properly openned

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #3)
> A test mail message whose attachment cannot be properly openned

Mail structure:
> Content-Type: multipart/mixed; boundary="------------090309080506070209000206"
> 
> This is a multi-part message in MIME format.
> 
>   (Part-1)
>   --------------090309080506070209000206
>   Content-Type: multipart/alternative; boundary="------------060505070504000406080307"
> 
>     (Part-1-1)
>     --------------060505070504000406080307
>     Content-Type: text/plain; ...
> 
>     (snip)
> 
>     (Part-1-2)
>     --------------060505070504000406080307
>     Content-Type: multipart/related; boundary="------------010606040802050906070702"
> 
>       (Part-1-2-1)
>       --------------010606040802050906070702
>       Content-Type: text/html; ...
>       
>       (snip)
>             <img src="cid:[e-mail]" border="0">
>       (snip)
> 
>       (Part-1-2-2)
>       --------------010606040802050906070702
>       Content-Type: image/gif; ...; name="firma_miguel.gif"
>       Content-ID: <[e-mail]>
> 
>       (snip)
>       --------------010606040802050906070702--
> 
>     --------------060505070504000406080307--
> 
>   (Part-2)
>   --------------090309080506070209000206
>   Content-Type: =?windows-1252?q?application/pdf; name="Prueba.pdf"
> 
>   (snip)
>   --------------090309080506070209000206--

Which part in above mail structure do you call "Part 4"/"Part 2" of your next statement in comment #0?
> Actual Results:  
> I try to open Part 4 and Part 2 gets saved instead of part 4

For "PDF attachment" you call in above mail.
Where can we see "PDF attachment" defined by mail system, RFCs for mail?  
There is no PDF attachment because "Content-Type: =?windows-1252?q?application/pdf;" can't be a mime-type of a PDF attachment.

Assume "Part 4"==part of "Content-Type: =?windows-1252?q?application/pdf;".
Which part of avove mail structure is saved by your save operation stated in comment #0?  

Even if such bad/broken content-type(unknown content type), I think "Open" + "Choose application to open" is usually possible. But it may be intepreted as text and ".txt" is added to file nme. In this case, even if Adobe Reader is invoked, Adobe Reader doesn't display it due to .txt.   

What did you do for such bad attachment in the past? You opened by Adobe Reader or somthing and you checked "Do this always..." or "Always open by this application..." at open dialog?
Read Bug 476400, Bug 476133, Bug 579682, for bad mimeTypes.rdf entry related phenomena, please.

To isolate problem, do next first, please.
1. Terminate Tb.
2. Rename mimeTypes.rdf in profile directory to mimeTypes.rdf.Backup.
   (keep back up for future analysis and future recovery)
3. Restart Tb, try to open the PDF attachment of currupted Content-Type.
What will happen?

Comment 5

[rsx11m]

Also see bug 659355 where the wrong file-type association was actually picked up in mimeTypes.rdf (which may be bug 503309). It would be interesting to figure out how (by which application) that incompliant Content-Type heading was created in the first place and then "inherited" in Thunderbird by opening it.

Comment 6

[rsx11m]

Still reproducible on today's nightly TB 7.0 trunk build on Windows 7, thus confirming after talking to Ludo. The difference to bug 659355 is that this bug is on considering the wrong "Content-Type:" header to start with, whereas picking it up in mimeTypes.rdf is bug 659355.

Ideally, I'd suggest that the wrong syntax should be identified (which may already happen, hence the treatment as "text/plain" based on guessing) and then the attachment treated as "application/octet-stream", thus using the ".pdf" as indication that it is an "application/pdf" attachment.

This bug may prevent bug 659355, and bug 503309 may cover both issues.

Comment 7

[WADA:World Anti-bad-Duping Agency]

FYI.

Opener of Bug 685112 has reported pheomenon of "email's text is opened by application", and was reproduced in that bug with simple test mails.
So, I opened next part of test mail attached to this bug by Notepad on Win.
> Content-Type: =?windows-1252?q?application/pdf; name="Prueba.pdf"

Data shown by Notepad was next;
(content of file created by Tb in \Temp directory)
(checked by Tb 6.0 and Tb trunk 20110828 build)
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
> <html>
>   <head>
>
>     <meta http-equiv="content-type" content="text/html; ">
>   </head>
>   <body bgcolor="#ffffff" text="#000000">
>     ESTO ES UNA PRUEBA<br>
>     <div class="moz-signature">-- <br>
>       <img src="mailbox:///C|/wada/@@@/Mail0/bug-685112-B?number=0&part=1.1.2.2&filename=Prueba.pdf&type==?windows-1252?q?application/pdf&filename=Prueba.pdf&filename=firma_miguel.gif" border="0"></div>
>   </body>
> </html>

(1) "email's text"(message body, text/html part) was shown as-is, except src attribute of <img> tag. This is phenomenon found by Bug 685112.
(2) name or filename in message header is correctly used as filename in \Temp as expected. This is same as Bug 685112.
(3) src attribute of <img> tag was changed. (not checked in that bug)
[Original in HTML source]
 (cid: correctly points image/gif part in same multipart/related )]
>     <div class="moz-signature">-- <br>
>       <img src="cid:[e-mail]" border="0"></div>
[HTML text shown by Notepad]
 (src of <img> was replaced by internal path name of the broken mime-part)
>       <img src="mailbox:///C|/wada/@@@/Mail0/bug-685112-B?number=0&part=1.1.2.2&filename=Prueba.pdf&type==?windows-1252?q?application/pdf&filename=Prueba.pdf&filename=firma_miguel.gif" border="0"></div>

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to rsx11m from comment #6)
> Still reproducible on today's nightly TB 7.0 trunk build on Windows 7,

rsx11m, see bug 685112 comment #6 for "funny cid: url of previous comment" which is seen with test mail attached to this bug, please.

Comment 9

[David :Bienvenu]

what e-mail program is generating these content types?

Comment 10

[David :Bienvenu]

From irc:
9:21:23 AM - bienvenu: hery - so are you asking where you might change this in the code?
9:21:58 AM - hery: I've identified nsMessenger.cpp in OpenAttachment as the best place to change this
9:22:33 AM - squib: hery: i'd think a better place would be in libmime
9:23:01 AM - bienvenu: squib, hery - yeah, I tend to agree
9:23:26 AM - hery: but I've some difficulties to deal with string utilities...
9:23:48 AM - hery: if you say so, I can search where to fix that in libmime
9:24:10 AM - bienvenu: hery, one possiblity in libmime is nsMimeHtmlDisplayEmitter::StartAttachment
9:24:26 AM - bienvenu: or you could try to find the libmime code that parses the content type header
9:24:50 AM - bienvenu: and just set it "correctly" in the mime structures
9:29:41 AM - bienvenu: hery - that might be more complicated (more places setting the content type), but the earlier we fix the error, the more general the fix
9:29:42 AM - bienvenu: MimeObject_initialize
9:29:45 AM - bienvenu: is one such place
9:30:44 AM - bienvenu: hery - looking at calls like this: MimeHeaders_get (obj->headers, HEADER_CONTENT_TYPE
9:31:18 AM - bienvenu: and finding the ones that are involved with actual attachments like this is an other possibility

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to David :Bienvenu from comment #9)
> what e-mail program is generating these content types?

User agent of mail with corrupted mime-type which is attached to bugs.

(1) bug 580971 (this bug)
> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4
Someone sent mail with corrupted mime-type
=> due to bug 503309, wrong data is generated in mimeTypes.rdf of Tb
=> Tb sent the corrupted mime-type

(2) bug 659355
Original sender of corrupted mime-type is unknown. But mimeTypes.rdf of Tb is referred in bug. So, it's similar case to bug 580971(this bug).

(3) bug 685112
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.21) Gecko/20110830 Thunderbird/3.1.13
Same case as bug 580971(this bug).

(4) bug 738284
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0.3) Gecko/20120306 Thunderbird/10.0.3
Same case as bug 580971(this bug).

In any of above 4 cases, who sent many mails with corrupted mime-type is Tb who received a mail with corrupted mime-type from some one.
Tb has capability to deliver corrupted mime-type to many peoples very easiliy, like virus program :-)
Sorry but I couldn't reach bug which refers to original sender of this kind of corrupted mime-type. 

Cause is perhap non-escaped "&" of filename in internal request parameter string, because "?" is used as delimiter of keyword like "?part=", "?type=" etc.
Phenomenon looks depends on number of "?" in corrupted mime-type. See bug 739025(summary of above 4 cases) for it, please.

Comment 12

[maimic]

A workaround seems to work with Thunderbird/Linux by creating a $HOME/.mailcap file with this content :
?windows-1252?q?application/pdf;evince %s

Comment 13

[WADA:World Anti-bad-Duping Agency]

Fix was proposed to Bug 659355, so closing as DUP.

Comment 14

[WADA:World Anti-bad-Duping Agency]

Following is a Web page found by Google search for '"=?iso-8859-1?q?application/pdf"'(no single quot).
> http://isarc10internetforum.wikispaces.com/file/detail/RC10+ISA+Forum+2012+sessions.pdf/330930570

Following is HTTP headers for the PDF file of the page obtained by Firefox+LiveHTTPHeaders. 
> http://isarc10internetforum.wikispaces.com/file/view/RC10+ISA+Forum+2012+sessions.pdf/330930570/RC10%20ISA%20Forum%202012%20sessions.pdf
>
> GET /file/view/RC10+ISA+Forum+2012+sessions.pdf/330930570/RC10%20ISA%20Forum%202012%20sessions.pdf HTTP/1.1
> Host: isarc10internetforum.wikispaces.com
> User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: ja,en-us;q=0.7,en;q=0.3
> Accept-Encoding: gzip, deflate
> Connection: keep-alive
> Referer: http://isarc10internetforum.wikispaces.com/file/detail/RC10+ISA+Forum+2012+sessions.pdf/330930570
> Cookie: test=1; slave=b%3A0953cc49e110d9828f42da5f805f17b3%3A04d6753b23bd38360926dd200f7b3798d6fdcb01bc7e43f7a01a2c8b572ddba1d77c0a6d11ff2726692eafd795bd74c7ef0f91f89d6f539caf5b0ee927163aa2231ea6f15f68cb726290ef678126b89a15b973b1eecc04c760b8adbf79ea82ea5f61586d93c0253b3d4f7794f5832d4190b32f9e5eccc3236195be11a779d401; __utma=188148142.1277545661.1343856859.1343856859.1343856859.1; __utmb=188148142.18.5.1343856960299; __utmc=188148142; __utmz=188148142.1343856859.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
>
> HTTP/1.1 200 OK
> Server: nginx
> Date: Wed, 01 Aug 2012 21:36:14 GMT
> Content-Type: =?iso-8859-1?q?application/pdf
> Connection: keep-alive
> Cache-Control: public
> Expires: Thu, 01 Aug 2013 21:36:14 GMT
> Content-Disposition: inline; filename="RC10 ISA Forum 2012 sessions.pdf"; size="1267587"
> Content-Length: 1267587
> Last-Modified: Mon, 07 May 2012 06:34:44 GMT
> X-Whom: s9
> P3P: CP: ALL DSP COR CURa ADMa DEVa CONo OUR IND ONL COM NAV INT CNT STA

If Web Mail is used on this kind of site, or if Web application on this kind site sends mail, "Content-Type: =?iso-8859-1?q?application/pdf" is probably sent to mail recipients.

Comment 15

[WADA:World Anti-bad-Duping Agency]

Culprit of "=?iso-8859-1?q?application/pdf of a PDF file of Web site" or "=?iso-8859-1?q?application/pdf in a Web mail" can be Firefox, because "capability to create bogus entry in mimeTypes.rdf silently" was initially Firefox's feature and it was blindly ported to Thunderbird.

(1) =?iso-8859-1?q?application/pdf is generated by someone.
(1-a) =?iso-8859-1?q?application/pdf is generated by someone for PDF of Web.
(1-b) =?iso-8859-1?q?application/pdf is generated in a mail by someone
      and the mail is held by Web Mail.
(2) Firefox creates entry of =?iso-8859-1?q?application/pdf in mimeTypes.rdf.
(2-a) PDF of =?iso-8859-1?q?application/pdf is downloaded by Firefox.
(2-b) PDF attachment of =?iso-8859-1?q?application/pdf is saved via Web Mail 
      using Firefox.
(3) Firefox generates =?iso-8859-1?q?application/pdf from mimeType.rdf.
(3-a) PDF is uploaded with =?iso-8859-1?q?application/pdf by Firefox.
(3-b) PDF is attached to Web mail with =?iso-8859-1?q?application/pdf
      via Web Mail using Firefox.