558075 - mimeTypes.rdf modified by "save to disk" sends attachments with wrong mime-types

Status: RESOLVED INCOMPLETE

(Thunderbird :: Mail Window Front End, defect)

Description

[anne.le-guellec]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)
Build Identifier: 2.0.0.23 (20090812)

There are 2 different ways I know to save an attachment :
- Doucle-click on the name of the attachment, then select "save to disk"
- Right-click on the name of the attachment, and select "save as"

But there seems to be a bug with the first method.
It doesn't only save the attachment, it also modifies the file mimeTypes.rdf so that sometimes files aren't associed anymore with their right type (ex: rich text for .doc). A modification seems to happen each first time a new type of attachment is saved (tested with .doc, .pdf). And sometimes, when the mimeTypes.rdf is incorrectly modified this way (I wasn't able to reproduce that part), attached documents sent by this user will not be correctly read by the receiver: It causes the (binary) contents of the file to be displayed inline (like body text): plenty of weird characters, hundreds of question marks, etc, like describdes in https://bugs.launchpad.net/ubuntu/+source/thunderbird/+bug/500367
It's quite simple to correct it by deleting mimeTypes.rdf, but it can come back if you use this method of saving again and meet a "bad" attachment that incorrectly modifies the file.

The second method doesn't modify the mimeTypes.rdf, and seems to work really fine, whatever the attachment is.

Reproducible: Always

Steps to Reproduce:
1.Save an attachment (.doc or .pdf) by double click and "save to disk". 
2.Chack MimeTypes.rdf : It has been modified
Actual Results:  
After saving a pdf attachment, the mimeTypes.rdf contents is as follows:

<?xml version="1.0"?>
<RDF:RDF xmlns:NC="http://home.netscape.com/NC-rdf#"
         xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <RDF:Description RDF:about="urn:mimetypes">
    <NC:MIME-types RDF:resource="urn:mimetypes:root"/>
  </RDF:Description>
  <RDF:Description RDF:about="urn:mimetype:application/pdf"
                   NC:value="application/pdf"
                   NC:editable="true"
                   NC:fileExtensions="pdf"
                   NC:description="Adobe Acrobat Document">
    <NC:handlerProp RDF:resource="urn:mimetype:handler:application/pdf"/>
  </RDF:Description>
  <RDF:Description RDF:about="urn:mimetype:handler:application/pdf"
                   NC:alwaysAsk="true"
                   NC:saveToDisk="true">
    <NC:externalApplication RDF:resource="urn:mimetype:externalApplication:application/pdf"/>
  </RDF:Description>
  <RDF:Description RDF:about="urn:mimetype:externalApplication:application/pdf"
                   NC:prettyName=""
                   NC:path="" />
  <RDF:Seq RDF:about="urn:mimetypes:root">
    <RDF:li RDF:resource="urn:mimetype:application/pdf"/>
  </RDF:Seq>
</RDF:RDF>

After saving a MSWord attachment, the mimeTypes.rdf contents is as follows:

<?xml version="1.0"?>
<RDF:RDF xmlns:NC="http://home.netscape.com/NC-rdf#"
         xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <RDF:Description RDF:about="urn:mimetypes">
    <NC:MIME-types RDF:resource="urn:mimetypes:root"/>
  </RDF:Description>
  <RDF:Description RDF:about="urn:mimetype:handler:application/msword"
                   NC:alwaysAsk="true"
                   NC:saveToDisk="true">
    <NC:externalApplication RDF:resource="urn:mimetype:externalApplication:application/msword"/>
  </RDF:Description>
  <RDF:Description RDF:about="urn:mimetype:externalApplication:application/msword"
                   NC:prettyName=""
                   NC:path="" />
  <RDF:Seq RDF:about="urn:mimetypes:root">
    <RDF:li RDF:resource="urn:mimetype:application/msword"/>
  </RDF:Seq>
  <RDF:Description RDF:about="urn:mimetype:application/msword"
                   NC:value="application/msword"
                   NC:editable="true"
                   NC:fileExtensions="doc"
                   NC:description="Document Microsoft Word">
    <NC:handlerProp RDF:resource="urn:mimetype:handler:application/msword"/>
  </RDF:Description>
</RDF:RDF>



Expected Results:  
The mimeTypes.rdf should have remaned unchanged

Comment 1

[Ludovic Hirlimann [:Usul]]

DO you run MS word on your machine ?

And is the reference to PDF completely removed in the second example ?

Does it happens in safe mode? (see https://support.mozillamessaging.com/en-US/kb/Safe+Mode for more information)

Comment 2

[anne.le-guellec]

Yes I sometimes run MS word on my machine.

The reference to PDF is not completely removed in the second example, because it wasn't there in the first place. To make my tests, I re-started each time with a default mimeTypes.rdf, which is the minimal one re-created by Thunderbird when I delete the existing one aud relaunch my mail. It looks as follows, and works very well for all the attachments I have tested, no addition seems necessary:

 <?xml version="1.0"?>  

<!--
 This file is used as a persistent data store for helper application
 information.

 The root of the data is the <RDF:Seq about="urn:mimetypes:root"/>. This
 contains one <RDF:li/> entry per MIME type.  Each <RDF:li/> entry corresponds
 to the "urn:mimetype:major/minor" resource, where "major/minor" is the MIME
 type.  For example, for HTML we would have "urn:mimetype:text/html".
 Typically, this resource will be in the <RDF:Description/> node which has the
 corresponding "about" attribute.

 Each "urn:mimetype:major/minor" resource can have the following properties:

   NC:Value - the MIME type string
   NC:editable - a "true" or "false" depending on whether this entry is
                 editable
   NC:description - a description of the type ("HTML Document" for text/html)
   NC:fileExtensions - there will be one of these properties per extension that
                       corresponds to this MIME type, each one having a single
                       extension as its value.
   NC:handlerProp - the way the type should be handled.  This corresponds to a
                    "urn:mimetype:handler:major/minor" resource.  Eg, the way
                    HTML is handled would be stored in the
                    "urn:mimetype:handler:text/html" resource

 Each "urn:mimetype:handler:major/minor" resource can have the following
 properties:

   NC:useSystemDefault - "true" if we should handle per default OS setting,
                          "false" or not set otherwise
   NC:saveToDisk - "true" if the data should be saved to disk, "false" or not
                   set otherwise.
     (Note - if both of these are false, that means "open in helper app")
   NC:alwaysAsk - "true" if the user should always be prompted before handling
                  data of this type, false otherwise.
   NC:externalApplication - the helper application to use for this type.  This
                            corresponds to a
                            "urn:mimetype:externalApplication:major/minor"
			    resource

 Each "urn:mimetype:externalApplication:major/minor" resource can have the
 following properties:

   NC:path - the path to the application
   NC:prettyName - the "pretty name" of the application ("Acrobat Reader" for
                   /usr/bin/acroread, eg).
-->
			
<RDF:RDF xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:NC="http://home.netscape.com/NC-rdf#">

  <RDF:Description about="urn:mimetypes"> 
    <NC:MIME-types> 
      <RDF:Seq about="urn:mimetypes:root"> 
      </RDF:Seq>
    </NC:MIME-types> 
  </RDF:Description> 
</RDF:RDF>



I have tested in safe mode, and it modifies the mimeTypes.rdf too when attachments are saved the first way.

Comment 3

[diesmo]

Hi,

Here is a very simple fix for this problem.

It's provided as an extension :
http://diesmo.free.fr/misc/fix-open-bad-mime-attachments-1.0.0.xpi

Before :
 - when clicking on "open" for an attachment, thunderbird used the mime-type, even though it has the exact path to the attachment to open (the message  + the mime part in the message, eg part=1.2);

Now :
 - we "null" the content-type, so that thunderbird does not use it.

Results :
 - the attachment opens correctly;
 - if you double-click the attachment and choose "Save", it works also;
 - if you click "remember the choice" the mimetypes.rdf gets the correct mime-type.
 - you still need to clean the mimetypes.rdf if you had corrupted entries.
 - for old emails with corrupted mime-types for attachments, you can open the attachments;

I hope this helps ?

Cheers,
diesmo.

Comment 4

[diesmo]

A sample test message :
http://diesmo.free.fr/misc/PDF-bad-mime.eml

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
> A sample test message :

> Content-Type: =?windows-1252?q?application/pdf;
>  name="Document-de-test.pdf"
RFC2047 encoding of mime-type is not permitted, so this is unknown/bad mime-type. Sender side bug.
If you ask Tb to create mimeTypes.rdf entry by "Do it always", entry for this bad mime-type and .pdf is create in mimeTypes.rdf according to your request.

Bug is already opened for protection from creation of such entry due to bad mail.
Bug is already opened for protection of user after creation of such entry due to bad mail.
"delete of action" to delete such bad entry at Tools/Options/Attachments is already available in Tb 3.1. So, user can remove bad entry even after such entry is creted due to bad mail like sample test message.

Comment 6

[diesmo]

The end-user does not ask anything, they just open the attachment. And they probably won't edit the source of every message to check that the encoding is fine.

The proposed fix is very simple, Mozilla guys may use it or not, i just propose it.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #6)
> The end-user does not ask anything, they just open the attachment.

No. End user did ask "do it always", in addition to "just open".
Problem is;
User can't know what happens by it, and user surely doesn't know what bad will happen by it if mime-type or file extension in mail is bad, and user surely doesn't view message source to know whether mime-type in mail is good or bad.
Even though so, Tb unconditionally creates mimeTypes.rdf entry for bad or unwanted mime-type if "do it always" is requested by user, and Tb unconditionally uses it with highest priority once bad entry is created.
And, once created, it's hard to know "what kind of wrongness exists" via Tools/Option/Attachments UI, and, even if user noticed about bad entry, "how to delete entry" is not so obvious because it's placed as an action item in action drop down menu which is for Tb's behaviour when the mime-type is detected in mail("Choice of an action is similar to "Edit".)

Behaviour on mimeTypes.rdf around the "do it always" was imported from Firefox. As mime-type is obtained from HTTP(s) server when browser, mime-type is correct in almost all cases. So, problem due to bad mime-type is very exceptional if browser(note: never "not occur" even if browser. problems due to bad mime-type were really reported for browser too, and still exists).
See bug 417590 comment #6, and bug 417590 comment #8 which is answer to it, please.

Bug 462629 is for concern like bug 417590 comment #6.
Bug 462629 is for "once created" phase.

I think your proposal is a kind of "extension to the dialog" stated in Bug 417590 comment #8, although "dialog" in that context is dialog at Tools/Options/Attachments if Thunderbird, and "dialog" in your extension is "open/save dialog".
Your fix seems for improvements in "creation" phase. No improvements in "use of entry" phase after creation by your patch?

> The proposed fix is very simple, Mozilla guys may use it or not, i just propose it.

To land your fix on Tb, your extention should be approved as standard/embed extention, or should be converted to a patch and approved by developers.
I think your extention is better provided as ordinal add-on first, like "Extra Folder Columns", "FiltaQuilla", at addons.mozilla.org.

Comment 8

[Wayne Mery (:wsmwk)]

Reporter, 

do you still see this problem when using a current version of thunderbird? 

And if you do not, please close bug report by changing resolution to worksforme

Comment 9

[Joshua Cranmer [:jcranmer]]

RESOLVED INCOMPLETE due to lack of response to last question. If you feel this change was made in error, please respond to this bug with your reasons why.

Comment 10

[Fabian Keßler]

(In reply to Joshua Cranmer [:jcranmer] from comment #9)
> RESOLVED INCOMPLETE due to lack of response to last question. If you feel
> this change was made in error, please respond to this bug with your reasons
> why.

This bug is still a problem and is not solved. Thunderbird should be forced to use the standard mimetypes, eg. defined on http://www.sitepoint.com/web-foundations/mime-types-complete-list/ instead of using probably wrong saved ones in the mimetypes.rdf

Comment 11

[Kent James (:rkent)]

There is clearly some sort of issue in Thunderbird that I see in support requests all of the time, that sent messages have odd mimetypes because of presumably unexpected changes to mimeTypes.rdf. Is that this bug, or is there another that tracks this?

Comment 12

[Fabian Keßler]

I have found a lot of bug reports since 2006 that describe the same problem in the core. The most are ignored or reported as duplicate. But noone whanted to fix it. Another report is the Bug 503309 which contains more informations and a hint how to solve this problem.