538407 - Attachments are not accessible for 'Content-Type:application/octet-stream, Content-Disposition:inline;filename="xxxx.yyy"', if mime-type for ".yyy" is unknown

Status: RESOLVED FIXED

(Thunderbird :: Message Reader UI, defect)

Description

[Vladimir]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6
Build Identifier: Gecko/20100107 Lanikai/3.1a1pre

Attachments show as a raw MIME after a rectangular divider following the message body.

Thunderbird "Gecko/20091204 Thunderbird/3.0" does not exhibit the same behavior, i.e. attachments are processed properly.

Reproducible: Always

Steps to Reproduce:
1. Open the message with an attachment defined as 'Content-Disposition: inline.'
2. Observe that attachment cannot be accessed
3.
Actual Results:  
Attachments cannot be accessed.

Expected Results:  
Attachments should be accessible.

A sample offending message with MIME partially skipped:

Return-path: <[e-mail]>
Received: from xxxxxx.com (xxxxxx.com [192.168.0.9])
	by xxxxxx.com with SMTP; Thu, 07 Jan 2010 02:56:34 -0600
From: <[e-mail]>
Date: Thu, 07 Jan 2010 02:56:33 -0600
Subject: CA Alert Notification
To: <[e-mail]>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=inoculateit_default_boundary

--inoculateit_default_boundary
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit

COMM-KDH: [JobID:0 ] PE2800: I0013 Job 1058: Rotation BackUp Operation Successful.
--inoculateit_default_boundary
Content-Type: application/octet-stream; name="Job1058.log"
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="Job1058.log"

QmFja3VwIE9wZXJhdGlvbg0KDQpKYW4tMDYgMTk6NTY6MTAgICAxMDU4ICAg
ICBGb3JtYXQgbWVkaWEgV0VETkVTREFZIFtzZXJpYWwgIyAxNTg2NTg5ODE1
XSBhcw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgV0VETkVTREFZLg0K
SmFuLTA2IDE5OjU4OjEwICAgMTA1OCAgICAgRGlzYXN0ZXIgUmVjb3Zlcnkg
.........
.........
IDENCjMuIFdFRE5FU0RBWSAgICAgICAgICAgICAgIDE1ODY1ODk4MDQgICAg
IFJPVEFUSU9OICAgICAxDQo0LiBNT05EQVkgICAgICAgICAgICAgICAgICAx
NTg2NTg5ODAyICAgICBST1RBVElPTiAgICAgMQ0KNS4gPEJMQU5LPg0KSmFu
LTA3IDAyOjU1OjQ1ICAgMTA1OCAgICAgSm9iIDEwNTg6IFJvdGF0aW9uIEJh
Y2tVcCBPcGVyYXRpb24gU3VjY2Vzc2Z1bA0K




--inoculateit_default_boundary--

Comment 1

[Vladimir]

Thunderbird "Gecko/20091204 Thunderbird/3.0" is fine.

Lanikai was having this problem at least as of 01/05/2010 nightly build.

Comment 2

[Joe Sabash [:JoeS1]]

It's difficult (if not impossible) to construct a message in Thunderbird with:
Content-Type: application/octet-stream;

Could you provide an eml example of the problem mail (keeping in mind privacy as to content)

Windows Live Mail seems to send this content-type for inline attachments, and I might have seen this on some newsgroups.

Comment 3

[Vladimir]

Here it comes.

Return-path: <[e-mail]>
Received: from xxxxxxxxxxx.com (xxxxxxxxxx.com [192.168.0.9])
	by xxxxxxxxxxxx.com with SMTP; Thu, 19 Jun 2008 23:27:19 -0500
From: <[e-mail]>
Date: Thu, 19 Jun 2008 23:27:19 -0500
Subject: CA Alert Notification
To: <[e-mail]>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=inoculateit_default_boundary

--inoculateit_default_boundary
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit

COMM-KDH: [JobID:0 ] PE2800: E0015 Job 581: Rotation MakeUp Operation Failed.
--inoculateit_default_boundary
Content-Type: application/octet-stream; name="Job581.log"
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="Job581.log"

SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv
ZiB0aGUgZm9sbG93aW5nIG1lZGlhIGluIGRldmljZQ0KICAgICAgICAgICAg
ICAgICAgICAgICAgICAgZ3JvdXAgQUlULTM6DQpKdW4tMTkgMjM6MjI6MjAg
ICAgNTgxICAgICBKb2Igd2lsbCBiZSBjYW5jZWxsZWQgaW4gNSBtaW51dGVz
DQpKdW4tMTkgMjM6MjI6NTAgICAgNTgxICAgICBQbGVhc2UgbW91bnQgb25l
IG9mIHRoZSBmb2xsb3dpbmcgbWVkaWEgaW4gZGV2aWNlDQogICAgICAgICAg
ICAgICAgICAgICAgICAgICBncm91cCBBSVQtMzoNCkp1bi0xOSAyMzoyMzoz
MCAgICA1ODEgICAgIFBsZWFzZSBtb3VudCBvbmUgb2YgdGhlIGZvbGxvd2lu
ZyBtZWRpYSBpbiBkZXZpY2UNCiAgICAgICAgICAgICAgICAgICAgICAgICAg
IGdyb3VwIEFJVC0zOg0KSnVuLTE5IDIzOjI0OjEwICAgIDU4MSAgICAgUGxl
YXNlIG1vdW50IG9uZSBvZiB0aGUgZm9sbG93aW5nIG1lZGlhIGluIGRldmlj
ZQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAgZ3JvdXAgQUlULTM6DQpK
dW4tMTkgMjM6MjQ6NTAgICAgNTgxICAgICBQbGVhc2UgbW91bnQgb25lIG9m
IHRoZSBmb2xsb3dpbmcgbWVkaWEgaW4gZGV2aWNlDQogICAgICAgICAgICAg
ICAgICAgICAgICAgICBncm91cCBBSVQtMzoNCkp1bi0xOSAyMzoyNTozMCAg
ICA1ODEgICAgIFBsZWFzZSBtb3VudCBvbmUgb2YgdGhlIGZvbGxvd2luZyBt
ZWRpYSBpbiBkZXZpY2UNCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGdy
b3VwIEFJVC0zOg0KSnVuLTE5IDIzOjI2OjEwICAgIDU4MSAgICAgUGxlYXNl
IG1vdW50IG9uZSBvZiB0aGUgZm9sbG93aW5nIG1lZGlhIGluIGRldmljZQ0K
ICAgICAgICAgICAgICAgICAgICAgICAgICAgZ3JvdXAgQUlULTM6DQpKdW4t
MTkgMjM6MjY6NTAgICAgNTgxICAgICBQbGVhc2UgbW91bnQgb25lIG9mIHRo
ZSBmb2xsb3dpbmcgbWVkaWEgaW4gZGV2aWNlDQogICAgICAgICAgICAgICAg
ICAgICAgICAgICBncm91cCBBSVQtMzoNCkp1bi0xOSAyMzoyNzoxMCAgICA1
ODEgICAgIFRoZSBmb2xsb3dpbmcgbWVkaWEgY2FuIGJlIHVzZWQgaW4gZGV2
aWNlIGdyb3VwDQogICAgICAgICAgICAgICAgICAgICAgICAgICBBSVQtMyBm
b3IgbmV4dCBqb2Igb24gZGF0ZSAwNi8yMC8wOCAyMToyMjowNA0KICAgTUVE
SUEgTkFNRSAgICAgICAgICAgICAgU0VSSUFMICMgICAgICAgTUVESUEgUE9P
TCAgU0VRDQoxLiBGUklEQVkgICAgICAgICAgICAgICAgICAxNTg2NTg5ODAx
ICAgICBST1RBVElPTiAgICAgMQ0KMi4gV0VETkVTREFZICAgICAgICAgICAg
ICAgMTU4NjU4OTgwNCAgICAgUk9UQVRJT04gICAgIDENCjMuIE1PTkRBWSAg
ICAgICAgICAgICAgICAgIDE1ODY1ODk4MDIgICAgIFJPVEFUSU9OICAgICAx
DQo0LiBUSFVSU0RBWSAgICAgICAgICAgICAgICAxNTg2NTg5ODA4ICAgICBS
T1RBVElPTiAgICAgMQ0KNS4gPEJMQU5LPg0K




--inoculateit_default_boundary--

Comment 4

[Joe Sabash [:JoeS1]]

I can confirm that the testcase message works in:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100104 Shredder/3.0.1pre ID:20100104034505

And fails in:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20100104 Shredder/3.2a1pre
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b6pre) Gecko/20100104 Lanikai/3.1a1pre

Not sure if this isn't a core gecko issue, rather than a thunderbird bug,
but confirming to keep it on the radar.

Comment 5

[Joe Sabash [:JoeS1]]

Attachment: testcase eml

Comment 6

[Joe Sabash [:JoeS1]]

Vladimir,
I have resolved this problem by taking the following steps:
1 Send yourself a message with an attachment with a file extension of .log
2 Double click on the attachment, and pick an application to open it
3 Click on the "always do this" checkbox

This will add a relationship in MimeTypes.rdf which you will see in:
Tools>>Options>>Attachments
After that it should work OK for you.

So the real problem is, that we must be doing a check in MimeTypes.rdf for a valid extension type in the case of application/octet-stream.
And if not found there, not recognizing the attachment.

I don't know really where the problem is based, Gecko core or TB
But without the MimeTypes entry the attachment is not handled properly.

Comment 7

[Vladimir]

Joe,

Unfortunately it did not work for me. In fact, I believe I discovered a couple new bugs in the course of trying your solution.

First, I saved Job581.log as a file. Then I sent it as an attachment using Thunderbird 2.0.0.23. The attachment came as:

Content-Type: text/plain;
 name="Job581.log"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="Job581.log"

It showed in the message body and was processed as a valid attachment.

Upon double click on the attachment I was presented with the dialog requesting to pick an application. I chose notepad.exe which failed to open the attachment with the following message "The filename, directory name, or volume label syntax is incorrect."

In Options/Attachments I discovered the Content Type "imap (imap)" with Action "Use Notepad"

Another UI problem. There was no vertical scroll bar in the "Attachments" screen which would not allow to review all content types.

Comment 8

[Vladimir]

Joe,

In order to complete the experiment I sent the same message to a POP account. The message I received was the same as above. In Options/Attachments the Content Type was "maibox (mailbox)."

Vladimir

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> Content-Type: text/plain;
> Upon double click on the attachment I was presented with the dialog requesting
> to pick an application. I chose notepad.exe which failed to open the attachment

It's bug 533462.

> --inoculateit_default_boundary
> Content-Type: application/octet-stream; name="Job581.log"
> Content-Transfer-Encoding: base64
> Content-Disposition: inline; filename="Job581.log"

To all problem reporters.
What is your choice of View/Display Attachments Inline?
No attachment is displayed in attachment box even if "Display Attachments Inline"=off?

Comment 10

[Joe Sabash [:JoeS1]]

View attachments inline makes no difference.
I am using a test profile with nothing but a default mimeTypes.rdf
The test builds were made from zip builds, and are not set to be the default
registered programs.
With an "untrained" mimetypes.rdf (see attached screenshot) it appears to display the attachment inline as the content-disposition suggests.
Also, Tools>> Options>> Attachments is blank, not showing any of my windows file associations. 
I'm quite sure that just copying a "good" mimetypes.rdf from another profile would fix things, but I'll keep things broken there for testing.

Comment 11

[Joe Sabash [:JoeS1]]

Attachment: screenshot

Comment 12

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #11)
> screenshot

Phenomeon looks:
 1. If "inline" is specified, Tb displays it in "inline", even though
    Content-Type:application/octet-stream;.
 2. Content-Transfer-Encoding:base64 is not processed correctly upon
    display. Data before Base64 decoding(row string in the part) is used.
Why application/octet-stream is considered displayable in inline?
Content-sniffing when application/octet-stream is done on data before Base64 decoding(row string in the part)?

(In reply to comment #10)
> I'm quite sure that just copying a "good" mimetypes.rdf from another profile would fix things,

Does it mean that one of next is a workaround?
  association of ".log" extenstion to "text/xxx" mime-type in mimeTypes.rdf
  association of "application/octetstream" to a file extension in mimeTypes.rdf

Comment 13

[Joe Sabash [:JoeS1]]

(In reply to comment #12)
> (In reply to comment #11)
> > screenshot
> 
> Phenomeon looks:
>  1. If "inline" is specified, Tb displays it in "inline", even though
>     Content-Type:application/octet-stream;.
>  2. Content-Transfer-Encoding:base64 is not processed correctly upon
>     display. Data before Base64 decoding(row string in the part) is used.


> Why application/octet-stream is considered displayable in inline?
> Content-sniffing when application/octet-stream is done on data before Base64
> decoding(row string in the part)?

I'm not sure, but I have a windows file association of .log with notepad.
(this does not show up in the attachment pref pane)
 
> (In reply to comment #10)
> > I'm quite sure that just copying a "good" mimetypes.rdf from another profile would fix things,

 
> Does it mean that one of next is a workaround?
>   association of ".log" extenstion to "text/xxx" mime-type in mimeTypes.rdf
>   association of "application/octetstream" to a file extension in mimeTypes.rdf

This was an incorrect assumption on my part.
I copied my "working" (my everyday) mimetypes.rdf into the TB3.1 profile and the symptoms remained the same. However if I execute the steps in my comment 6 then the inline attachment shows as an attachment, and not inline.

Wada, what happens for you if you save the testcase eml and then open it?

Comment 14

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #13)
> Wada, what happens for you if you save the testcase eml and then open it?

I'm unable to produce your problem with Tb 3.0 release build on MS Win, with clean mimeTypes.rdf(mimeTypes.rdf is deleted before restart of Tb). The attachment is displayed in attachment box, and "text document" is displayed at open dialog and "Notepad" is displayed as defaulted application(obtained from HKEY_CLASSES_ROOT\.log).

But I could observe this bug with Tb 3.2a1pre.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20100109 Shredder/3.2a1pre

> I copied my "working" (my everyday) mimetypes.rdf into the TB3.1 profile and
> the symptoms remained the same.
> However if I execute the steps in my comment 6 then the inline attachment shows as an attachment, and not inline.

(your workaround in comment#6)
> I have resolved this problem by taking the following steps:
> 1 Send yourself a message with an attachment with a file extension of .log
> 2 Double click on the attachment, and pick an application to open it
> 3 Click on the "always do this" checkbox
If Content-Type:application/octet-stream;name="xxx.log", "always do this" checkbox is grayed out at dialog. You probably did it for Content-Type: text/plain; name="xxx.log".
  association of ".log" and "text/plain" is generated in mimeTypes.rdf
It looks, as you say, next;
- If association to ".log" is found in mimeTypes.rdf, Tb properly considers
  that application/octet-stream is not able to display in inline.
- If association to ".log" is not found in mimeTypes.rdf, Tb may try to fall
  back to text/plain, and check data before base64 decoding.
  ("base64 is not decoded" is seen in bugs, including bugs relevant to search)  
  As base64 part is 7bit ascii-only, data before base64 decoding is displayed.
In latter case, if the application/octet-stream;name="...log" part is sent in pure text without base64 encoding, text in the part is displayed in inline. It may be problem of lack of care for Content-Transfer-Encoding:base64, in quirks for application/octet-stream.

Comment 15

[WADA:World Anti-bad-Duping Agency]

Base64, or quoted-printable. or no content-transfer-encoding, was irrelevant to phenomenon. In any case, if association of file extension to a mime-type is not defined in mimeTypes.rdf, Tb 3.2 displayed string in the application/octet-stream in inline, without decoding if encoded.
  No asssociation of ".log"(and ".logx") in mimeTypes.rdf;
    Unknown mime-type(abc/def, filename="...log")   
      Displayed as attachment
    text/plain(filename="...log")   
      Displayed according to "View/Display Attachments Inline" setting
    application/octet-stream(filename="...log", filename="...logx")   
      This bug occurs
  After creation of asssociation of ".log"(to abc/def) in mimeTypes.rdf;
    application/octet-stream(filename="...log")   
      Displayed as attachment
    application/octet-stream(filename="...logx")   
      This bug still occurs

If content-sniffing for application/octet-stream is executed,
  - It should be executed on data after decoding, if data is encoded.
  - If inline display is done after sniffing, check for "text only" should be
    done. However, how charset can be determined? Invoke auto-detect?
If quirks for application/octet-stream(mainly for filename="...pdf" etc.) is required, quirks should work properly in this bug's case.
  - No association of file extention to any mime-type in mimeTypes.rdf
    and in predefined(internally defined) mime-types.

Comment 16

[WADA:World Anti-bad-Duping Agency]

Adding Content-Type:application/octet-strean to bug summary for ease of search.

Comment 17

[WADA:World Anti-bad-Duping Agency]

Following is open/fixed/dup bugs which has "application/octet-stream" in bug summary and which is updated in these three months. 
> bug 327323 2009-10-22 RESO Can't "Open with" files that are send as application/octet-stream (or other "unknown to firefox" mime types)
> bug 517796 2009-11-07 NEW  thunderbird does not let me view application/octet-stream attachments
> bug 532603 2009-12-18 RESO plain text attachment with application/octet-stream Content-Type does not shown inline

Regression by bug 532603?

Comment 18

[Joe Sabash [:JoeS1]]

Regression Range:
Works:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20091218 Shredder/3.1a1pre ID:20091218032714
This bug's symptoms:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20091219 Shredder/3.1a1pre ID:20091219031754

Comment 19

[Jie Zhang]

(In reply to comment #17)
> Regression by bug 532603?

I'm using thunderbird 3.0 release + the fix for bug 532603. I tested with the testcase eml in this bug. It looks OK for me. The attachment is shown inline and can be saved as a file.

Comment 20

[Joe Sabash [:JoeS1]]

(In reply to comment #19)
> (In reply to comment #17)
> > Regression by bug 532603?
> 
> I'm using thunderbird 3.0 release + the fix for bug 532603. I tested with the
> testcase eml in this bug. It looks OK for me. The attachment is shown inline
> and can be saved as a file.

Your patch was checked into comm-central branch (TB3.1 TB3.2) not TB3.0 source.
While it might be working perfectly well for you, it seems to have caused this regression here.

Comment 21

[WADA:World Anti-bad-Duping Agency]

http://hg.mozilla.org/comm-central/pushloghtml?startdate=2009-12-18&enddate=2009-12-19+12%3A00
> Changes pushed after 2009-12-18 00:00:00, before 2009-12-19 12:00:00
> Sat Dec 19 06:44:17 2009 -0800	d4ca11753853	Serge Gautherie — Bug 496225 - Replace MOZ_DISABLE_VISTA_SDK_REQUIREMENTS in comm-central; (Av1-SM) Just replace it. r=kairo. [e-mail]
> Fri Dec 18 07:55:33 2009 -0800	fb0f87f848a8	Jie Zhang — fix bug 532603, r=bienvenu, sr=neil, plain text attachment with application/octet-stream content type doesn't show inline [e-mail]
> Fri Dec 18 01:50:24 2009 -0800	6368ab714c7c	Mark Banner — Backout bug 534462 / changesets c4775a7bc5bf and a4fec503c731 due to bloat test bustage.

Comment 22

[Jie Zhang]

Attachment: Screenshot on Debian AMD64 unstable

Comment 23

[Jie Zhang]

I just updated source code from trunk and built a new thunderbird to test it. It works for me on my Debian AMD64 unstable. I attached my screenshot. So it seems a problem on Windows. But I have no experience on building thunderbird on Windows.

Comment 24

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #23)
> It works for me on my Debian AMD64 unstable. I attached my screenshot. So it
> seems a problem on Windows.

Following is additional test result for ".logx" on Win.
  After creation of asssociation of ".logx"(to text/plain) in mimeTypes.rdf;
    application/octet-stream(filename="...logx")   
       Displayed according to "View/Display Attachments Inline" setting.
       Decoded data is displayed when inline display. 
Do you have association of ".log" to "text/plain" in your mimeTypes.rdf?
Linux has association of ".log" to "text/plain" at system level? (Windows Registry if MS Win)

Comment 25

[Jie Zhang]

How can I know if Linux has association of ".log" to "text/plain" at system level? I have no knowledge in this area.

Comment 26

[Jie Zhang]

Neither mime.types nor the mimeTypes.rdf in my profile indicates that ".log" has type "text/plain".

Comment 27

[Jie Zhang]

Just one more point: if ".log" is not associated with "text/plain" on Windows, thunderbird should not display it inline.

Comment 28

[Jie Zhang]

Just vague guessing. It seems like Content-Type has been decided as "text/plain" on both Linux and Windows. But thunderbird forget to decode it according to Content-Transfer-Encoding, which is base64, before displaying it inline on Windows. Which piece of code handles this?

Comment 29

[Jie Zhang]

Below is the backtrace of the stack on my machine. If anyone, who has a windows machine to work on, can set a breakpoint in the middle of the call sequence and track this bug down, I will appreciate.

(gdb) bt
#0  mime_decode_base64_buffer (data=0x7fb1a3573700, buffer=0x7fb1a356b000 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n", length=60) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeenc.cpp:255
#1  0x00007fb1b258fee0 in MimeDecoderWrite (data=0x7fb1a3573700, buffer=0x7fb1a356b000 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n", size=60) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeenc.cpp:838
#2  0x00007fb1b259a159 in MimeLeaf_parse_buffer (buffer=0x7fb1a356b000 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n", size=60, obj=0x7fb1a8b2b9e0) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeleaf.cpp:174
#3  0x00007fb1b25a4e48 in MimeMultipart_parse_child_line (obj=0x7fb1ae5ed790, line=0x7fb1a356b000 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n", length=60, first_line_p=1) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemult.cpp:740
#4  0x00007fb1b25a433a in MimeMultipart_parse_line (line=0x7fb1a356b000 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n", length=61, obj=0x7fb1ae5ed790) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemult.cpp:419
#5  0x00007fb1b25b0bda in convert_and_send_buffer (buf=0x7fb1a356b000 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n", length=61, convert_newlines_p=1, per_line_fn=0x7fb1b25a310a <MimeMultipart_parse_line>, closure=0x7fb1ae5ed790) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:184
#6  0x00007fb1b25b0e96 in mime_LineBuffer (net_buffer=0x7fb1a351c400 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n\n", net_buffer_size=61, bufferP=0x7fb1ae5ed7d0, buffer_sizeP=0x7fb1ae5ed7e0, buffer_fpP=0x7fb1ae5ed7e8, convert_newlines_p=1, per_line_fn=0x7fb1b25a310a <MimeMultipart_parse_line>, closure=0x7fb1ae5ed790) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:272
#7  0x00007fb1b25a5c8a in MimeObject_parse_buffer (buffer=0x7fb1a351c400 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n\n", size=61, obj=0x7fb1ae5ed790) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeobj.cpp:275
#8  0x00007fb1b259fb03 in MimeMessage_parse_line (aLine=0x7fb1a351c400 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n\n", aLength=61, obj=0x7fb1ae5ed040) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemsg.cpp:232
#9  0x00007fb1b25b0bda in convert_and_send_buffer (buf=0x7fb1a351c400 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\n\n", length=61, convert_newlines_p=1, per_line_fn=0x7fb1b259f78f <MimeMessage_parse_line>, closure=0x7fb1ae5ed040) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:184
#10 0x00007fb1b25b0e96 in mime_LineBuffer (net_buffer=0x7fb1a352a3f8 "SnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\r\nZiB0aGUgZm9sbG93aW5nIG1lZGlhIGluIGRldmljZQ0KICAgICAgICAgICAg\r\nICAgICAgICAgICAgICAgZ3JvdXAgQUlULTM6DQpKdW4tMTkgMjM6MjI6MjAg\r\nICAgNTgxICAgIC"..., net_buffer_size=2064, bufferP=0x7fb1ae5ed080, buffer_sizeP=0x7fb1ae5ed090, buffer_fpP=0x7fb1ae5ed098, convert_newlines_p=1, per_line_fn=0x7fb1b259f78f <MimeMessage_parse_line>, closure=0x7fb1ae5ed040) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:272
#11 0x00007fb1b25a5c8a in MimeObject_parse_buffer (buffer=0x7fb1a352a000 "X-Mozilla-Status: 0801\r\nX-Mozilla-Status2: 00000000\r\nX-Mozilla-Keys:", ' ' <repeats 81 times>, "\r\nFCC: mailbox://nobody@Local%20Folders/Sent\r\nX-Ide"..., size=3080, obj=0x7fb1ae5ed040) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeobj.cpp:275
#12 0x00007fb1b25b3802 in mime_display_stream_write (stream=0x7fb1a33e7ec0, buf=0x7fb1a352a000 "X-Mozilla-Status: 0801\r\nX-Mozilla-Status2: 00000000\r\nX-Mozilla-Keys:", ' ' <repeats 81 times>, "\r\nFCC: mailbox://nobody@Local%20Folders/Sent\r\nX-Ide"..., size=3080) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemoz2.cpp:949
#13 0x00007fb1b25c467f in nsStreamConverter::OnDataAvailable (this=0x7fb1ac2c2300, request=0x7fb1a33cb248, ctxt=0x7fb1ac3527c0, aIStream=0x7fb1a8c6b358, sourceOffset=0, aLength=3080) at /home/jie/sources/thunderbird/src/mailnews/mime/src/nsStreamConverter.cpp:957
#14 0x00007fb1b83d5861 in nsDocumentOpenInfo::OnDataAvailable (this=0x7fb1abc32d30, request=0x7fb1a33cb248, aCtxt=0x7fb1ac3527c0, inStr=0x7fb1a8c6b358, sourceOffset=0, count=3080) at /home/jie/sources/thunderbird/src/mozilla/uriloader/base/nsURILoader.cpp:306
#15 0x00007fb1b08eaf43 in nsMailboxProtocol::ReadMessageResponse (this=0x7fb1a33cb240, inputStream=0x7fb1a8c6b358, sourceOffset=0, length=3080) at /home/jie/sources/thunderbird/src/mailnews/local/src/nsMailboxProtocol.cpp:585
#16 0x00007fb1b08eb3e5 in nsMailboxProtocol::ProcessProtocolState (this=0x7fb1a33cb240, url=0x7fb1ac3527c8, inputStream=0x7fb1a8c6b358, offset=0, length=3080) at /home/jie/sources/thunderbird/src/mailnews/local/src/nsMailboxProtocol.cpp:682
#17 0x00007fb1c0f09c4e in nsMsgProtocol::OnDataAvailable (this=0x7fb1a33cb240, request=0x7fb1ad723e80, ctxt=0x7fb1ac3527c0, inStr=0x7fb1a8c6b358, sourceOffset=0, count=3080) at /home/jie/sources/thunderbird/src/mailnews/base/util/nsMsgProtocol.cpp:359
#18 0x00007fb1be60d4bf in nsInputStreamPump::OnStateTransfer (this=0x7fb1ad723e80) at /home/jie/sources/thunderbird/src/mozilla/netwerk/base/src/nsInputStreamPump.cpp:508
#19 0x00007fb1be60d013 in nsInputStreamPump::OnInputStreamReady (this=0x7fb1ad723e80, stream=0x7fb1a8c6b358) at /home/jie/sources/thunderbird/src/mozilla/netwerk/base/src/nsInputStreamPump.cpp:398
#20 0x00007fb1cf28f4f5 in nsInputStreamReadyEvent::Run (this=0x7fb1a33e7580) at /home/jie/sources/thunderbird/src/mozilla/xpcom/io/nsStreamUtils.cpp:112
#21 0x00007fb1cf2c09a1 in nsThread::ProcessNextEvent (this=0x7fb1c72311f0, mayWait=1, result=0x7fff2061677c) at /home/jie/sources/thunderbird/src/mozilla/xpcom/threads/nsThread.cpp:527
#22 0x00007fb1cf244fcd in NS_ProcessNextEvent_P (thread=0x7fb1c72311f0, mayWait=1) at nsThreadUtils.cpp:250
#23 0x00007fb1ba214a77 in nsBaseAppShell::Run (this=0x7fb1b2172e40) at /home/jie/sources/thunderbird/src/mozilla/widget/src/xpwidgets/nsBaseAppShell.cpp:177
#24 0x00007fb1b5b571b1 in nsAppStartup::Run (this=0x7fb1b21d88d0) at /home/jie/sources/thunderbird/src/mozilla/toolkit/components/startup/src/nsAppStartup.cpp:182
#25 0x00007fb1cf79ca62 in XRE_main (argc=1, argv=0x7fff206170d8, aAppData=0x7fb1c7221080) at /home/jie/sources/thunderbird/src/mozilla/toolkit/xre/nsAppRunner.cpp:3476
#26 0x0000000000401d4e in main (argc=1, argv=0x7fff206170d8) at /home/jie/sources/thunderbird/src/mail/app/nsMailApp.cpp:103

Comment 30

[Jie Zhang]

It comes from GNOME:

(gdb) p mimeType 
$28 = 0x7f447ec1f8f8 "text/x-log"
(gdb) bt
#0  nsGnomeVFSService::GetMimeTypeFromExtension (this=0x7f447ce2b9a0, aExtension=..., aMimeType=...) at /home/jie/sources/thunderbird/src/mozilla/toolkit/system/gnome/nsGnomeVFSService.cpp:202
#1  0x00007f448ce9876d in nsGNOMERegistry::GetFromExtension (aFileExt=...) at /home/jie/sources/thunderbird/src/mozilla/uriloader/exthandler/unix/nsGNOMERegistry.cpp:145
#2  0x00007f448ce95616 in nsOSHelperAppService::GetFromExtension (this=0x7f4480bdd6a0, aFileExt=...) at /home/jie/sources/thunderbird/src/mozilla/uriloader/exthandler/unix/nsOSHelperAppService.cpp:1321
#3  0x00007f448ce9692f in nsOSHelperAppService::GetMIMEInfoFromOS (this=0x7f4480bdd6a0, aType=..., aFileExt=..., aFound=0x7fff4ef8116c) at /home/jie/sources/thunderbird/src/mozilla/uriloader/exthandler/unix/nsOSHelperAppService.cpp:1557
#4  0x00007f448ce841c7 in nsExternalHelperAppService::GetTypeFromExtension (this=0x7f4480bdd6a0, aFileExt=..., aContentType=...) at /home/jie/sources/thunderbird/src/mozilla/uriloader/exthandler/nsExternalHelperAppService.cpp:2586
#5  0x00007f4484bcc081 in mime_file_type (filename=0x7f4477998ba0 "Job581.log", stream_closure=0x7f4477520800) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemoz2.cpp:741
#6  0x00007f4484baf0bf in mime_create (content_type=0x7f447799c7c0 "application/octet-stream", hdrs=0x7f447c3f60c0, opts=0x7f4480502f00) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimei.cpp:859
#7  0x00007f4484bbd751 in MimeMultipart_create_child (obj=0x7f44787d3b80) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemult.cpp:523
#8  0x00007f4484bbce09 in MimeMultipart_parse_line (line=0x7f44779a4800 "\n", length=1, obj=0x7f44787d3b80) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemult.cpp:310
#9  0x00007f4484bc9bda in convert_and_send_buffer (buf=0x7f44779a4800 "\n", length=1, convert_newlines_p=1, per_line_fn=0x7f4484bbc10a <MimeMultipart_parse_line>, closure=0x7f44787d3b80) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:184
#10 0x00007f4484bc9e96 in mime_LineBuffer (net_buffer=0x7f44779a2000 "\n\n", net_buffer_size=1, bufferP=0x7f44787d3bc0, buffer_sizeP=0x7f44787d3bd0, buffer_fpP=0x7f44787d3bd8, convert_newlines_p=1, per_line_fn=0x7f4484bbc10a <MimeMultipart_parse_line>, closure=0x7f44787d3b80) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:272
#11 0x00007f4484bbec8a in MimeObject_parse_buffer (buffer=0x7f44779a2000 "\n\n", size=1, obj=0x7f44787d3b80) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeobj.cpp:275
#12 0x00007f4484bb8b03 in MimeMessage_parse_line (aLine=0x7f44779a2000 "\n\n", aLength=1, obj=0x7f44787d2bc0) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemsg.cpp:232
#13 0x00007f4484bc9bda in convert_and_send_buffer (buf=0x7f44779a2000 "\n\n", length=1, convert_newlines_p=1, per_line_fn=0x7f4484bb878f <MimeMessage_parse_line>, closure=0x7f44787d2bc0) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:184
#14 0x00007f4484bc9e96 in mime_LineBuffer (net_buffer=0x7f44779973f6 "\r\nSnVuLTE5IDIzOjIyOjA5ICAgIDU4MSAgICAgUGxlYXNlIG1vdW50IG9uZSBv\r\nZiB0aGUgZm9sbG93aW5nIG1lZGlhIGluIGRldmljZQ0KICAgICAgICAgICAg\r\nICAgICAgICAgICAgICAgZ3JvdXAgQUlULTM6DQpKdW4tMTkgMjM6MjI6MjAg\r\nICAgNTgxICAg"..., net_buffer_size=2066, bufferP=0x7f44787d2c00, buffer_sizeP=0x7f44787d2c10, buffer_fpP=0x7f44787d2c18, convert_newlines_p=1, per_line_fn=0x7f4484bb878f <MimeMessage_parse_line>, closure=0x7f44787d2bc0) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimebuf.cpp:272
#15 0x00007f4484bbec8a in MimeObject_parse_buffer (buffer=0x7f4477997000 "X-Mozilla-Status: 0801\r\nX-Mozilla-Status2: 00000000\r\nX-Mozilla-Keys:", ' ' <repeats 81 times>, "\r\nFCC: mailbox://nobody@Local%20Folders/Sent\r\nX-Ide"..., size=3080, obj=0x7f44787d2bc0) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimeobj.cpp:275
#16 0x00007f4484bcc802 in mime_display_stream_write (stream=0x7f447758c800, buf=0x7f4477997000 "X-Mozilla-Status: 0801\r\nX-Mozilla-Status2: 00000000\r\nX-Mozilla-Keys:", ' ' <repeats 81 times>, "\r\nFCC: mailbox://nobody@Local%20Folders/Sent\r\nX-Ide"..., size=3080) at /home/jie/sources/thunderbird/src/mailnews/mime/src/mimemoz2.cpp:949
#17 0x00007f4484bdd67f in nsStreamConverter::OnDataAvailable (this=0x7f448031ba00, request=0x7f44785575e8, ctxt=0x7f4478797e80, aIStream=0x7f4478545048, sourceOffset=0, aLength=3080) at /home/jie/sources/thunderbird/src/mailnews/mime/src/nsStreamConverter.cpp:957
#18 0x00007f448ce70861 in nsDocumentOpenInfo::OnDataAvailable (this=0x7f44779990b0, request=0x7f44785575e8, aCtxt=0x7f4478797e80, inStr=0x7f4478545048, sourceOffset=0, count=3080) at /home/jie/sources/thunderbird/src/mozilla/uriloader/base/nsURILoader.cpp:306
#19 0x00007f448548ff43 in nsMailboxProtocol::ReadMessageResponse (this=0x7f44785575e0, inputStream=0x7f4478545048, sourceOffset=0, length=3080) at /home/jie/sources/thunderbird/src/mailnews/local/src/nsMailboxProtocol.cpp:585
#20 0x00007f44854903e5 in nsMailboxProtocol::ProcessProtocolState (this=0x7f44785575e0, url=0x7f4478797e88, inputStream=0x7f4478545048, offset=0, length=3080) at /home/jie/sources/thunderbird/src/mailnews/local/src/nsMailboxProtocol.cpp:682
#21 0x00007f4486d2ac4e in nsMsgProtocol::OnDataAvailable (this=0x7f44785575e0, request=0x7f44775f8580, ctxt=0x7f4478797e80, inStr=0x7f4478545048, sourceOffset=0, count=3080) at /home/jie/sources/thunderbird/src/mailnews/base/util/nsMsgProtocol.cpp:359
#22 0x00007f44923d94bf in nsInputStreamPump::OnStateTransfer (this=0x7f44775f8580) at /home/jie/sources/thunderbird/src/mozilla/netwerk/base/src/nsInputStreamPump.cpp:508
#23 0x00007f44923d9013 in nsInputStreamPump::OnInputStreamReady (this=0x7f44775f8580, stream=0x7f4478545048) at /home/jie/sources/thunderbird/src/mozilla/netwerk/base/src/nsInputStreamPump.cpp:398
#24 0x00007f44a0d684f5 in nsInputStreamReadyEvent::Run (this=0x7f4477524dc0) at /home/jie/sources/thunderbird/src/mozilla/xpcom/io/nsStreamUtils.cpp:112
#25 0x00007f44a0d999a1 in nsThread::ProcessNextEvent (this=0x7f4498d2a1f0, mayWait=1, result=0x7fff4ef8232c) at /home/jie/sources/thunderbird/src/mozilla/xpcom/threads/nsThread.cpp:527
#26 0x00007f44a0d1dfcd in NS_ProcessNextEvent_P (thread=0x7f4498d2a1f0, mayWait=1) at nsThreadUtils.cpp:250
#27 0x00007f448ecafa77 in nsBaseAppShell::Run (this=0x7f4481e5e940) at /home/jie/sources/thunderbird/src/mozilla/widget/src/xpwidgets/nsBaseAppShell.cpp:177
#28 0x00007f448a7b51b1 in nsAppStartup::Run (this=0x7f4481ed4880) at /home/jie/sources/thunderbird/src/mozilla/toolkit/components/startup/src/nsAppStartup.cpp:182
#29 0x00007f44a1275a62 in XRE_main (argc=1, argv=0x7fff4ef82c88, aAppData=0x7f4498d21080) at /home/jie/sources/thunderbird/src/mozilla/toolkit/xre/nsAppRunner.cpp:3476
#30 0x0000000000401d4e in main (argc=1, argv=0x7fff4ef82c88) at /home/jie/sources/thunderbird/src/mail/app/nsMailApp.cpp:103

Comment 31

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #30)
> It comes from GNOME:
> $28 = 0x7f447ec1f8f8 "text/x-log"
> #5  0x00007f4484bcc081 in mime_file_type (filename=0x7f4477998ba0 "Job581.log",

Jie Zhang, check with ".xyz" etc.(unknown file extension), please.
Tb generates next for ".xyz" by "Send Later". Change attachment; in mail folder file to inline; by text editor, and restart Tb.
> --------------040301040002040104080302
> Content-Type: application/octet-stream; name="xxx.xyz"
> Content-Transfer-Encoding: base64
> Content-Disposition: attachment; filename="xxx.xyz"

Comment 32

[Jie Zhang]

Attachment: thunderbird-mime-bug538407.diff

Comment 33

[Jie Zhang]

Attachment: The test case also works for GNOME

Comment 34

[Jie Zhang]

Finally I reproduced it on my GNOME desktop with the newly attached test case. A proposed patch is also attached.

This bug is because I failed to notice that opts->file_type_fn () return empty string on unknown types.

Comment 35

[Jie Zhang]

(In reply to comment #31)
> (In reply to comment #30)
> > It comes from GNOME:
> > $28 = 0x7f447ec1f8f8 "text/x-log"
> > #5  0x00007f4484bcc081 in mime_file_type (filename=0x7f4477998ba0 "Job581.log",
> 
> Jie Zhang, check with ".xyz" etc.(unknown file extension), please.
> Tb generates next for ".xyz" by "Send Later". Change attachment; in mail folder
> file to inline; by text editor, and restart Tb.
> > --------------040301040002040104080302
> > Content-Type: application/octet-stream; name="xxx.xyz"
> > Content-Transfer-Encoding: base64
> > Content-Disposition: attachment; filename="xxx.xyz"

Thanks. I have fixed it now. Hope my patch can be quickly reviewed and committed.

Comment 36

[WADA:World Anti-bad-Duping Agency]

Thanks for quick solvong of this bug.

Some questions about quirks for Content-Type:application/octet-stream.

There are three kinds of quirks for Content-Type:application/octet-stream.
(i)  for known file extension, non-inline-displayable : .pdf, .doc, ...
     => invoke application based on file extension
        (already requested, and already fixed, probably)
(ii) for known file extension, inline-displayable, Content-Disposition:inline
     (ii-a) file extension known as text/xxx (your bug 532603)
            => display in inline, if View/Display Attachments Inline=On
     (ii-b) file extension known as other than text(image/jpeg, image/png, ...)
            => display in inline, if View/Display Attachments Inline=On
            (already requested?)

(Q1) I think quirks should be consistent among falled back mime-type for file extension. What do you think?
(Q2) charset is not set usually for part of Content-Type:application/octet-stream, Content-Disposition:inline;filename="xxx.log". If Content-Type:application/octet-stream instead of Content-Type:text/xxx is used for such text type file, I think charset of the text is not unpredictable. What do you think?

Comment 37

[Jie Zhang]

(Q1) Yes.

(Q2)
I just tested .jpg. It works.

Content-Type: image/jpeg; name="baby32.jpg"
Content-Disposition: attachment; filename="baby32.jpg"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_g4fhsss40

I also tested an email with attachment test.log, which contains Chinese words, and got charset removed. Thunderbird displays them in unreadable characters. Hopefully, the email client should set charset correctly. GMail does. If it does not, users can still click on the attachment and save it as a file and open it using an editor. For a better solution, thunderbird should be able to detect the charset as most editors do. But I feel not committed for this now.

Comment 38

[[e-mail]]

Comment on attachment 421608 [details] [diff] [review]
thunderbird-mime-bug538407.diff

>+      if (override_content_type
>+          && *override_content_type
>+          && (PL_strcasecmp(override_content_type, UNKNOWN_CONTENT_TYPE)))
Nit: we generally prefer to have the operator at the end of the previous line i.e.
if (override_content_type &&
    *override_content_type &&
    PL_strcasecmp(override_content_type, UNKNOWN_CONTENT_TYPE))

Comment 39

[Jie Zhang]

Attachment: New patch with coding style fix

Comment 40

[Jie Zhang]

Thanks. I have attached a new patch with the coding style change.

Comment 41

[Phil Lacy]

is this the same problem
STR
1. new text/plain email
2. drag zip file to attachments
3. save

4 check draft folder
  the zip file is now an inline base64 glob. with the email showing no attachments

5. in draft mail file. go to the newly created email and change application/octet-stream to application/pdf

6. open Tb and check again and it show ok as pdf file.

Comment 42

[Jie Zhang]

Phil,

Yes. I think it's the same issue.

Comment 43

[Vladimir]

Guys,

I hope this case is covered by the patch. But just in case, the following attachment shows in-line as MIME although there is no "Content-Disposition: inline;" in the message body. MIME Type ".MSG" is not defined in MimeTypes.rdf.

Content-Type: application/octet-stream;
	name="FSTAPE.MSG"
Content-Transfer-Encoding: base64
Content-Description: FSTAPE.MSG
Content-Disposition: attachment;
	filename="FSTAPE.MSG"

Tested on:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2pre) Gecko/20100118 Lanikai/3.1a1pre

-Vladimir

Comment 44

[Jie Zhang]

Vladimir,

Did you test with my patch or without my patch for this bug? If without, could you try it with my patch?

Jie

Comment 45

[Vladimir]

Jie,

How can I obtain the patched version? Unfortunately I do not have an environment set to compile the build.

I tested with the regular nightly build.

-Vladimir

Comment 46

[Jie Zhang]

Vladimir,

I just hope David :Bienvenu can get this patch a quick review and get it committed.

And I think this case should be covered by this patch, too.

Jie

Comment 47

[David :Bienvenu]

Thx for the patch, this fixes the bug, but I've been trying to figure out how we can add a unit test for this so that it doesn't regress again. We have two choices; we could add an xpcshell test that exercises the changed code, or would could add a mozmill test that loads the test message and verifies that the attachment area exists in the message pane and contains an entry for the attachment (or that the main document doesn't contain the attachment text).

I think the latter might be the better way to go, since we should be building up some tests for message display. Please see http://mxr.mozilla.org/comm-central/source/mail/test/mozmill/content-policy/test-video-content-policy.js for an existing test that does something similar...

Comment 48

[David :Bienvenu]

I should say, if you haven't been reading the Thunderbird developer newsgroup, we now require unit tests before a patch can land, if it makes sense to have a unit test for the bug, and I think it does in this case. 

Here's some info about writing and running mozmill tests - https://developer.mozilla.org/en/Thunderbird/Thunderbird_MozMill_Testing

If you'd rather do an xpcshell test, I think streaming the message with a quoting stream listener like the reply code does also exercises the patched code, and that could be done with an xpcshell test.

Comment 49

[Dan Mosedale (:dmosedale, :dmose)]

While this is an unfortunate bug, and one that should definitely block 3.1, I think we can live without it for our first alpha release.  Marking as blocking and targeting beta 1.  Assigning the bug to Jie as well, since they're doing the hard work (thanks Jie!).

Comment 50

[Jie Zhang]

I'll try to come up with a test for this bug in a few days.

And I set platform to All:All since it's not a x86:Windows specific bug.

Comment 51

[David :Bienvenu]

Thx, Jie, if you need any help, please let me know. I think cloning the content-policy/test-video-content-policy.js would be a good place to start...

Comment 52

[Vladimir]

Jie,

As you are working on fixing this bug here is another case.

What is interesting about this one it also fails on TB 3.0.1 production, Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1

It looks like TB fails to filter out some not viewable attachments when "View / Display Attachments Inline" is checked. "xlsx" is not on the Tools/Options/Attachments list.

Thank you,
Vladimir


Message pane contents:
======================

Attached is a spread sheet....

--List.xlsx --------------------------------------------------------------------
|                                                                              |
--------------------------------------------------------------------------------
PKßMRP©hZ.±âÄ3³;³Ž·e‘l ¢õ.ƒ´/pÚë–™xŸ½ôîE‚¤œQ…w‰ ®¯†³]SÀñ—…¥"~K”^©%ÈÛ~ÿNjïõ¨Â£á,Ôº äyËÛ’¹u"ylþ«¨2¡B(¬VÄBåÆ™$=¿XX
ÆëuÉÐ)†Ê`@e‘†h™1NˆC!GÃ7.:ZÉDEzU%3Èm!‰+€æ9H¹†N"j°›
åwB¤]Ø™ê¸ÞôfRsf–õÒ½Öc5èžÿ„½
<¯µ_JùdÌmÀ†vïÚ=ùðq5÷~uiWª4¦¥²n¯ûT9“èJ|gPM”Ó



Message body:
=============

Mime-Version: 1.0
X-Mailer: GroupWise 8.0
Subject: List
Date: Tue, 19 Jan 2010 14:37:12 -0600
Message-ID: <[e-mail]>
From: "X Y Z" <[e-mail]>
To: "V M" <[e-mail]>
Content-Type: multipart/mixed; boundary="____WCRAQBYZYWOYSIACEIHF____"


--____WCRAQBYZYWOYSIACEIHF____
Content-Type: multipart/alternative; boundary="____SRDVHGLOLYVOWLRNXGUW____"


--____SRDVHGLOLYVOWLRNXGUW____
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Attached is a spread sheet....


--____SRDVHGLOLYVOWLRNXGUW____
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<HTML><HEAD>
<META content=3D"text/html; charset=3Dutf-8" http-equiv=3DContent-Type>
<META name=3DGENERATOR content=3D"MSHTML 8.00.6001.18854"></HEAD>
<BODY style=3D"MARGIN: 4px 4px 1px; FONT: 10pt Tahoma">
<DIV>&nbsp;</DIV>
<DIV>Attached is a spread sheet....</DIV>

<DIV>&nbsp;</DIV>
<DIV>Mike</DIV></BODY></HTML>
--____SRDVHGLOLYVOWLRNXGUW____--

--____WCRAQBYZYWOYSIACEIHF____
Content-Type:
	 application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="List.xlsx"

UEsDBBQABgAIAAAAIQC1ensdkAEAAAEGAAATANIBW0NvbnRlbnRfVHlwZXNdLnhtbCCizgEooAAC
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

..............................................

ABMAAAAAAAAAAAAAAAAA3UYAAGRvY1Byb3BzL2N1c3RvbS54bWxQSwUGAAAAAA4ADgCpAwAAFUkA
AAAA
--____WCRAQBYZYWOYSIACEIHF____--

Comment 53

[rsx11m]

> Content-Type:
>      application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
> Content-Transfer-Encoding: base64
> Content-Disposition: attachment; filename="List.xlsx"

That's a different case though (not application/octet-stream), the line break immediately after the Content-Type heading may be an issue. Anyway, that would be for another bug.

Comment 54

[Vladimir]

Alright. I will try to file another bug report.

Comment 55

[David :Bienvenu]

Jie, any progress on a mozmill test for this?

Comment 56

[Jie Zhang]

I have been occupied by my works recently. So if anyone could write this test to get this patch in, I will appreciate. There will be a long holiday next week here. If no one pick this up, I will try to find time to do it in the next week.

Comment 57

[Matteo]

Just to confirm this bug on nightly Mozilla/5.0 (Windows; U; Windows NT 5.2; it; rv:1.9.3a2pre) Gecko/20100223 Shredder/3.2a1pre.

Mail headers (generated by Thunderbird itself):
---------------------------------------
Content-Type: application/octet-stream;
 name="trippi.dwg"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="trippi.dwg"
---------------------------------------

The message, saved as draft, doesn't display any attachment bar, but only attachment plain text (also with Content-Disposition: inline); 'view attachment inline' ON or OFF doesn't change anything.
If I select "modify as new", attachment is correctly here (quick workaround to save it).

Regards.

Comment 58

[Jie Zhang]

I'm writing test case for this bug. I have written half of the test. It can be loaded by Mozmill and run. I see from the screen that the test case can reproduce this bug. But I don't know how to write Mozmill code to verify that the attachment area exists in the message pane and contains an entry for the attachment.

if (mozmill.getMail3PaneController().window.content.document.getElementById('attachmentView').collapsed)
    throw new Error(test.type + " has no attachment");

Does not work.

Comment 59

[Blake Winton (:bwinton) (:☕️)]

I think you can just check 
mc.eid("attachmentView").node.collapsed

What do you get if you print out:
mc
mc.eid("attachmentView")
mc.eid("attachmentView").node
mc.eid("attachmentView").node.collapsed
?

Thanks,
Blake.

Comment 60

[Jie Zhang]

:bwinton, thanks! mc.eid("attachmentView").node.collapsed works! I now got a test case for this bug. The new diff is attached. It contains the unchanged patch and the new test case.

Comment 61

[Jie Zhang]

Attachment: The patch with the test case.

Comment 62

[[e-mail]]

Comment on attachment 431333 [details] [diff] [review]
The patch with the test case.

(IIRC tests don't need sr, so you could have carried over my original sr.)

Comment 63

[David :Bienvenu]

Comment on attachment 431333 [details] [diff] [review]
The patch with the test case.

thx very much, that works. I'll land this shortly.

Comment 64

[David :Bienvenu]

fix pushed, thx very much Jie!

Comment 65

[Jie Zhang]

bienvenu, thanks for help me install it.

Comment 67

[Vladimir]

Everything works fine for several days by now.

Thank you Jie.

-Vladimir