243504 - filename of attachment is truncated at the first # (numbersign) or ; (semicolon)

Status: RESOLVED FIXED

(Thunderbird :: Message Compose Window, defect)

Description

[Klaus Nuber]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Gecko, Opera; .NET CLR 1.0.3705)
Build Identifier: 

The filename of an attachment is truncated at the first # (numbersign). This 
error seems to be introduced in Version 0.6



Reproducible: Always
Steps to Reproduce:
1.
2.
3.

Comment 1

[OstGote!]

Could this be a dupe of Bug 239192 (or Bug 245935)?

Comment 2

[Remo Del Bello]

*** Bug 246270 has been marked as a duplicate of this bug. ***

Comment 3

[OstGote!]

@REPORTER: 
Klaus, auch wenn Du es gerne möchtest, es ist nicht 'Dein' Zeichen. ;-) Könntest
Du bitte die summary korrigieren? Danke. "... first # (nuMbersign)"
 
Please change the typo in the summary.

Comment 4

[Fabian Rodriguez:MagicFab]

@OWNER: Please change severity to "Major" since there is no known workaround and
the truncating character is part of the ASCII character set. Also please change
 the component to Message Compose Window, as this is where it occurs. Also
change OS to "All".

Platform: PC
Operating system: Mandrake Linux 10, Windows 2000 pro & Win98
Thunderbird build: version 0.7+ (20040727)
Firefox build: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7) Gecko/20040626
Firefox/0.9.1

Reproductible: Always

I don't have permission yet to mark this as confirmed.

Steps to reproduce:

1. Create a text file with or without content, named "text #44.txt"
2. Create a new message window in Thunderbird, using either CTRL-M or the Write
button
3. Using the ATTACH button or dragging the file into the message composition
window, attach the "text #44.txt" file
4. Repeat the same test but using ".xls" as extension
5. When the recipient gets the file, tooltips and Save as... operations confirm
that the file name has been truncated. Message headers indicate this for the
.xls file:

Content-Type: application/msexcell;
 name="text "
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="text "

ACTUAL RESULTS:
The filename is truncated even before it's been sent or further processed. It's
still possible to send the file and its file type is recognized and stored. When
the recipient gets it, TB correctly detects its filetype but the filename part
starting at the "#" symbol is missing, including the extension.

EXPECTED RESULTS:
The filename should be preserved in its entirety.

ADDITIONAL BUILDS & PLATFORMS:
Also reproduced in Windows 2000 Pro and Mandrake Linux 10 using latest TB
release (not nightly). 

Comment 5

[Jo Hermans]

confirmed in bug 243504 (semicolon in filename)
(changing summary of this bug)

Comment 6

[Jo Hermans]

*** Bug 256569 has been marked as a duplicate of this bug. ***

Comment 7

[Patrick]

Couldn't this be used to trick a user into opening an attachment
"innocent.txt;DESTROY_PC.exe" 

It's not really a security risk in itself but more a social engineering thing.
But maybe the security flag should be set for this one? Hmpf. where did that
flag go? 

Comment 8

[Daniel Veditz [:dveditz]]

The security flag is only available at bug creation time (or to members of the
security group). There is not much point hiding this one, the flaw is obvious
enough to have garnered dupes. We wouldn't be protecting anyone much, and that's
the only argument to override our strong preference for openness.

Mozilla suite seems to have this problem as well. I guess that's covered by bug
239192.

Comment 9

[Daniel Veditz [:dveditz]]

Note that this truncation happens when you *attach* the file. If something
malicious could come out of that an attacker could simply lie about the name
and/or MIME type in the first place (and they do). If you are sent an attachment
with those chars in the name, they show up in the attachment pane.

Comment 10

[chris hofmann]

out of time for tbird 0.8. try for final

Comment 11

[Mike Cowperthwaite]

*** Bug 240608 has been marked as a duplicate of this bug. ***

Comment 12

[Mike Cowperthwaite]

*** Bug 254251 has been marked as a duplicate of this bug. ***

Comment 13

[Mike Cowperthwaite]

*** Bug 251169 has been marked as a duplicate of this bug. ***

Comment 14

[Mike Cowperthwaite]

*** Bug 245935 has been marked as a duplicate of this bug. ***

Comment 15

[Scott MacGregor]

plussing for now

Comment 16

[Scott MacGregor]

cc'ing jshin for some I18N input.

The problem seems to be with what the UTF-8 converter service is doing to our
escaped URL in nsMsgCompose::AttachmentPrettyName.

Our URL is initially escaped:

scheme: "file:///C:/temp/sig%23.txt"

that's good.

Then we run ConvertURISpecToUTF8 which ends up giving us a url that is unescaped:

UTFI converted file spec: "file:///C:/temp/sig#.txt"

We then turn that string into a URI and ask for the leaf name:

rv = url->GetFileName(leafName); // leafName is in UTF-8 (escaped).

But the leaf name is no longer escaped and necko gives us "sig" as the file name.

jshin, does that look like a bug in the utf-8 converter that it took our escaped
URL and unescaped it? 

Comment 17

[Scott MacGregor]

Attachment: possible fix

This patch re-escapes the utf8 convertered URL before we try to get the leaf
name from it. This fixes the problem for the ASCII case, I don't know what the
impact is for the non ascii case when we re-escape the utf8 string. I'm also
not sure if my escape flags are correct...:)

I can now attach files with # signs in them with this patch and they look right
in the UI.

Comment 18

[Scott MacGregor]

Oops this comment went in the wrong bug:

http://lxr.mozilla.org/seamonkey/source/intl/uconv/src/nsUTF8ConverterService.cpp#118

shows that the utf8 converter is indeed returning an unescaped url even though
we passed in an escaped one.

Comment 19

[Scott MacGregor]

Comment on attachment 160655 [details] [diff] [review]
possible fix

jshin, what do you think of a patch like this? Will re-escaping the URL after
it has gone through the utf8 conveter cause i18n issues?

In particular read:

https://bugzilla.mozilla.org/show_bug.cgi?id=243504#c16

Comment 20

[Jungshik Shin]

Comment on attachment 160655 [details] [diff] [review]
possible fix


Scott, sorry that I came here late (again, bugzilla is doing something strange
to me. I haven't received any bug mail from this bug). 


>+    // re-escape the url since the utf8 converter unescaped us
>+    NS_EscapeURL(unescapedUtf8Scheme.get(), unescapedUtf8Scheme.Length(),
>+                 esc_SkipControl | esc_AlwaysCopy | esc_FilePath, utf8Scheme);

You don't need to use esc_SkipControl because it's only meaningful for
unescaping, but you have to specify 'esc_OnlyASCII' because we don't want to
escape non-ASCII characters in 'PrettyAttachmentName'. 

Having said that, I found it strange that 'ASCII characters' like '#' and ';'
are unescaped in the first place because I used 'esc_OnlyNonASCII' at

http://lxr.mozilla.org/seamonkey/source/intl/uconv/src/nsUTF8ConverterService.c
pp#134

Hmm.. it should work because NS_UnescapeURL seems to do the right thing :

http://lxr.mozilla.org/seamonkey/source/xpcom/io/nsEscape.cpp#468

Comment 21

[Scott MacGregor]

Attachment: improved fix based on jshin's comments

Comment 22

[Scott MacGregor]

Comment on attachment 161048 [details] [diff] [review]
improved fix based on jshin's comments

jshin, this is the same patch but it uses:

esc_OnlyASCII

thanks again for the review!

Comment 23

[Mike Cowperthwaite]

*** Bug 263300 has been marked as a duplicate of this bug. ***

Comment 24

[Jungshik Shin]

Comment on attachment 161048 [details] [diff] [review]
improved fix based on jshin's comments

Soott, as you saw, I fixed it upstream. I was wrong to say that NS_UnescapeURL
does the right thing. It doesn't and I had to fix it.

Comment 25

[Scott MacGregor]

jshin part of this original patch is still necessary right? We still need to
escape the string that comes back from the utf-8 service since it unescaped it.
Or are you saying the utf8 service now won't unescsape ascii characters like #
sign when it is doing it's thing?

Comment 26

[chris hofmann]

it would be good if we could figure this out and get it checked in soon.

Comment 27

[Scott MacGregor]

this has been fixed on the branch and trunk by jshin's bug fix.

Comment 28

[Scott MacGregor]

*** Bug 267394 has been marked as a duplicate of this bug. ***

Comment 29

[Scott MacGregor]

removing the fixed aviary 1.0. This fix had to be backed out as it broke saving
of non ascii file names.

Comment 30

[[e-mail]]

Shouldn't this bug be opened back up if the fix was removed in 1.0?


I know I had this problem with TB 1.0 on WinXP SP1:

I was composing a message with the filenames HW #3.pdf and HW #4.pdf.  When I
attached the files, only HW showed up, twice.

So I closed the email compose window, renamed the files to HW_3.pdf and HW_4.pdf
and reattached the files to a new message.  The attached files still had the
name HW and HW.

Only thing I didn't try was to close TB down completely and try again.

Comment 31

[[e-mail]]

I just found the the reason it was still attaching HW was because the folder
they were under was named "HW #3" and "HW #4".  So bug 239192 is back again.

Comment 32

[WADA:World Anti-bad-Duping Agency]

Change 239192 from "depends on" to "blocks" of this bug because there is no
patch for 239192.

Comment 33

[Graham Freeman]

Confirmed that the bug is back in Thunderbird v1.0 (20041206) on MacOS X v10.3.6.

Comment 34

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #31 and comment #33)
See Bug 263462 Comment #11 for current status.

Comment 35

[WADA:World Anti-bad-Duping Agency]

Reopen according to comment #30 & comment #31 & comment #33.
This is because patch of prerequisite Bug 263462 was backed out on Thunderbird
1.0 branch.

Comment 36

[dopfer]

Hi, 

I am new to both Bugzilla and this thread, but I have some additional
information about the scope of this bug for you.  

It is present in Netscape 7.2 on Win 2000 as well as Thunderbird 1.0 and Mozilla
1.7.3 on WinXP.  I upgraded a work machine from Netscape 7.1 to 7.2 and this
apparently re-introduced the bug to Netscape, as we had been sending # in
character names before.  (I would post this on the Netscape bug forum, but I
don't know where it is.)  I then tested for it on my home system and found the
TB and Mozilla bugs, which I am sure are related.

Is there any way to patch this on the installed version of the software rather
than at the source?  I am sure that most English speakers (and really anyone who
uses a standard ASCII character set) would rather have this bug fixed than to
have the ability to save to non-ASCII file names.  Perhaps this fix can be
implemented in an language (or character set) specific way?

Thanks, good luck, and good work.

Comment 37

[sairuh (rarely reading bugmail)]

nominating for tbird1.1.

Comment 38

[Scott MacGregor]

sarah jshin fixed this for 1.8a6 (Bug #274264)

Comment 39

[sairuh (rarely reading bugmail)]

thanks for clarifying, scott!

Comment 40

[WADA:World Anti-bad-Duping Agency]

Please be careful because this bug is not resolved on 1.7 branch yet.
See Bug 263462 Comment #17.
(Waiting for approval of re-landing of patch for Bug 263462 on 1.7 branch.) 

Comment 41

[Scott MacGregor]

*** Bug 279008 has been marked as a duplicate of this bug. ***

Comment 42

[Scott MacGregor]

*** Bug 279010 has been marked as a duplicate of this bug. ***

Comment 43

[Hiro]

*** Bug 281460 has been marked as a duplicate of this bug. ***

Comment 44

[Frank Wein [:mcsmurf]]

*** Bug 281959 has been marked as a duplicate of this bug. ***

Comment 45

[Mike Cowperthwaite]

*** Bug 284812 has been marked as a duplicate of this bug. ***

Comment 46

[Roberto Salomon]

I've been facing this problem for a while now. It is still present on
Thunderbird 1.0.2. I have developed a series of scripts that send files to
several translators in the pt-BR version of OpenOffice.org. At one point, the
script calls Thunderbird's mozilla-xremote-client, as in the following example:

<path to thunderbird>/mozilla-xremote-client -a thunderbird
"xfeDoCommand(composeMessage,to=[e-mail],subject=Another,attachment=file:///<path>/test#file#1)"

This results in a new message window being shown with the fields to and subject
correctly filled but with the attachment pane only showing test. When attempting
to send the mail, Thunderbird issues the following message:

Sending of message failed.
There was an error attaching test. Please check if you have access to the file.

Manually deleting the attachment from the pane and using the attachment button
or the File-Attach-File(s) menu results in the file still being displayed as
test but the message can be sent.

When placing the mouse pointer over the attachment name on the first case (sent
via script), the hover shows the file name as file:///<path>/test#file#1.
However, when adding the attachment through the GUI, the hover shows the file
name as file:///<path>/test%23file%231.

It seems that two different code paths are being used when attaching a file from
xmozilla-remote-client and the gui.

Hope this description helps.

Comment 47

[Roberto Salomon]

I forgot two things on my previous post:

1. I'm using Thunderbird 1.0.2 under SuSE Linux 9.2

2. If I change my script to replace the # character with %23, the interface
still displays the file name as test#file#1 and still can't send it.

Comment 48

[Roberto Salomon]

The latest trunk builds apparently solve the UI problem when handling file names
with the pound (#) sign in them. However, as I pointed out in my previous posts,
when using a script, the file name is still truncated at the first # character
and Thunderbird complains that it can't find the file to attach.

I'm not really sure if this problem should be handled in this bug or a new one
should be opened.

Comment 49

[Elmar Ludwig]

*** Bug 299922 has been marked as a duplicate of this bug. ***

Comment 50

[Mike Cowperthwaite]

*** Bug 300063 has been marked as a duplicate of this bug. ***

Comment 51

[Adam Guthrie]

*** Bug 310592 has been marked as a duplicate of this bug. ***

Comment 52

[OstGote!]

*** Bug 312525 has been marked as a duplicate of this bug. ***

Comment 53

[Jungshik Shin]

Roberto, can you test the latest trunk build in several hours (not now because
the patch for bug 274264 got just landed) and see whether your command line
problem is still there? 

Comment 54

[Roberto Salomon]

I'll test the latest trunk as soon as it's available. Currently the latest trunk
build is dated October 16th. I'll monitor the ftp and download the latest trunk
as soon as it's available.

Comment 55

[Roberto Salomon]

Tested on latest trunk (version 1.6a1 (20051017)). Via menus everything works
fine. However I have to wait for Bug 308196 so I can test the fix through the
command line.

Comment 56

[Mike Cowperthwaite]

*** Bug 315419 has been marked as a duplicate of this bug. ***

Comment 57

[Roberto Salomon]

Tested on 1.6a1 (build 20051116) and the problem is still there when composing from the command line via xfeDoCommand or when dragging and dropping a file in the attachment area.

Comment 58

[jsheinz]

How do I fix this problem? It says resolved and I click the link, but I have absolutely no idea what to do with all that code. Can someone please help me. 

I use Tbird at work and I send PDF's with Work Order numbers in them all the time example: WO#12345.pdf and all I get is WO in the attachement.

Comment 59

[Jesse Molina]

(In reply to comment #58)
> How do I fix this problem? It says resolved and I click the link, but I have
> absolutely no idea what to do with all that code. Can someone please help me. 
> 
> I use Tbird at work and I send PDF's with Work Order numbers in them all the
> time example: WO#12345.pdf and all I get is WO in the attachement.
> 

You need to download and install a version that has this bug fixed.  I don't know if this has been fixed it any of the Thunderbird releases yet or not.  Can someone else please comment on Thunderbird?

You can try Seamonkey though -- it's definitely fixed there.  Try the Seamonkey 1.0 beta, or get the nightly build here;  http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-trunk/seamonkey-1.5a.en-US.win32.installer.exe  

I highly recomend a nightly build over the beta.

Comment 60

[Mike Cowperthwaite]

(In reply to comment #59)
> I don't know if this has been fixed it any of the Thunderbird releases yet
> or not.  Can someone else please comment on Thunderbird?

The TB 1.5rc1 and rc2 builds have the original patch from this fix, and both fixes from bug 274264.  As far as I know, the problem from comment 57 (Roberto Salomon) is still unresolved.

Comment 61

[jsheinz]

I installed 1.5 RC2 and it fixed the problem. Now to wait for January and this official release, which if things go the same as FF will be a bit for bit copy of the final RC. Thanks for your help. 

Comment 62

[Mike Cowperthwaite]

*** Bug 338425 has been marked as a duplicate of this bug. ***

Comment 63

[Mike Cowperthwaite]

*** Bug 343211 has been marked as a duplicate of this bug. ***

Comment 65

[Thomas]

I can clearly reproduce this in following version:

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1

Path of files with space are truncated. The associated application can not find the file.
Saving with special characters is not problem at all.

Comment 66

[Thomas]

I see there is something related alread: https://bugzilla.mozilla.org/show_bug.cgi?id=1601905