Closed Bug 359148 Opened 13 years ago Closed 13 years ago

When saving a file, "unicode" characters in filename are lost (turned into '_')

Categories

(Core Graveyard :: File Handling, defect)

x86
Windows XP
defect
Not set

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: lapsap7+mz, Assigned: jshin1987)

References

Details

(Keywords: intl, verified1.8.1.1)

Attachments

(1 file)

I was trying the Unicode support of Firefox 2.0 and found that downloading a file and saving it with Unicode name will end up having '_' (underscore) replacing every non system locale character.

Actually, I was downloading attached file from my GMail account and the filename is in Chinese (whereas my system locale is in Western European).  At the end of the download, the filename has a lot of '_'.

I later found out how to reproduce this in other cases.  I hope the following steps are equivalent to my problem in GMail:

1. Download any file. For example, Firefox 2.0 setup file.

2. When asked where to save, change its name to include Unicode characters.  I suggest using mathematical symbols because they're not in any legacy/traditional encoding, and I'm pretty sure they're all Unicode no matter what system locale you're using and thus the test is valid for everybody.  My suggestion is ∂∇∰.exe (partial derivative U+2202, nabla U+2207 and volume integral U+2230).

3. Since the setup file is more than 5 MB, it would take a while to finish.  You could see in the target folder that there're two files: ∂∇∰.exe and ___.exe.part

4. Once the download is done, it will only remain one file: ___.exe

It seems like FF2 renames ___.exe.part to ___.exe while discarding ∂∇∰.exe
You have to switch the encoding of this page to UTF8 in order to see what those three Unicode characters that I suggested.
I just like to confirm something that doesn't really need to be confirmed:
the same problem happens in Win2000 Pro and Win Server 2003
"Save As" and "Save Link As" work fine. Downloading to the default location also works fine. The problem only shows up when firefox is configured to prompt for  the download location everytime you download a file. 

It's a bug in 'download manager'. 


Component: File Handling → Download Manager
Keywords: intl
Summary: Saving filename with Unicode characters ends up with '_' replacing every non system locale character → after prompting for download folder, "unicode" characters in filename are lost (turned into '_')
It turned out that uriloader uses 'native' methods where it should use unicode methods of nsIFile. 

This one is easy enough for me to fix without much effort. perhaps, this will be the last bug I'll fix this year :-)
Assignee: nobody → jshin1987
Component: Download Manager → File Handling
Product: Firefox → Core
Version: 2.0 Branch → Trunk
Attached patch patch Splinter Review
this is a simple patch. This part is not perf-sensitive so that we can just use Unicode methods (rather than adding platform-dependent #ifdef).
No, no, even if Firefox isn't configured to prompt for download location, it doesn't work.

Actually, the fact is that it's hard to find Unicode-named file from web server, that's why I suggested changing the name.  I mostly get them as mail attachments. That's why I'm going to send one to you in mail as a test.

OTOH, I had also tested to save an image from any web page (right-click and "Save image as") and changed its name to include Unicode and this has no problem.

As to "Save Link As": yes, this works.  But sometimes it's not possible to right-click and choose "Save Link As".  For example, during server push, the very technique used by mozilla.org to send the setup file to user:
http://www.mozilla.com/en-US/products/download.html?product=firefox-2.0&os=win&lang=en-US
QA Contact: file.handling → file-handling
(In reply to comment #6)
> No, no, even if Firefox isn't configured to prompt for download location, it
> doesn't work.

Thanks for bringing up this issue. The patch I made fix this case as well. 

I couldn't test your test case sent via gmail because my debug build asserts too many times when I connect to gmail, but I made my own test case (with Devanagari characters) and confirmed that the patch works.
Status: NEW → ASSIGNED
Summary: after prompting for download folder, "unicode" characters in filename are lost (turned into '_') → When saving a file, "unicode" characters in filename are lost (turned into '_')
Comment on attachment 244565 [details] [diff] [review]
patch 

asking for r/sr.

I tested this in 1.8 branch, but it should also work on trunk.
Attachment #244565 - Flags: superreview?(darin.moz)
Attachment #244565 - Flags: review?(cbiesinger)
Attachment #244565 - Flags: superreview?(darin.moz) → superreview+
Comment on attachment 244565 [details] [diff] [review]
patch 

thanks!
Attachment #244565 - Flags: review?(cbiesinger) → review+
thanks for r/sr. landed in trunk. Teng-Fong, can you verify it with a *trunk* build (maybe you have to wait a while before trying it to be sure that it's applied). 
1.8 branch will be fixed after baking this patch in the trunk for some time.

Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
(In reply to comment #10)
> thanks for r/sr. landed in trunk. Teng-Fong, can you verify it with a *trunk*
> build (maybe you have to wait a while before trying it to be sure that it's
> applied). 
> 1.8 branch will be fixed after baking this patch in the trunk for some time.
> 

Should a blocker flag get thrown for this?
What is a trunk build?  Is that a nightly build?  I don't know why, but I can't find Firefox nightly build anymore.  Where are they?  I was able to find them before ... or had I mixed up with Sunbird?
Oh I think I've just found it.  I had to go up up up in the ftp server and down down down the other branch:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/

But I have the choice between 2006-11-04-05-mozilla1.8/firefox-2.0.en-US.win32.zip  and 2006-11-04-08-mozilla1.8.0/firefox-1.5.0.8.en-US.win32.zip

I suppose I have to get the 2006-11-04-05-mozilla1.8/firefox-2.0.en-US.win32.zip right?
I've tried the "Gecko/20061105 BonEcho/2.0" (BonEcho??) but the bug is not yet fixed.  I'll download the trunk next week and try again.
(In reply to comment #14)
> I've tried the "Gecko/20061105 BonEcho/2.0" (BonEcho??) but the bug is not yet

I have NOT landed the patch for 1.8 branch (Firefox 2.0). The bug is ONLY fixed in trunk (which will eventually become FF 3.0). See comment #10. 

What you need to download is in 
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/

Oh sorry, I was quite confused by your jargon and your comment #8.  And I have to admit I'd never have thought that your trunk (means the mean build?) is a FF3.0 instead of FF2.0.

Anyway, I've tested the FF3 build from that directory in a XP Pro, and I can confirm that the bug is corrected.

> 1.8 branch will be fixed after baking this patch in the trunk for some time.

How is this sentence to be understood? 1.8 branch implies the FF2.0 branch?  That means this fix will eventually be applied to FF2 as well and will appear in the next, probably, FF2.0.1?
(In reply to comment #11)

> > 1.8 branch will be fixed after baking this patch in the trunk for some time.
> > 
> 
> Should a blocker flag get thrown for this?

Perhaps not a blocker, but it needs to be flagged as 'wanted 1.8.1.x'. I'm not privileged enough to request it, though. ( only drivers can do it). 

mconnor, will you do it? 


(In reply to comment #16)
> Oh sorry, I was quite confused by your jargon and your comment #8.  

  I admit comment #8 could have been confusing. I meant that I had applied the patch to the *local* 1.8 branch source tree and tested it with *my own* 1.8 build. I can't just land the patch without testing and r/sr and approval. 
 
> Anyway, I've tested the FF3 build from that directory in a XP Pro, and I can
> confirm that the bug is corrected.

Thanks for testing it.
 
> > 1.8 branch will be fixed after baking this patch in the trunk for some time.
> 
> How is this sentence to be understood? 1.8 branch implies the FF2.0 branch? 
> That means this fix will eventually be applied to FF2 as well and will appear
> in the next, probably, FF2.0.1?

Yes, that's what I want to do unless an unexpected regression is reported for our trunk nightly builds. 
I think we want this at some point, definitely.
Flags: wanted1.8.1.x+
Comment on attachment 244565 [details] [diff] [review]
patch 

Asking for a1.8.1.1.

it's been about 10 days and no regression has been reported for trunk builds. 
This patch is rather safe and flagged as 'wanted 1.8.1.x'.
Attachment #244565 - Flags: approval1.8.1.1?
Comment on attachment 244565 [details] [diff] [review]
patch 

approved for 1.8 branch, a=dveditz for drivers
Attachment #244565 - Flags: approval1.8.1.1? → approval1.8.1.1+
Keywords: fixed1.8.1
sorry for bug spam. the patch was just landed in 1.8 branch (for inclusion in FF 2.0.1 when it's released)

Seak, Teng-Fong:  Can you please also help us verify this fix on the 1.8 branch?  You can test with http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2006-12-01-03-mozilla1.8/ builds.  If everything works for you, please replace "fixed1.8.1.1" keyword with "verified1.8.1.1" so we can be sure it will work in the next Firefox 2.0.0.1 release (coming soon).  Thanks!
OK, everything is OK.  I've tested
1) saving the link as and changing name: OK;
2) saving a Unicode-named file to default location (ie Desktop): OK;
3) saving the same file but to a specified location: OK
thanks for testing. 
Status: RESOLVED → VERIFIED
I always thought that in order to close a bug, it needs to be RESOLVED, but not just VERIFIED, or is it not?
I'm quite lost by these terms. 
bugs are first RESOLVED, then the new builds are tested and the bug is marked VERIFIED (or REOPENED if the bug isn't actually fixed yet)
(In reply to comment #26)
> bugs are first RESOLVED, then the new builds are tested and the bug is marked
> VERIFIED (or REOPENED if the bug isn't actually fixed yet)

OK, I see.  I've seen so many bugs which are simply marked as RESOLVED and they're indeed solved in official releases.  But no more steps are taken to mark the real end......
Flags: wanted1.8.1.x+
Duplicate of this bug: 278530
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.