upgrade Mac CFM file system code to use Unicode when HFS+ exists

RESOLVED WONTFIX

Status

()

Core
Internationalization
RESOLVED WONTFIX
16 years ago
15 years ago

People

(Reporter: Frank Tang, Assigned: Conrad Carlen (not reading bugmail))

Tracking

({intl, qawanted})

Trunk
mozilla1.3beta
PowerPC
Mac OS X
intl, qawanted
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

16 years ago
read 
http://gemma.apple.com/techpubs/macosx/Carbon/Files/FileManager/File_Manager/
Functions/FunctionGroups.html

It looks like at least the following APIs support Unciode/CFString
Creating a File System Reference (FSRef)
 FSMakeFSRefUnicode     	Constructs an FSRef for a file or directory, given a 
parent directory and a Unicode name. 
 PBMakeFSRefUnicodeSync	 	Constructs an FSRef for a file or directory, 
given a parent directory and a Unicode name. 
 PBMakeFSRefUnicodeAsync	Constructs an FSRef for a file or directory, given a 
parent directory and a Unicode name. 

Creating Directories
 FSCreateDirectoryUnicode	Creates a new directory (folder) with a Unicode name. 
 PBCreateDirectoryUnicodeSync	Creates a new directory (folder) with a Unicode 
name. 
 PBCreateDirectoryUnicodeAsync	  Creates a new directory (folder) with a Unicode 
name. 

Creating Files
 FSCreateFileUnicode	Creates a new file with a Unicode name. 
 PBCreateFileUnicodeSync	Creates a new file with a Unicode name. 
 PBCreateFileUnicodeAsync	Creates a new file with a Unicode name. 

Determining the Unicode Names of the Data and Resource Forks
 FSGetDataForkName	Returns a Unicode string constant for the name of the data 
fork.
 FSGetResourceForkName	Returns a Unicode string constant for the name of the 
resource for

 Moving and Renaming Files or Directories
 FSRenameUnicode	Renames a file or folder.
 PBRenameUnicodeSync	Renames a file or folder.
 PBRenameUnicodeAsync	Renames a file or folder.
(Reporter)

Updated

16 years ago
Status: NEW → ASSIGNED
(Reporter)

Comment 1

16 years ago
see also 95478 about file picker
(Reporter)

Comment 2

16 years ago
All the Unicode related funciont is using FSRef instead of FSSpec. This could be 
hard since we have FSSpec in mnay many places-
including plugin, nspr, liveconnect, java, mrj, xpcom/io, etc.

Comment 3

16 years ago
It's also hard because an FSRef must point to an existing file system entity; you 
can't have one point to a file that is yet to be created, like you can with an 
FSSpec.
(Assignee)

Comment 4

16 years ago
If you look at the Availability info for these Unicode routines, they are
available in InterfaceLib 9.0 and later. In other words, this does not require
OS X an should be done for any system on which the Gestalt selector
gestaltFSAttr returns a response with the gestaltHasHFSPlusAPIs bit set.

Can I have this bug?
(Reporter)

Comment 5

16 years ago
sfraser: about your comment 
"FSRef must point to an existing file system entity".

see http://developer.apple.com/technotes/tn/tn2022.html
TN2022 The Death of typeFSSpec: moving along to typeFileURL

"In a nutshell, typeFileURL is a core-foundation URL en"coded to a stream of
byte s in UTF8 format. This is the suggested data type to use when your
application would like to create a reference to 
a file that has yet to be created. Furthermore, there are a number of other good
reasons to use this type; depending on your processing requirements, you may
wish to use this data type in a number of different circumstances. Here are some
properties and features of the typeFileURL data format:

    * typeAlias and typeFSRef require deterministic reference and cannot refer
to files that have yet to be created. typeFileURL uses the weak "by name" style
reference provided by URLs, and as such is quite capable of providing
pre-determined references to files.

"


wtc- please read
http://gemma.apple.com/techpubs/macosx/SystemOverview/SystemOverview/FileSystem/File_Encodings_and_Fonts.html

"In addition, all code that calls BSD system routines should ensure that the
const *char parameters of these routines are in UTF-8 encoding. All BSD system
functions expect their string parameters to be in UTF-8 encoding and nothing else.
"

and "QA1028 ICLaunchURL, "file:///" URLs and Mac OS X"
http://developer.apple.com/qa/qa2001/qa1028.html
"    * Text Encodings -- In Mac OS 9 the URL components are assumed to be
percent-encoded PStrings, as returned by PBGetCatInfo. This is inadequate for
Mac OS X, where the File Manager fully supports Unicode file names. In Mac OS X
the URL components are defined as percent-encoded UTF-8.
...
One possible workaround is to change your code to generate the "file:///" URL in
the right format. You can do this with a sequence of File Manager, CFURL, and
CFString calls.

An easier workaround is to avoid ICLaunchURL entirely and change your code to
call the Launch Services routine LSOpenCFURLRef. This code is very simple.

   1. If you're starting with an FSRef, just skip to step 2. If you're starting
with an FSSpec, convert it to an FSRef using FSpMakeFSRef.
   2. Create a CFURL based on the FSRef using CFURLCreateFromFSRef.
   3. Pass that CFURL to LSOpenCFURLRef.
   4. Release your reference to the CFURL.
"
(Reporter)

Comment 6

16 years ago
see 95560 about return "UTF-8" as filename charset in nsIPlatformCharset on
MacOS X. This will make the current file: url handling code assume "UTF-8" as
the file name encoding. However, this may introduce some problem with some Mac
file operation. 
(Reporter)

Comment 7

16 years ago
reassign to ccarlen since he ask for it.
Assignee: ftang → ccarlen
Status: ASSIGNED → NEW
(Reporter)

Updated

16 years ago
Blocks: 103669
(Assignee)

Updated

16 years ago
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla0.9.7
(Assignee)

Comment 8

16 years ago
taking
(Assignee)

Comment 9

16 years ago
*** Bug 87831 has been marked as a duplicate of this bug. ***

Comment 10

16 years ago
Add myself to CC for reviewing bug 87831 later.

Updated

16 years ago
Blocks: 108000
(Assignee)

Comment 11

16 years ago
-> 0.9.8
Target Milestone: mozilla0.9.7 → mozilla0.9.8

Comment 12

16 years ago
I am attending Apple's Carbon kiitchen.
There are clients of nsLocalFile which use methods with FSSpec. How about adding methods with FSRef, so 
clients do not have to use FSSpec? There are conversion API which convert between FSSpec and FSRef. I 
added methods which takes FSRef and just convert to FSSpec which is used for nsLocaleFileMac internal 
implementation.
Conrad, what do you think?

Comment 13

16 years ago
Created attachment 59874 [details] [diff] [review]
Adding methods with FSRef.

Comment 14

16 years ago
Note that an FSRef cannot refer to a file that does not yet exist, whereas an 
FSSpec can.

Comment 15

16 years ago
Simon, so is nsLocalFile used for that situtation, I mean for a file does not
exist? How about CFURL, can that be used for non existing files?
I think we cannot stick with FSSpec (e.g., the user can assign non default
language file names in Finder in OSX).
Keywords: intl
(Assignee)

Comment 16

16 years ago
> so is nsLocalFile used for that situtation, I mean for a file does not
exist?

Yes. Frequently.

> How about CFURL, can that be used for non existing files?

That may be the way to go. Another is to use an FSRef + appended unicode path.
Some portion of a file path (at least the volume) must exist. A non-existent
file path can be specified with the combination of those 2.

It will either be done with CFURL or FSRef + unicode appended path, depending on
which gives better performance. But, either way, FSSpec will be going away.

Comment 17

16 years ago
Sorry, I still don't get it about the file not exist situation. Could you give
me examples? If that happens frequently, why not Apple supports it in FSRef?


(Assignee)

Comment 18

16 years ago
See comment #5.

In mozilla, it happens all over the place. Typical scenario is:
(1) get a directory from directory service
(2) append a node or two (neither of them exist)
(3) if it doesn't exist, call create

After (2), the nsIFile specifies the file system object even though it doesn't
exist.
(Assignee)

Comment 19

16 years ago
*** Bug 106271 has been marked as a duplicate of this bug. ***
(Assignee)

Updated

16 years ago
Blocks: 56295
(Reporter)

Comment 20

16 years ago
*** Bug 104935 has been marked as a duplicate of this bug. ***
(Assignee)

Comment 21

16 years ago
Mass move to 0.9.9
Target Milestone: mozilla0.9.8 → mozilla0.9.9

Comment 22

16 years ago
Conrad, any decision about CFURL or FSRef which one to use for nsLocalFileMac?
(Assignee)

Comment 23

16 years ago
  There should be an InitWithCFURL() on nsILocalFileMac (maybe nsILocalFileMacX
which would be a subclass). This seems useful for specifying files which may or
may not exist.
  Internally, the implementation will probably use an FSRef + mAppendedPath just
as the current impl uses FSSpec + mAppendedPath. In the HFSPlus version of the
object, mAppendedPath will be a unicode string instead of a char string.
  That's what I'm thinking now - it'll become more clear after actually working
on it.

Updated

16 years ago
Target Milestone: mozilla0.9.9 → mozilla1.0
(Assignee)

Updated

16 years ago
Keywords: mozilla1.0
(Assignee)

Comment 24

16 years ago
nominating nsbeta1
Keywords: nsbeta1

Updated

16 years ago
Keywords: nsbeta1 → nsbeta1+
(Assignee)

Comment 25

16 years ago
*** Bug 89064 has been marked as a duplicate of this bug. ***

Comment 26

16 years ago
Just my usual reminder that FSRefs can only point to exisitng file system
objects making them  less than exact replacements for FSSpecs.

Comment 27

16 years ago
Adding this as blocker for 102998 (since 89064 got closed as a duplicate of this)
Blocks: 102998
(Assignee)

Comment 28

16 years ago
> Just my usual reminder that FSRefs can only point to exisitng file system
objects making them  less than exact replacements for FSSpecs.

You don't have to remind me of that ;-) Got it under control. Read some of the
previous comments.

Comment 29

16 years ago
Sorry, comment #23 scared me :-)
(Assignee)

Comment 30

16 years ago
Changing summary since OSX is not required to take advantage of this.
Summary: upgrade file system code to use Unicode directly on MacOS X → upgrade file system code to use Unicode when HFS+ exists

Comment 31

16 years ago
Will this do anything for bug 125621 (non-ASCII path interpreted incorrectly by
input type=file)?
(Assignee)

Comment 32

16 years ago
*** Bug 133611 has been marked as a duplicate of this bug. ***
(Reporter)

Comment 33

16 years ago
I think it is too risky and too late to do this one for beta1. I suggest we
change this bug from nsbeta1+ to nsbeta1- and try this again later. 

Comment 34

16 years ago
There are a number of problems with Japanese and non-ASCII file names on Mac OS X:
bug 107181, bug 125614, bug 125621, bug 125640, bug 137846, bug 138008, bug 138350

I don't know how many of them pertain to this issue (if any) but if many of them
do, that may make it more worthwhile to try to get this change into Mozilla 1.0.

Comment 35

16 years ago
Just to give everyone an example of how badly this needs to be fixed for 1.0, I
was not able to download Mozilla 1.0 RC1 for Windows off of mozilla.org using
the Mozilla 1.0 RC1 build for Mac OS X, because the file name is too long. It's
too long because Mozilla on OS X doesn't understand the fact that OS X actually
supports file names longer than 31 characters. This really makes Mozilla look
pretty lame on OS X. Can you hear IE uses now? "What? I can't download files
with names longer than 31 characters? And they call themselves standards complient."
(Assignee)

Comment 36

16 years ago
I agree that this needs to be fixed ASAP. Believe me, if I had my choice of what
to work on today, it would be this. Unfortunately, there are some higher
priority bugs I have to deal with first.

> I was not able to download Mozilla 1.0 RC1 for Windows off of mozilla.org using
the Mozilla 1.0 RC1 build for Mac OS X

Hmm. That used to work - See bug 56295 which is resolved. In the meanwhile, the
file downloading code was heavily re-worked and that may have circumvented the
fix from 56295.

Comment 37

16 years ago
Believe you me, I want this bug fixed as much as anyone.

That being said, the latest version of IE in OS X does exactly the same thing:
downloads are limited to 31 characters.

So the worst that can be said is that Moz is "average" instead of "ahead of the
pack" in this area.

Comment 38

16 years ago
To continue the chatter on this:

> That being said, the latest version of IE in OS X does exactly the same thing:
> downloads are limited to 31 characters.

> So the worst that can be said is that Moz is "average" instead of "ahead of the
> pack" in this area.

No!  It is behind the pack, as opposed to IE in OS X, Mozilla won't even
download files longer than 31 characters.  IE truncates the name at least, and
saves it to file.  Mozilla gives you the impression that its saving, but the
file vanishes into the ether.
(Assignee)

Comment 39

16 years ago
*** Bug 142043 has been marked as a duplicate of this bug. ***

Updated

16 years ago
Blocks: 142043

Comment 40

16 years ago
Chris:
I think parts of that download problem were recently addressed in bug 139360.
You should check the fix date, and report remaining problems there.
No longer blocks: 142043
(Assignee)

Updated

16 years ago
Blocks: 142043
(Reporter)

Comment 41

15 years ago
also, please see 
http://developer.apple.com/technotes/tn/tn1150.html

look at HFSUniStr255 

also
Text Encodings

Current Mac OS programming interfaces pass filenames as Pascal strings (either
as a StringPtr or as a Str63 embedded in an FSSpec). The characters in those
strings are not Unicode; the encoding varies depending on how the system
software was localized and what language kits are installed. Identical sequences
of bytes can represent vastly different Unicode character sequences. Similarly,
many Unicode characters belong to more than one Mac OS text encoding.

HFS Plus includes two features specifically designed to help Mac OS handle the
conversion between Mac OS-encoded Pascal strings and Unicode. The first feature
is the textEncoding field of the file and folder catalog records. This field is
defined as a hint to be used when converting the record's Unicode name back to a
Mac OS- encoded Pascal string.

....

struct HFSPlusCatalogKey {
    UInt16              keyLength;
    HFSCatalogNodeID    parentID;
    HFSUniStr255        nodeName;
};

...
struct HFSPlusCatalogFolder {
....
    UInt32              textEncoding;
....
};

...
struct HFSPlusCatalogThread {
....
    HFSUniStr255        nodeName;
};
(Assignee)

Comment 42

15 years ago
Frank, an HFS+ file implementation is already checked in on the Chimera branch.
It does no conversion at all between Mac OS-encoded Pascal strings and Unicode.
The *Unicode* methods on nsIFile go straight to HFS+ APIs as Unicode. The
*Native* methods use UTF-8. So, the only conversion is between Unicode & UTF-8.
UTF-8 was chosen as native because all of the BSD-level routines on OSX take
*only* UTF-8. So, a path from GetNativePath() can be passed to fopen() and the
right thing happens. Also, the conversion between Unicode & UTF-8 is quick and
non-lossy.

That said, there is a problem in getting this code to work in a CFM (Code
Warrior)  environment because MSL's stdio routines take only HFS, narrow
charset, colon-delimited paths (bogus) and because nsIFileSpec for Mac CFM is
FSSpec-based.

Comment 43

15 years ago
> That said, there is a problem in getting this code to work in a CFM (Code
> Warrior)  environment because MSL's stdio routines take only HFS, narrow
> charset, colon-delimited paths (bogus) and because nsIFileSpec for Mac CFM is
> FSSpec-based.

Does that change with Pro 8?
(Reporter)

Updated

15 years ago
Blocks: 157673

Comment 44

15 years ago
*** Bug 137846 has been marked as a duplicate of this bug. ***
*** Bug 159637 has been marked as a duplicate of this bug. ***

Updated

15 years ago
Blocks: 167753

Comment 46

15 years ago
How about an updated target for this?
(Assignee)

Comment 47

15 years ago
-> 1.3

BTW, the HFS+ file impl is now in use by the Mach-0 build. Fixing summary to
reflect that this is a CFM-only problem.
Summary: upgrade file system code to use Unicode when HFS+ exists → upgrade Mac CFM file system code to use Unicode when HFS+ exists
Target Milestone: mozilla1.0 → mozilla1.3beta

Comment 48

15 years ago
BTW, how well does the Mach-O build work? I've been using OSX latest-trunk
builds for the past year, and I've been pretty happy with them. Can I just drop
replace the CFM app with Mach-O and everything will keep working?

Updated

15 years ago
Blocks: 180209

Updated

15 years ago
No longer blocks: 102998

Comment 49

15 years ago
Changing QA contact to bobj for now. Bob, please re-assign further as you see is
appropriate.
QA Contact: andreasb → bobj

Comment 50

15 years ago
Now that CFM has been retired in favor of Mach-O, has this bug become more or
less irrelevant? Some of the bug dependencies probably should be changed.

Comment 51

15 years ago
Sorry for the spam. Fizzilla CFM build is dead; should this bug go with it?
Keywords: qawanted

Comment 52

15 years ago
To paraphrase Dr. McCoy, CFM is dead Jim.
Status: ASSIGNED → RESOLVED
Last Resolved: 15 years ago
Resolution: --- → WONTFIX

Comment 53

15 years ago
Does that mean 1.3b will only have a macho build?

Comment 54

15 years ago
To answer the 1.3b question, yes, it's Mach-O only for Mac

Comment 55

15 years ago
*** Bug 197003 has been marked as a duplicate of this bug. ***
You need to log in before you can comment on or make changes to this bug.