Closed Bug 95481 Opened 23 years ago Closed 22 years ago

upgrade Mac CFM file system code to use Unicode when HFS+ exists

Categories

(Core :: Internationalization, defect)

Product:
Core
Component:
Internationalization
Platform:
PowerPC
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX
Milestone:
mozilla1.3beta

People

(Reporter: ftang, Assigned: ccarlen)

References

Details

(Keywords: intl, qawanted)

Attachments

(1 file)

Adding methods with FSRef.
3.25 KB, patch
Details | Diff | Splinter Review 
read 
http://gemma.apple.com/techpubs/macosx/Carbon/Files/FileManager/File_Manager/
Functions/FunctionGroups.html

It looks like at least the following APIs support Unciode/CFString
Creating a File System Reference (FSRef)
 FSMakeFSRefUnicode     	Constructs an FSRef for a file or directory, given a 
parent directory and a Unicode name. 
 PBMakeFSRefUnicodeSync	 	Constructs an FSRef for a file or directory, 
given a parent directory and a Unicode name. 
 PBMakeFSRefUnicodeAsync	Constructs an FSRef for a file or directory, given a 
parent directory and a Unicode name. 

Creating Directories
 FSCreateDirectoryUnicode	Creates a new directory (folder) with a Unicode name. 
 PBCreateDirectoryUnicodeSync	Creates a new directory (folder) with a Unicode 
name. 
 PBCreateDirectoryUnicodeAsync	  Creates a new directory (folder) with a Unicode 
name. 

Creating Files
 FSCreateFileUnicode	Creates a new file with a Unicode name. 
 PBCreateFileUnicodeSync	Creates a new file with a Unicode name. 
 PBCreateFileUnicodeAsync	Creates a new file with a Unicode name. 

Determining the Unicode Names of the Data and Resource Forks
 FSGetDataForkName	Returns a Unicode string constant for the name of the data 
fork.
 FSGetResourceForkName	Returns a Unicode string constant for the name of the 
resource for

 Moving and Renaming Files or Directories
 FSRenameUnicode	Renames a file or folder.
 PBRenameUnicodeSync	Renames a file or folder.
 PBRenameUnicodeAsync	Renames a file or folder.
Status: NEW → ASSIGNED
see also 95478 about file picker

All the Unicode related funciont is using FSRef instead of FSSpec. This could be 
hard since we have FSSpec in mnay many places-
including plugin, nspr, liveconnect, java, mrj, xpcom/io, etc.

It's also hard because an FSRef must point to an existing file system entity; you 
can't have one point to a file that is yet to be created, like you can with an 
FSSpec.

If you look at the Availability info for these Unicode routines, they are
available in InterfaceLib 9.0 and later. In other words, this does not require
OS X an should be done for any system on which the Gestalt selector
gestaltFSAttr returns a response with the gestaltHasHFSPlusAPIs bit set.

Can I have this bug?
sfraser: about your comment 
"FSRef must point to an existing file system entity".

see http://developer.apple.com/technotes/tn/tn2022.html
TN2022 The Death of typeFSSpec: moving along to typeFileURL

"In a nutshell, typeFileURL is a core-foundation URL en"coded to a stream of
byte s in UTF8 format. This is the suggested data type to use when your
application would like to create a reference to 
a file that has yet to be created. Furthermore, there are a number of other good
reasons to use this type; depending on your processing requirements, you may
wish to use this data type in a number of different circumstances. Here are some
properties and features of the typeFileURL data format:

    * typeAlias and typeFSRef require deterministic reference and cannot refer
to files that have yet to be created. typeFileURL uses the weak "by name" style
reference provided by URLs, and as such is quite capable of providing
pre-determined references to files.

"


wtc- please read
http://gemma.apple.com/techpubs/macosx/SystemOverview/SystemOverview/FileSystem/File_Encodings_and_Fonts.html

"In addition, all code that calls BSD system routines should ensure that the
const *char parameters of these routines are in UTF-8 encoding. All BSD system
functions expect their string parameters to be in UTF-8 encoding and nothing else.
"

and "QA1028 ICLaunchURL, "file:///" URLs and Mac OS X"
http://developer.apple.com/qa/qa2001/qa1028.html
"    * Text Encodings -- In Mac OS 9 the URL components are assumed to be
percent-encoded PStrings, as returned by PBGetCatInfo. This is inadequate for
Mac OS X, where the File Manager fully supports Unicode file names. In Mac OS X
the URL components are defined as percent-encoded UTF-8.
...
One possible workaround is to change your code to generate the "file:///" URL in
the right format. You can do this with a sequence of File Manager, CFURL, and
CFString calls.

An easier workaround is to avoid ICLaunchURL entirely and change your code to
call the Launch Services routine LSOpenCFURLRef. This code is very simple.

   1. If you're starting with an FSRef, just skip to step 2. If you're starting
with an FSSpec, convert it to an FSRef using FSpMakeFSRef.
   2. Create a CFURL based on the FSRef using CFURLCreateFromFSRef.
   3. Pass that CFURL to LSOpenCFURLRef.
   4. Release your reference to the CFURL.
"
see 95560 about return "UTF-8" as filename charset in nsIPlatformCharset on
MacOS X. This will make the current file: url handling code assume "UTF-8" as
the file name encoding. However, this may introduce some problem with some Mac
file operation. 
reassign to ccarlen since he ask for it.
Assignee: ftang → ccarlen
Status: ASSIGNED → NEW
Blocks: 103669
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla0.9.7
taking
*** Bug 87831 has been marked as a duplicate of this bug. ***
Add myself to CC for reviewing bug 87831 later.
Blocks: 108000
-> 0.9.8
Target Milestone: mozilla0.9.7 → mozilla0.9.8
I am attending Apple's Carbon kiitchen.
There are clients of nsLocalFile which use methods with FSSpec. How about adding methods with FSRef, so 
clients do not have to use FSSpec? There are conversion API which convert between FSSpec and FSRef. I 
added methods which takes FSRef and just convert to FSSpec which is used for nsLocaleFileMac internal 
implementation.
Conrad, what do you think?

Note that an FSRef cannot refer to a file that does not yet exist, whereas an 
FSSpec can.

Simon, so is nsLocalFile used for that situtation, I mean for a file does not
exist? How about CFURL, can that be used for non existing files?
I think we cannot stick with FSSpec (e.g., the user can assign non default
language file names in Finder in OSX).
Keywords: intl
> so is nsLocalFile used for that situtation, I mean for a file does not
exist?

Yes. Frequently.

> How about CFURL, can that be used for non existing files?

That may be the way to go. Another is to use an FSRef + appended unicode path.
Some portion of a file path (at least the volume) must exist. A non-existent
file path can be specified with the combination of those 2.

It will either be done with CFURL or FSRef + unicode appended path, depending on
which gives better performance. But, either way, FSSpec will be going away.
Sorry, I still don't get it about the file not exist situation. Could you give
me examples? If that happens frequently, why not Apple supports it in FSRef?



See comment #5.

In mozilla, it happens all over the place. Typical scenario is:
(1) get a directory from directory service
(2) append a node or two (neither of them exist)
(3) if it doesn't exist, call create

After (2), the nsIFile specifies the file system object even though it doesn't
exist.
*** Bug 106271 has been marked as a duplicate of this bug. ***
Blocks: 56295
*** Bug 104935 has been marked as a duplicate of this bug. ***
Mass move to 0.9.9
Target Milestone: mozilla0.9.8 → mozilla0.9.9
Conrad, any decision about CFURL or FSRef which one to use for nsLocalFileMac?
  There should be an InitWithCFURL() on nsILocalFileMac (maybe nsILocalFileMacX
which would be a subclass). This seems useful for specifying files which may or
may not exist.
  Internally, the implementation will probably use an FSRef + mAppendedPath just
as the current impl uses FSSpec + mAppendedPath. In the HFSPlus version of the
object, mAppendedPath will be a unicode string instead of a char string.
  That's what I'm thinking now - it'll become more clear after actually working
on it.
Target Milestone: mozilla0.9.9 → mozilla1.0
Keywords: mozilla1.0
nominating nsbeta1
Keywords: nsbeta1
Keywords: nsbeta1 → nsbeta1+
*** Bug 89064 has been marked as a duplicate of this bug. ***
Just my usual reminder that FSRefs can only point to exisitng file system
objects making them  less than exact replacements for FSSpecs.
Adding this as blocker for 102998 (since 89064 got closed as a duplicate of this)
Blocks: 102998
> Just my usual reminder that FSRefs can only point to exisitng file system
objects making them  less than exact replacements for FSSpecs.

You don't have to remind me of that ;-) Got it under control. Read some of the
previous comments.
Sorry, comment #23 scared me :-)
Changing summary since OSX is not required to take advantage of this.
Summary: upgrade file system code to use Unicode directly on MacOS X → upgrade file system code to use Unicode when HFS+ exists
Will this do anything for bug 125621 (non-ASCII path interpreted incorrectly by
input type=file)?
*** Bug 133611 has been marked as a duplicate of this bug. ***
I think it is too risky and too late to do this one for beta1. I suggest we
change this bug from nsbeta1+ to nsbeta1- and try this again later. 
There are a number of problems with Japanese and non-ASCII file names on Mac OS X:
bug 107181, bug 125614, bug 125621, bug 125640, bug 137846, bug 138008, bug 138350

I don't know how many of them pertain to this issue (if any) but if many of them
do, that may make it more worthwhile to try to get this change into Mozilla 1.0.
Just to give everyone an example of how badly this needs to be fixed for 1.0, I
was not able to download Mozilla 1.0 RC1 for Windows off of mozilla.org using
the Mozilla 1.0 RC1 build for Mac OS X, because the file name is too long. It's
too long because Mozilla on OS X doesn't understand the fact that OS X actually
supports file names longer than 31 characters. This really makes Mozilla look
pretty lame on OS X. Can you hear IE uses now? "What? I can't download files
with names longer than 31 characters? And they call themselves standards complient."
I agree that this needs to be fixed ASAP. Believe me, if I had my choice of what
to work on today, it would be this. Unfortunately, there are some higher
priority bugs I have to deal with first.

> I was not able to download Mozilla 1.0 RC1 for Windows off of mozilla.org using
the Mozilla 1.0 RC1 build for Mac OS X

Hmm. That used to work - See bug 56295 which is resolved. In the meanwhile, the
file downloading code was heavily re-worked and that may have circumvented the
fix from 56295.
Believe you me, I want this bug fixed as much as anyone.

That being said, the latest version of IE in OS X does exactly the same thing:
downloads are limited to 31 characters.

So the worst that can be said is that Moz is "average" instead of "ahead of the
pack" in this area.
To continue the chatter on this:

> That being said, the latest version of IE in OS X does exactly the same thing:
> downloads are limited to 31 characters.

> So the worst that can be said is that Moz is "average" instead of "ahead of the
> pack" in this area.

No!  It is behind the pack, as opposed to IE in OS X, Mozilla won't even
download files longer than 31 characters.  IE truncates the name at least, and
saves it to file.  Mozilla gives you the impression that its saving, but the
file vanishes into the ether.
*** Bug 142043 has been marked as a duplicate of this bug. ***
Blocks: 142043
Chris:
I think parts of that download problem were recently addressed in bug 139360.
You should check the fix date, and report remaining problems there.
No longer blocks: 142043
Blocks: 142043
also, please see 
http://developer.apple.com/technotes/tn/tn1150.html

look at HFSUniStr255 

also
Text Encodings

Current Mac OS programming interfaces pass filenames as Pascal strings (either
as a StringPtr or as a Str63 embedded in an FSSpec). The characters in those
strings are not Unicode; the encoding varies depending on how the system
software was localized and what language kits are installed. Identical sequences
of bytes can represent vastly different Unicode character sequences. Similarly,
many Unicode characters belong to more than one Mac OS text encoding.

HFS Plus includes two features specifically designed to help Mac OS handle the
conversion between Mac OS-encoded Pascal strings and Unicode. The first feature
is the textEncoding field of the file and folder catalog records. This field is
defined as a hint to be used when converting the record's Unicode name back to a
Mac OS- encoded Pascal string.

....

struct HFSPlusCatalogKey {
    UInt16              keyLength;
    HFSCatalogNodeID    parentID;
    HFSUniStr255        nodeName;
};

...
struct HFSPlusCatalogFolder {
....
    UInt32              textEncoding;
....
};

...
struct HFSPlusCatalogThread {
....
    HFSUniStr255        nodeName;
};

Frank, an HFS+ file implementation is already checked in on the Chimera branch.
It does no conversion at all between Mac OS-encoded Pascal strings and Unicode.
The *Unicode* methods on nsIFile go straight to HFS+ APIs as Unicode. The
*Native* methods use UTF-8. So, the only conversion is between Unicode & UTF-8.
UTF-8 was chosen as native because all of the BSD-level routines on OSX take
*only* UTF-8. So, a path from GetNativePath() can be passed to fopen() and the
right thing happens. Also, the conversion between Unicode & UTF-8 is quick and
non-lossy.

That said, there is a problem in getting this code to work in a CFM (Code
Warrior)  environment because MSL's stdio routines take only HFS, narrow
charset, colon-delimited paths (bogus) and because nsIFileSpec for Mac CFM is
FSSpec-based.
> That said, there is a problem in getting this code to work in a CFM (Code
> Warrior)  environment because MSL's stdio routines take only HFS, narrow
> charset, colon-delimited paths (bogus) and because nsIFileSpec for Mac CFM is
> FSSpec-based.

Does that change with Pro 8?
Blocks: 157673
*** Bug 137846 has been marked as a duplicate of this bug. ***
*** Bug 159637 has been marked as a duplicate of this bug. ***
Blocks: 167753
How about an updated target for this?
-> 1.3

BTW, the HFS+ file impl is now in use by the Mach-0 build. Fixing summary to
reflect that this is a CFM-only problem.
Summary: upgrade file system code to use Unicode when HFS+ exists → upgrade Mac CFM file system code to use Unicode when HFS+ exists
Target Milestone: mozilla1.0 → mozilla1.3beta
BTW, how well does the Mach-O build work? I've been using OSX latest-trunk
builds for the past year, and I've been pretty happy with them. Can I just drop
replace the CFM app with Mach-O and everything will keep working?
Blocks: 180209
No longer blocks: 102998
Changing QA contact to bobj for now. Bob, please re-assign further as you see is
appropriate.
QA Contact: andreasb → bobj
Now that CFM has been retired in favor of Mach-O, has this bug become more or
less irrelevant? Some of the bug dependencies probably should be changed.

Sorry for the spam. Fizzilla CFM build is dead; should this bug go with it?
Keywords: qawanted
To paraphrase Dr. McCoy, CFM is dead Jim.
Status: ASSIGNED → RESOLVED
Closed: 22 years ago
Resolution: --- → WONTFIX
Does that mean 1.3b will only have a macho build?
To answer the 1.3b question, yes, it's Mach-O only for Mac
*** Bug 197003 has been marked as a duplicate of this bug. ***
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: