Closed Bug 194618 Opened 22 years ago Closed 17 years ago

Mail folders names with accented characters have become corrupted when upgrading from Mozilla 1.2.1 to 1.3beta

Categories

(MailNews Core :: Internationalization, defect)

Product:
MailNews Core
Component:
Internationalization
Platform:
PowerPC
macOS
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: vargenau, Unassigned)

Details

(Keywords: dataloss, intl, Whiteboard: [adt2])

Attachments

(2 files)

screenshot as requested by Comment #15
2.45 KB, image/jpeg
Details
a screen shot of the corrupted accented chars after upgrading
114.77 KB, image/jpeg
Details 
User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.3b) Gecko/20030212
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.3b) Gecko/20030212

Mail folders names with accented characters have become corrupted when upgrading
from Mozilla 1.2.1 to 1.3beta.

For instance, "Valérie" became "ValeÃÅrie".


Reproducible: Always

Steps to Reproduce:
1. Create a folder named "Valérie" with Mozilla 1.2.1
2. Open Mozilla 1.3beta
3. Check to name of the folder

Actual Results:  
The name of the folder became "ValeÃÅrie"

Expected Results:  
The name of the folder should be "Valérie".


I also upgraded from Mac OS X 10.2 to Mac OS X 10.2.4 the same day
I upgraded from Mozilla 1.2.1 to Mozilla 1.3beta.
A CFM-to-Mach transition issue?
Since Mozilla 1.3b, Mozilla Mail/Newsgroups is uanble to display folders and
subfolders named in Japanese.

This seems to be caused by the transition from Carbon-based builds to Mach-O
builds. This bug happens with a recent nightly build:

Mac OS X ver 10.2.4
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.3) Gecko/20030312

This issue needs an immediate attention, and should be fixed as soon as 
possible, for this bug prevents Japanese users from using Japanese folders and
subfolders in Mozilla.
Flags: blocking1.4a+
Request blocking 1.4a
This is critical bug for Japanese Mac OS X users.

Related bug: bug 172472
Flags: blocking1.4a+ → blocking1.4a?
This is critical also for French users and I suppose all international users.

In my opinion, it should have blocked 1.3 final.

See also bug 194616

Regards,
Severity: normal → critical
Keywords: dataloss
Keywords: intl
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking1.4a? → blocking1.4a+
> A CFM-to-Mach transition issue

let's hope so, maybe the HashIfNecessary code.
Assignee: smontagu → sspitzer
maybe not NS_MsgHashIfNecessary(), but I am able to reproduce this.

on a related note to NS_MsgHashIfNecessary(), see also mozilla\xpcom\ds\nsCRT.h

#if defined(XP_MAC)
  #define FILE_PATH_SEPARATOR       ":"
  #define FILE_ILLEGAL_CHARACTERS   ""
#elif defined(XP_WIN) || defined(XP_OS2)
  #define FILE_PATH_SEPARATOR       "\\"
  #define FILE_ILLEGAL_CHARACTERS   "/:*?\"<>|"
#elif defined(XP_UNIX) || defined(XP_BEOS)
  #define FILE_PATH_SEPARATOR       "/"
  #define FILE_ILLEGAL_CHARACTERS   ""
#else
  #error need_to_define_your_file_path_separator_and_illegal_characters
#endif
Flags: blocking1.4a+ → blocking1.4a-
Any relation to bug 199384?
Marina, could you try this with today's trunk?
bug 199384 might have fixed this

Marc-Etienne, you also could test using a current nightly.
is this a regression? should this be nsbeta1?
The bug is still present in 1.4alpha.
I will test with a nightly ASAP (Tuesday 8th of April).

Regards,
At nightly 2003040608 (on Mac OS X 10.2.4),
Any decomposed Unicode characters are shown individually.
See http://developer.apple.com/qa/qa2001/qa1235.html. From that:

"This isn't a problem as long as you use system-provided APIs to process text.
Apple's APIs correctly handle both precomposed and decomposed Unicode."

If the system-provided APIs have that ability, why doesn't our text rendering
handle it?


New tests made with :
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4b) Gecko/20030406

1) when starting this new Mozilla, the folder names still appear corrupted
e.g. Valérie appears as ValeÃÅrie

2) It impossible to rename the folder to Valérie

3) It is possible to rename the folder in two steps :
first rename it to Valerie, then to Valérie

Regards,

>e.g. Valérie appears as ValeÃÅrie
Could you attach a screen shot? This does not seem to be the decomposed char
problem but not sure because non-Ascii text in bugzilla shown differently based
on the charset.
Keywords: nsbeta1
Is it possible that a file system path is stored by the 1.2.1 version using a
raw path in the (then) system charset?   
I am not sure if the file name for the folder is stored somewhere (in system
charset or UTF-8).
I think Naving or Seth know about it.

panacea.dat is full of file paths, which is verboten on the Mac. That's why most
everything else uses persistent descriptors. Anyway, when reading these file
paths, if it could be determined that the strings are not valid UTF-8, which is
what they would be in Mach-O build, they could be assumed to be legacy encodings
made by CFM build and mapped. For the IsUTF-8 util, see bug 191541.
Mail & I18n triage: nsbeta1+/adt2
Keywords: nsbeta1 → nsbeta1+
Whiteboard: [adt2]
over to cavin
Assignee: sspitzer → cavin
I didn't see the problme using today's mozilla build (042903). However, I did see the 
problem using 040103 build. So it looks like the problem was fixed somehow. Reporter, 
can you try the latest build and verify this? Thanks.  Note: I did not try to figure out the 
earliest build that fixed the problem.
Hello,

I am the reporter.
I am sorry, but I am in vacation with only a 56 kbit/s modem, and I will not be able
to download a nightly build before mid-May.

Regards,
Marina, can you also try the latest build and see if the problem is fixed? Thanks. 
WFM also on my current trunk build. I created a local folder called "Valérie"
with mozilla 1.1. I'm pretty sure that 1.1 vs. 1.2.1 would not make a difference.

What may:
If the folder was originally created when the machine was running OS9 and/or the
system script was not MacRoman.

Here's part of my panacea.dat:
(98FF3
    =/Users/conrad/Library/Mozilla/Profiles/default/huwz5dlv.slt/Mail/Local Fo\
lders/Val$8Erie.msf)(98FF4=Val$C3$A9rie)

$8E is eacute in MacRoman. How that's getting converted back to a file since the
native charset is UTF-8, I don't know. I'm running 10.2.5. Possible that this
update to the OS made some change which falls back to MacRoman if given an
invalid UTF-8 string?
i am seeing the corruption when i migrate the old profile on POP account using
2003050103 build
> when i migrate the old profile

Can you clarify "migrate?" I thought this bug was about using profiles made with
old CFM mozilla with a current Mach-O mozilla. There's no migration in that
case, just use of the same profile with a newer app.
well, i am migrating an old profile which was crreated with mozilla 1.1 (or 1.2)
(made with old CFM) and i am using now a new file format. I thought we can use
this test case. however i can install the old builds (remind me the right date
of 1.1 (1.2 ) and upgrade to the newest build
> i am migrating an old profile which was crreated with mozilla 1.1

There's no migration in that case. The same profiles are used with 1.1 as
current builds. They are the same files - not copied or migrated in any way.
I'm going to mark it as WORKSFORME, please reopen it if you are able to
reproduce it (with a detailed description).
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → WORKSFORME
i am going to open a new bug for the migration issue (it might be that i created
the profile before 7.0). 
i am seeing the corruption after upgrading from 1.1 (i used 20030826 build) to
current trunk builds (2003050703). install 1.1 build and upgrade it to the
current trunk build, use the same profile: see the corruption of non-ascii chars
in folder's name (POP account). reopening
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
> i am seeing the corruption after upgrading from 1.1 (i used 20030826 build) to
> current trunk builds (2003050703).
>
So you created the profile and the folder in 1.1 right? I think this is a typo,
so is it 20020826 build instead of 2003?
yes, it's a typo..., it should be 20020826 unless you already have 20030826:-)
marina, what version of OS X are you running? If not 10.2.5, can you try with that?
it is 10.1.5, i'll try 10.2.5
taking, but I'll check with ccarlen when I get to it.
Assignee: cavin → sspitzer
Status: REOPENED → NEW
Product: MailNews → Core
sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.
Assignee: sspitzer → nobody
Does this bug still matter? Or can we close it out given that Mozilla 1.2.1 hasn't been supported for a *long* time.
yeah, I'm going to mark WFM - if this is still an issue, there's probably a much newer bug filed on it.
Status: NEW → RESOLVED
Closed: 21 years ago → 17 years ago
Resolution: --- → WORKSFORME
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: