IMAP: can't create non-ascii folder names

VERIFIED FIXED in M14

Status

P3
normal
VERIFIED FIXED
19 years ago
10 years ago

People

(Reporter: Bienvenu, Assigned: Bienvenu)

Tracking

Trunk
All
Windows NT

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [PDT+] W/b minus on 3/3)

Attachments

(1 attachment)

(Assignee)

Description

19 years ago
all the folder creation routines take single byte names, so double byte folders 
can't be created. Not sure if this is a beta blocker for i18n or not.
(Assignee)

Comment 1

19 years ago
accepting. Bob, please nominate as beta if you want. Whatever I nominate gets 
shot down.
Status: NEW → ASSIGNED
Hardware: PC → All
(Assignee)

Comment 2

19 years ago
OK, I've checked in a fix (can't test it, though, since I don't know how to 
enter double byte folder names)
Status: ASSIGNED → RESOLVED
Last Resolved: 19 years ago
Resolution: --- → FIXED

Comment 3

19 years ago
Thanks.  Changing QA assigned to momoi.
QA Contact: lchiang → momoi

Comment 4

19 years ago
David, is this for IMAP folders only? or for both IMAP and local
folders? I had thought that it is for IMAP only but you
didn't qualify it that way.

Comment 5

19 years ago
CC'ing ji and marina. Need to check this out on Linux also.
(Assignee)

Comment 6

19 years ago
It's really for IMAP only, but I did a lot of the work that will be required to
do it for local folders. That's not done, however.

Comment 7

19 years ago
Thanks, I modified the summary line to make t easier to search for this later.
Summary: can't create double byte folder names → IMAP: can't create double byte folder names

Comment 8

19 years ago
** Checked with 2/7/2000 Win32 build **

I tried creating IMAP folder names in Japanese, I got 2 types of
results:

1. Server failure -- invalid mailbox name response. Some Kanji names like "Tokyo" had this problem.
2. A new folder got created but the name is not Japanese at all and wrong: I tried "momo" in Hiragana,
   the folder was created but it showed ASCII  "D".

2. 
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 9

19 years ago
I've found some more core msg code that assumes folder names are single byte - 
I'll fix those and check the changes in when the tree opens.
Target Milestone: M14
(Assignee)

Comment 10

19 years ago
checked in more fixes - if you could try it again with the 02/08/00 build, I'd 
appreciate it.
(Assignee)

Comment 11

19 years ago
I'll mark it fixed, though I don't know if it is.
Status: REOPENED → RESOLVED
Last Resolved: 19 years ago19 years ago
Resolution: --- → FIXED

Comment 12

19 years ago
With 2000020817 Win32 build, Japanese folder creation is not
working. I tried 3-4 different Japanese names under WinNT4-US
version using Global IME Japanese input editor and also 
under NT4-Japanese with native IME.  All 4 attempts
failed with "invalid mailbox name" msg.

Re-opening it for now...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 13

19 years ago
From Scott Putterman:

The problem was that the uri for this folder had a " " at the end.  So it was 
getting entered into the rdf resource hash table with a
space.  But, when we call the function parseURI, it calls nsStdURL::SetSpec.  
SetSpec erases any " " at the beginning or end. 
Because it's operating on the actual uri and the uri is the actual key into the 
resource hash table, every time it would try to find the
original uri in the hash table it couldn't because it had been modified to lose 
the " ".  So GetResource would keep creating the folder
over and over again which caused problems. 

Scott, if you check in this change, let me know so I can reassign this to you, 
or just mark it fixed./
The step I took to solve it is  in nsMsgFolder::parseURI instead of passing the 
folder's uri to SetSpec, I pass a copy.  This way the uri
is never modified.  When I did this, the Japanese folder showed up with the 
correct name and my problems went away.  The only
problem I see with this is that we use the result of SetSpec to get other 
information in parseURI and this won't necessarily be the same
as the information from the original uri since the uri passed to SetSpec could 
be modified. 
(Assignee)

Comment 14

19 years ago
Scott, did Andreas checkin his change to url parsing? Did that fix this for you?  
Perhaps someone in i18N could try this again? thanks.

Comment 15

19 years ago
Andreas just checked it in and it works for me.
(Assignee)

Comment 16

19 years ago
OK, I'm going to mark this fixed, then. PDT team, please note that I didn't 
actually change any source code for this, but it's going to show up as one of 
those evil evil non PDT+ bugs fixed anyway. Sorry 'bout that.
Status: REOPENED → RESOLVED
Last Resolved: 19 years ago19 years ago
Resolution: --- → FIXED

Comment 17

19 years ago
** Checked with 2/16/2000 Win32 build: 2000021615 **

I don't think this is fixed. I tried various Japanese
characters for creating a folder and none succeeded. 
Each I got a server error msg informing me that it is
"invalid mailbox name".

Re-opening ...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 18

19 years ago
weird, why would it work for Scott but not Momoi?
Status: REOPENED → ASSIGNED

Comment 19

19 years ago
It definitely worked for me when it wasn't working before.  But, I only tried 
one folder and there might be other names that don't work.

Comment 20

19 years ago
I'll re-try this with today's build. The build I looked at last night
may have been "bad" in more than one way. We need re-confirmation
with another build.
(Assignee)

Comment 21

19 years ago
I doubt it has to do with the build - it's probably more likely some difference 
in folder names you're entering. Could you try turning on imap protocol logging 
to see what kind of folder name we're creating?

set NSPR_LOG_MODULES=IMAP:5
set NSPR_LOG_FILE=c:\tmp\mapio.txt

Comment 22

19 years ago
I'm writing down a reminder to look at 0x5c (2nd byte) Japanese names as it requires special
handling. This needs to be done for local folders because "\" indicates a path
separator on Windows. We should probably look at other similar path seprator or directory related 
single-byte code points for IMAP.
(Assignee)

Comment 23

19 years ago
I've found a bug in the charset converter that converts to imap utf7 (or my use 
of it) - it's not ending the escaping. So if I pass in хий I should get 
"&AMgAyQDK-" but instead I get "&AMgAyQDK" or "AMgAyQDK"

Here's how I'm using the charset converter. Please let me know if I'm doing 
something wrong.

char*
CreateUtf7ConvertedStringFromUnicode(const PRUnichar * aSourceString)
{
  nsresult res;
  char *dstPtr = nsnull;
  PRInt32 dstLength = 0;
  char *convertedString = NULL;
  
  NS_WITH_SERVICE(nsICharsetConverterManager, ccm, kCharsetConverterManagerCID, 
&res); 

  if(NS_SUCCEEDED(res) && (nsnull != ccm))
  {
    nsString aCharset("x-imap4-modified-utf7");
    PRInt32 unicharLength;

      // convert from 8 bit ascii string to modified utf7
      nsString unicodeStr(aSourceString);
      nsIUnicodeEncoder* encoder = nsnull;
      aCharset.SetString("x-imap4-modified-utf7");
      res = ccm->GetUnicodeEncoder(&aCharset, &encoder);
      if(NS_SUCCEEDED(res) && (nsnull != encoder)) 
      {
        res = encoder->GetMaxLength(unicodeStr.GetUnicode(), 
unicodeStr.Length(), &dstLength);
        // allocale an output buffer
        dstPtr = (char *) PR_CALLOC(dstLength + 1);
        unicharLength = unicodeStr.Length();
        if (dstPtr == nsnull) 
        {
          res = NS_ERROR_OUT_OF_MEMORY;
        }
        else 
        {
          res = encoder->Convert(unicodeStr.GetUnicode(), &unicharLength, 
dstPtr, &dstLength);
          dstPtr[dstLength] = 0;
        }
      }
	  // ack, this is silly - why pass through unicodeStr2 back to 
convertedString?
      NS_IF_RELEASE(encoder);
      nsString unicodeStr2(dstPtr);
      convertedString = (char *) PR_Malloc(dstLength + 1);
      if (convertedString)
        unicodeStr2.ToCString(convertedString, dstLength + 1, 0);
        }
    PR_FREEIF(dstPtr);
    return convertedString;
}

Comment 24

19 years ago
For the return value, it can just return dstPtr instead of re-allocating the 
buffer. But that shouldn't change the result string.
cc to ftang and cata in case problem in the converter.
(Assignee)

Comment 25

19 years ago
yes, I knew I should have pointed out that I was doing that wrong, but I don't 
think that's what's broken.
(Assignee)

Comment 26

19 years ago
I've tried debugging this, but the code just seems wrong - I don't see 
FinishNoBuff getting called, which seems to be the code that would do the right 
thing.

Comment 27

19 years ago
How about calling nsIUnicodeEncoder::Finish explicitly after the conversion? 
I think that will ensure the buffer to be flushed.
(Assignee)

Comment 28

19 years ago
thanks, but that didn't work - it just stuck the '-' at the start of the string. 
Maybe I'm supposed to call it with the end of the string, but this just seems 
weird. Shouldn't the converter be doing this for me? Also, sometimes it seems to 
skip the '&' at the beginning.

Comment 29

19 years ago
>Shouldn't the converter be doing this for me?
That's because it needs to handle streams. The caller needs to call Finish() 
after Convert().
>I'm supposed to call it with the end of the string
Yes, the data to be appended to the data already converted by Convert().
(Assignee)

Comment 30

19 years ago
Wow, that's utterly brutal. Is there some other conversion mechanism I should 
use instead?

Comment 31

19 years ago
About the buffer size question (in 26877), GetMaxLength() should return the size 
big enough to hold the data generated by Finish(). But I need to double check 
that with Cata.
(Assignee)

Comment 32

19 years ago
I have a simple workaround for this problem in the utf7 converter. Nominating as 
beta1. As I understand it, the utf7 converter just doesn't work the way I'm 
using it, so I have to finish the conversion myself.
Keywords: beta1
Summary: IMAP: can't create double byte folder names → IMAP: can't create non-ascii folder names

Comment 33

19 years ago
why the summary got changed to "can not create a non-ascii folder name"? It is 
absolutely possible to create 8-bit folder name in ISO-8859 on IMAP.
(Assignee)

Comment 34

19 years ago
because if the last character is non-ascii, I don't think you'll be able to 
create the folder.

Comment 35

19 years ago
you're right....i didn't try this combination.
(Assignee)

Comment 36

19 years ago
neither did I :-(
(Assignee)

Comment 37

19 years ago
Created attachment 5921 [details] [diff] [review]
proposed fix

Comment 38

19 years ago
[PDT+] W/b minus on 3/3
Whiteboard: [PDT+] W/b minus on 3/3
(Assignee)

Comment 39

19 years ago
fix checked in
Status: ASSIGNED → RESOLVED
Last Resolved: 19 years ago19 years ago
Resolution: --- → FIXED

Comment 40

19 years ago
I used today's release win32 build and I was able to create a japanese folder 
which contains 0x5c. Looks like that is local folder only problem.

Comment 41

19 years ago
** Checked with 3/6/2000 Win32 build **

I can niw create JPN or Latin 1 folders with accents
with the above build on the IMAP.

Marking it verified as fixed.
Status: RESOLVED → VERIFIED

Comment 42

19 years ago
ji, please check on Linux and report here.
cwang, please check on Mac and report here.

Comment 43

19 years ago
Verified/fixed with linux 2000030709 build.
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.