Bugzilla

(In reply to Magnus Melin [:mkmelin] from comment #39)
> Did you change the CopyASCIItoUTF16 to use CopyUTF8toUTF16? https://searchfox.org/comm-central/rev/57ab5f2d0feff8be94d400b745d06aa55dbde61c/mailnews/imap/src/nsImapMailFolder.cpp#1858

Yes, been doing that all along.

I found a problem here https://searchfox.org/comm-central/rev/c3b6043c388edb0c72244958863fe2cbaf73149c/mailnews/db/msgdb/src/nsMsgDatabase.cpp#3394. Length() returns the number of UTF16 16-bit words and not the number of bytes after conversion to UTF8. That's why a byte was cut off of the onlineName UTF8 string. Doing strlen() on the null terminated converted string gives the right byte length (see diff fragment beflow). A similar use of strlen() is done here: https://searchfox.org/comm-central/rev/c3b6043c388edb0c72244958863fe2cbaf73149c/mailnews/base/src/nsMsgFolderCacheElement.cpp#107
```
/* static */ struct mdbYarn* nsMsgDatabase::nsStringToYarn(
     struct mdbYarn* yarn, const nsAString& str) {
   yarn->mYarn_Buf = ToNewCString(NS_ConvertUTF16toUTF8(str));
-  yarn->mYarn_Size = str.Length() + 1;
+  yarn->mYarn_Size = strlen((const char*)yarn->mYarn_Buf) + 1;
   yarn->mYarn_Fill = yarn->mYarn_Size - 1;
   yarn->mYarn_Form =
       0;  // what to do with this? we're storing csid in the msg hdr...
   return yarn;
 }
```
After this, I realized that if you create or rename a folder name with non-ascii UTF8, it is sent to the server as mUTF7. It works OK and looks right in tb but when you go to webmail you see the strange mUTF7 string for the folder name. (I've only checked this on gmail.) I'm still in the process of making create and rename send UTF8 when UTF8=ACCEPT is enabled.

(In reply to Magnus Melin [:mkmelin] from comment #39)
> Did you change the CopyASCIItoUTF16 to use CopyUTF8toUTF16? https://searchfox.org/comm-central/rev/57ab5f2d0feff8be94d400b745d06aa55dbde61c/mailnews/imap/src/nsImapMailFolder.cpp#1858

Yes, been doing that all along.

I found a problem here https://searchfox.org/comm-central/rev/c3b6043c388edb0c72244958863fe2cbaf73149c/mailnews/db/msgdb/src/nsMsgDatabase.cpp#3394. Length() returns the number of UTF16 16-bit words and not the number of bytes after conversion to UTF8. That's why a byte was cut off of the onlineName UTF8 string. Doing strlen() on the null terminated converted string gives the right byte length (see diff fragment beflow). A similar use of strlen() is done here: https://searchfox.org/comm-central/rev/c3b6043c388edb0c72244958863fe2cbaf73149c/mailnews/base/src/nsMsgFolderCacheElement.cpp#107
```
/* static */ struct mdbYarn* nsMsgDatabase::nsStringToYarn(
     struct mdbYarn* yarn, const nsAString& str) {
   yarn->mYarn_Buf = ToNewCString(NS_ConvertUTF16toUTF8(str));
-  yarn->mYarn_Size = str.Length() + 1;
+  yarn->mYarn_Size = strlen((const char*)yarn->mYarn_Buf) + 1;
   yarn->mYarn_Fill = yarn->mYarn_Size - 1;
   yarn->mYarn_Form =
       0;  // what to do with this? we're storing csid in the msg hdr...
   return yarn;
 }
```
After this, I realized that if you create or rename a folder name with non-ascii UTF8, it is sent to the server as mUTF7. It works OK and looks right in tb but when you go to webmail you see the strange mUTF7 string for the folder name. (I've only checked this on gmail.) I'm still in the process of making create and rename send UTF8 when UTF8=ACCEPT is enabled.
Edit 22 days later: In my patch I now send create and rename strings as UTF8 when UTF8=ACCEPT is in effect. So looks ok in webmail now.