1388081 - Folders with Japanese in them don't point to the proper mbox file

Status: RESOLVED INVALID

(Thunderbird :: Folder and Message Lists, defect)

Description

[Miss Andri]

Attachment: thunderbirdbug.jpg

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36

Steps to reproduce:

Have an existing folder/import a folder made prior to version 52.2.1 that has Japanese text in it.


Actual results:

Thunderbird creates an empty mbox with a gibberish file name and points the folder to it. This may happen multiple times, creating duplicate empty folders with the same name.


Expected results:

Japanese folders should point to their properly named mbox files (or, if 52.2.1 no longer allows unicode file names, it should properly CONVERT the files, not create new empty files).

Comment 1

[Jorg K (CEST = GMT+2)]

Thunderbird encodes folder names with non-ASCII characters, for example テスト becomes bc7e9936.
This is done in NS_MsgHashIfNecessary(). Thunderbird has never allowed unicode mbox names.

I don't know what you mean by "Have an existing folder/import a folder".
Surely folders created in, for example TB 45, will continue to work as TB 45 used the same encoding scheme.
TB doesn't support any folder import, so if the folder is "imported" by a non-compliant third-party program, contact the author.

Comment 2

[Miss Andri]

"Have an existing folder" means just that. If you look at the screenshot I uploaded, you'll see that the folders I made in Thunderbird are all saved on my harddrive as unicode. If it was never allowed then I'm not sure how it happened, but EVERY Japanese folder I ever made in Thunderbird is saved in unicode and I've been spending the past few hours manually converting them.

In case anyone else encounters the same bizarre problem, the fix is to delete the empty boxes with the correct mbox name and then rename the files to that name. Eg. delete bc7e9936, bc7e9936.msf, bc7e9936.sbd; then rename テスト, テスト.msf, テスト.sbd to bc7e9936, bc7e9936.msf, bc7e9936.sbd. When you reopen Thunderbird the folders you renamed should be back to normal.

Comment 3

[Jorg K (CEST = GMT+2)]

(In reply to Miss Andri from comment #2)
> "Have an existing folder" means just that. If you look at the screenshot I
> uploaded, you'll see that the folders I made in Thunderbird are all saved on
> my harddrive as unicode. If it was never allowed then I'm not sure how it
> happened, but EVERY Japanese folder I ever made in Thunderbird is saved in
> unicode and I've been spending the past few hours manually converting them.
There seems to be a misunderstanding here. Surely the folder name shown in TB is in Japanese, but the actual folder name was never using Japanese characters in Unicode. Your screenshot shows the actual files in the OS file system as encoded.

I don't know which program created files named テスト, テスト.msf, テスト.sbd.

Comment 4

[Miss Andri]

Well, the OS file system screenshot is not the best example now that I know TB encodes the files (although the bottom blue box does show a unicode file: アカウント). But, basically, what happens is that when TB encounters mbox files that are unicode it will create empty encoded files (the f25d3819 in the top blue box, also an msf file which isn't in the screenshot) and attempt to use them in place of the unicode files. Also, for some reason the encoded file without the extension created by TB is read (by Win 10 at least) as a folder, not a file. 

I created all my folders in the normal way through TB and they worked exactly as they were supposed to until today (well, since it's past 4am I guess yesterday), when I noticed the problem I described in my initial report. When I opened up my Mail folder to try and figure out what was wrong with my Japanese folders every single one of the files were in unicode so I mistakenly assumed that was how TB saved unicode folders. But since TB was programmed not to allow unicode mboxes then it only makes sense that it can't handle it properly when the files are in unicode.

Anyway, unless someone else encounters a similar problem or the problem happens again and I figure out how to replicate it, I can only assume that whatever broke my mboxes wasn't TB. It would be nice if TB could rename the unicode files instead of creating a new empty encoded file, but I know it doesn't make a lot of sense to spend time on a problem that is unlikely to be encountered again.