Closed Bug 1512053 Opened 6 years ago Closed 3 years ago

Problem with using / (slash) in folder name using IMAP

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: cra3yk, Assigned: gds)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file, 3 obsolete files)

allow-slash-in-folder-name-v3-cleaned-up.diff
15.68 KB, patch
Details | Diff | Splinter Review 
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36

Steps to reproduce:

0a) Configure IMAP (on server) that use Maildir++ and different folder separator than / (slash) - for example $ or ' (I used Ubuntu LTS 16 + Dovecot)
0b) Configure Thunderbird to use this IMAP account

1) Try to create folders using names:
z.a (works)
y\a (works)
x/a (not working)


Actual results:

Thunderbird replaces "/" (slash) character with . (dot) when creating and send this via IMAP to server and Dovecot created 'x' folder with 'a' subfolder inside.

Thunderbird shows this 'x' folder using grey italic font (as inaccessible) - 'A' mark
https://i.imgur.com/eXPhKYk.png

In linux file system it looks like:
https://i.imgur.com/TpDElLF.png


Expected results:

It should determine IMAP folder/hierarchy delimiter (separator) char and use it correctly and create folder as usual.

The (z/z) and (y\y) 'B' mark example was created using latest (but old) Opera Mail (it works) and RoundCube Webmail (both determined correctly folder/hierarchy delimiter char correcly)
And second thing: When Thunderbird download (mirror) folders from server it changed the / character into ^ (from "z/z" into z^z)
https://i.imgur.com/Jv4Le5J.png
And screen from RoundCube Webmail

https://i.imgur.com/v8S54nW.png
See Also: → 1162486

aaaand? No one tried? No one have used / in folder names? (like: "Reports for years 2018/2019")

Let me ask you some more info that you may have already provided just so I completely understand:

  1. What is your separation delimiter configured in dovecot?
  2. Is tb set up to use maildir format or are you using the default storage format in tb?
  3. When you create folder a/b in tb are you seeing two levels, folder b under gray folder a?
  4. I'm not really familiar with maildir++. I have a dovecot test server. Does it use maildir++? If not, is maildir++ significant for this bug?
  5. When you create a folder a/b with a non-tb client (e.g., roundcube) how does the folder appear in the dovecot maildir filesystem? Is the/escaped somehow? (Linux doesn't allow/in a file or folder name for me.)

I looked at a bug similar to this a few weeks ago and concluded it couldn't be fixed since/is not allowed in dir or filenames on unix or windows. But I could be wrong if escaping the/is possible.

(In reply to cra3y from comment #1)

And second thing: When Thunderbird download (mirror) folders from server it
changed the / character into ^ (from "z/z" into z^z)
https://i.imgur.com/Jv4Le5J.png

I guess tb does this since / not allowed in folder/dir names.

(In reply to cra3y from comment #0)

In linux file system it looks like:
https://i.imgur.com/TpDElLF.png

Is this on dovecot or possibly tb's maildir under /ImapMail in your tb profile, not sure.

(In reply to gene smith from comment #4)

Let me ask you some more info that you may have already provided just so I completely understand:

  1. What is your separation delimiter configured in dovecot?
    this: `
  1. Is tb set up to use maildir format or are you using the default storage format in tb?
    Doesn't matter - the same bug when maildir or berkeley is set
  1. When you create folder a/b in tb are you seeing two levels, folder b under gray folder a?
    yes!
  2. I'm not really familiar with maildir++. I have a dovecot test server. Does it use maildir++? If not, is maildir++ significant for this bug?

I have two servers: one Ubuntu 14.04.6 LTS (dovecot 2.2.9) and second Ubuntu 18.04.2 LTS (dovecot 2.2.33.2).

  1. When you create a folder a/b with a non-tb client (e.g., roundcube) how does the folder appear in the dovecot maildir filesystem? Is the/escaped somehow? (Linux doesn't allow/in a file or folder name for me.)

the creation folder with "a/b" name results: ".a\2fb"

I also checked outlook 2016 and 2019 - only TB has this problem

I looked at a bug similar to this a few weeks ago and concluded it couldn't be fixed since/is not allowed in dir or filenames on unix or windows. But I could be wrong if escaping the/is possible.

So there is listescape plugin enabled in Dovecot to handle ("escape") folders name with "bad" characters on it.
Please read:
https://wiki.dovecot.org/Plugins/Listescape

So there is no Dovecot issue at all (four different IMAP clients works ok: Roundcube, Afterlogic WebMail Lite, Outlook 2016/2019 and old Opera Mail 1.0)

So please do one of two option below:

  1. disallow to use / in folder names in TB
    or
  2. handle "/" by escaping in smilar manner like litescape plugin:
  • in linux based file systems by using: \2fb
  • windows based system by using: %2F
    in folder name and send the real "/" to IMAP and properly configured IMAP server should handle "/" without effort (maybe did some "test" by creating temporary randome dssfsd/fsdfds to check IMAP server capability, before creating real folders).

ah - in first . should be \2f (without "b" in the end)

I will share my working dovecot configs (the important parts) to help fix this issue:

10-mail.conf

namespace inbox {
separator = "`"
inbox = yes
}
mail_plugins = listescape
mail_location = maildir:~/Maildir:INDEX=MEMORY

20-imap.conf

protocol imap {
mail_plugins = $mail_plugins listescape
}

There is example of creating few folders with "bad/banned" name (using Opera Mail) and how these folder are really created by Dovecot (with litescape plugin):

  1. a.b
  2. c/d
  3. ~e
    https://i.imgur.com/UFtBY3N.png

and the result is:
https://i.imgur.com/UXDq4HX.png

PS: TB is able to create and handle 1) and 3) without problem

Thanks for the info. Have you tried non-dovecot servers with opera mail? Do other servers have a similar feature as listescape plugin? I'm wondering if a fix for this bug might only be possible with server dovecot/with listescape?

I checked:
gmail - the gmail doesn't allow to create folders at all.

outlook.com - i tried with outlook.live.com based account and they allow to create c/d, but the result is that they allow to create the a.b and ~e, but c/d result is (screenshot below)

https://i.imgur.com/ffmZuh1.png

https://i.imgur.com/EKkLi9T.png

This is funny, because Outlook (program, not website) allows to create folders with "/" in name via IMAP or directly in MS Exchange environment

I will look for another email vendor to check this thing.

Another funny thing is that TB (Windows version) is able to handle folders w "/" in name, when there are stored in "Local Folders" hierarchy

https://i.imgur.com/ylOPMOQ.png

My proposal for solutions:

  1. the quick fix - doesn't allow to create folders with "/" in IMAP-based hierarchy. It will save my time, when I need to recover (read: change c/d name to something c_d directly on server, when user have "created" c/d by purpose/mistake/wish)

  2. the long-way fix - implement correct TB ability to create folders with "/" in name in IMAP-based hierarchy.

Both solutions are suits my needs.

(In reply to cra3y from comment #14)

Another funny thing is that TB (Windows version) is able to handle folders w "/" in name, when there are stored in "Local Folders" hierarchy

https://i.imgur.com/ylOPMOQ.png

Works on linux too in local folders: create folder c/d produces files c1f9e39fd and
c1f9e39fd.msf and displays as c/d. Not sure what the "escape" values in the middle mean.

My proposal for solutions:

  1. the quick fix - doesn't allow to create folders with "/" in IMAP-based hierarchy. It will save my time, when I need to recover (read: change c/d name to something c_d directly on server, when user have "created" c/d by purpose/mistake/wish)

This changes the "normal" behavior of tb. Some users may expect create folder "a/b/c" to produce 3 folders and not one folder a_b_c. However, I don't see this as a big problem.

  1. the long-way fix - implement correct TB ability to create folders with "/" in name in IMAP-based hierarchy.

Maybe only possible with servers that support listescape-like functionality?

Both solutions are suits my needs.

I will look closer into this problem.

(In reply to gene smith from comment #15)

This changes the "normal" behavior of tb. Some users may expect create folder "a/b/c" to produce 3 folders and not one folder a_b_c. However, I don't see this as a big problem.

I don't think so about "default" expectation to create c in b in a subfolder in on shot using "a/b/c"

Now I'm providing support to 3-4 different firms/companies (two polish, two english) and in 99% when user want to use "/" is not to create folder in subfolder but to create folder with some kind of date in name (for example: "Orders 2019/04" or "Invoices 2018/2019") or to create folder with name of few companies in one (for example: "Intermarche/Bricomarche")

I have always heard, why several folder are created instead and if I can fix this, then I convince them to use "", but they don't like it :-)

  1. the long-way fix - implement correct TB ability to create folders with "/" in name in IMAP-based hierarchy.

Maybe only possible with servers that support listescape-like functionality?

I don't think so, because I had some experience with few conversions from Outlook/Exchange to Dovecot/TB and there were always complains form users like that "in old good outlook folders with "/" works, now not working, you sucks, fix this"

I will look closer into this problem.

Thank You :-)

hmm - the comment section changed \ to nothing

it should be: .... I convince them to use " \ " ... (instead of: I convince them to use "")

If You want to add ability to create and handle folders with / in name (in TB IMAP) and left the "default" behavior of creating folders in subfolders in one shot (for example a/b) then I have two proposals

  1. left the "default" behavior and add ability to create and handle folders with / when
    mail.imap.folders_with_slash_support is set to true

or

  1. add an option for this in IMAP server configuration in this window:
    https://i.imgur.com/tKJtJRJ.png

something like: "IMAP server supports slash (/) in folder name" tick/checkbox

and everyone will be happy :-)

And another funny thing:

The "default behavior" in Local folders is to use / in folder names (as desired) and the "default behavior" in IMAP folders is to use / to create folders hierarchy in one shot.

Let try to explain to typical, plain user the two default behaviors and why there is no only one :-)

I see your point but, unfortunately imap is not the same as local folders. I looked at the local folder code and if it sees a char in the user's folder name that is invalid for filesystems it hashes the filename into a valid hexidecimal string. This is fine for local folder since only the local tb user will ever see the folder so it will be decoded back to its original form (containing / or whatever) when listed and displayed in tb.

With imap, when you "create" a mailbox (folder) a / and other chars in the name are invalid. Tb could probably also hash the user's name into the hex string and use that now valid name in the imap command. Then when tb sees a "hashed" folder name it could decode it back to the string containing / and display that name. However, with imap, the idea is to allow more than one client to access the mailbox. So if another non-tb client sees the mailbox with the hashed string, how would it know that it is hashed and how to decode it? It would probably just display the hashed name, e.g., c1f9e39fd instead of c/d.

To do this with imap, a new database attribute "folder name created by tb is hashed" would be needed so tb knows to decode the string before displaying it. Otherwise, the same code used for local folders could be ported to imap code. But sure this would not be trivial.

Also, a pref to allow this new behavior would be needed, similar to what you propose.

(In reply to gene smith from comment #21)

With imap, when you "create" a mailbox (folder) a / and other chars in the name are invalid. Tb could probably also hash the user's name into the hex string and use that now valid name in the imap command.

No. I will give an example.

TB wants to create a.b and it send this request to IMAP (dovecot) - the . (dot) is invalid char, because Maildir using it as folder separator (this is a main reason, why listescape plugin was created for Dovecot long time ago).
Without listescape plugin, dovecot (with Maildir backend) will create a/b.

With mbox backend it will create a.b folder without problem.

TB didn't know, that the dot is forbidden character.

The same with ~ (it points to user home dir, for example ~user).

So when TB wants to create ~bla folder it sends ~bla via IMAP to dovecot and with listescape plugin it is escaped to \7e, so ~bla will create \7ebla folder.

Please do the same with slash (/) like You did the same with backslash ().

The properly configured IMAP service will escape bad characters by itself. This is a reason, why Outlook/Roundcube/Opera Mail works - it sending "unhashed" request with "forbidden" chars and dovecot (with listescape) will do the rest.

The only forbidden char is one that You configure by "separator" in dovecot confing

Please make this possible by allowing user to enable this by using "Config Editor" and setting mail.imap.folders_with_slash_support to true (for example).

And let the "default" behavior (by creating folder and subfolders in one shot by using / ) as is (when mail.imap.folders_with_slash_support is not set or set to false)

So you are saying you would be happy if it worked as expected with listescape? Tb does currently with . or ~ in the name but doesn't with / in the name. That might be easier to fix...

Bug 1499257. Would listescape also fix this bug with as-is tb?

No, because TB make folder and subfolder when / is in folder name (You want for example: 2017/2018 and TB will create 2017 folder and 2018 subfolder for some unknown reason to me, even when listescape is enable and different folder separator is in use)

This behavior is (in 99% certainty) hardcoded in TB

One more thing - TB has Maildir support (perhaps in beta stage) and I have read, that there is a plan to switch from berkeley storage format to Maildir for mails/messages that are stored in disc (locally).

How will You preserve Local Folders current ability to create folders with name like:
a.b
c/d
e\f
~g

?

You will face the same problems with . / and \ like accessing/handling folders via IMAP.

Be smart and try to create one folder-naming mechanism that will be compatible with IMAP (Maildir based and mbox based) and Maildir in local (disc) storage for Linux/Windows filesystems.

I will be happy if:

  1. You will deny to use / in folder name, when deal to IMAP-based folders (configurable via Config Editor)

or

  1. Allow to use / in folder name, when deal to IMAP-based folders (disable creating folder and subfolder instead - configurable via Config Editor)

Let the default behavior left for others and let interested ones to configure for their needs.

(In reply to gene smith from comment #24)

So you are saying you would be happy if it worked as expected with listescape? Tb does currently with . or ~ in the name but doesn't with / in the name. That might be easier to fix...

Yes - exactly that!

(In reply to cra3y from comment #29)

(In reply to gene smith from comment #24)

So you are saying you would be happy if it worked as expected with listescape? Tb does currently with . or ~ in the name but doesn't with / in the name. That might be easier to fix...

Yes - exactly that!

OK, I will take a look at this. I haven't yet tried to enable listescape with my dovecot server but I will. Thanks for the inputs.

(In reply to cra3y from comment #27)

One more thing - TB has Maildir support (perhaps in beta stage) and I have read, that there is a plan to switch from berkeley storage format to Maildir for mails/messages that are stored in disc (locally).

I haven't heard of a plan to completely switch. I have maildir tb running on another computer and, for me, it seems to work OK. I think the advantage of maildir over the default (Berkeley) is that each email is its own file so backup is faster (e.g., no HUGE Inbox file to save every day). Probably other advantages too.

Not sure if maildir format applies to Local Folders. Maybe it does, I'd have to check.

Ok, I found a simple solution for including/ in the folder name. First I enable the listescape plugin in dovecot and set the delimiter to`like you already have. Under Inbox I have this hierarchy:

Inbox
  7

I can click on Inbox and create a folder7/gene^smithand it appears under 7:

Inbox
  7
    gene/smith

(Clicking on 7 and creating foldergene^smithalso works, of course.)

Looking in dovecot's ~/Maildir it created the directory.7.gene\2fsmith and I am able to copy messages to it using tb.

Looking in tb profile under.thunderbird/<profile>/<accountName>/ImapMail/Inbox.sdb/7.sdbI see filesgene^smithandgene^smith.msf.

Looking with wireshark I see that the imap command sent to create the mailbox is:

57 create "INBOX`7`gene/smith"

Hopefully this is an acceptable solution that your customers can live with.

(In reply to cra3y from comment #3)

(like: "Reports for years 2018/2019")

Now if I create folderReports for years 2018^2019 you get folderReports for years 2018/2019!!!

Note: If you really want a^in the folder name, just enter two:^^.

I just noticed that if you rename a folder containing a/or^, you will see errors pop up and the folder names will show with the / or ^ opposite of what you want. The solution to this is to collapse the folder tree at the account level and then re-expand it. This will usually cause imap to list the folders again with the correct names and fix the problem. This always worked for me, but you may need to restart tb if this doesn't fix the names to what you renamed them to.

This seems to be only a UI problem in tb. The correct imap rename commands are sent via imap and the folder names change correctly in both dovecot's ~/Maildir and in tb's ImapMail/ profile.

One more thing: If you create or rename a folder to have a ^ in the name (using two ^'s of course) the name always displays with what looks like a null termination box (a small square with 4 zeros inside) at the end. Again, filenames in dovecot and tb profile are OK so this is a UI issue (and collapse/expand of tree doesn't get rid of it).

(In reply to gene smith from comment #32)

Ok, I found a simple solution for including/ in the folder name. First I enable the listescape plugin in dovecot and set the delimiter to`like you already have. Under Inbox I have this hierarchy:

Inbox
  7

I can click on Inbox and create a folder7/gene^smithand it appears under 7:

Inbox
  7
    gene/smith

(Clicking on 7 and creating foldergene^smithalso works, of course.)

Looking in dovecot's ~/Maildir it created the directory.7.gene\2fsmith and I am able to copy messages to it using tb.

Looking in tb profile under.thunderbird/<profile>/<accountName>/ImapMail/Inbox.sdb/7.sdbI see filesgene^smithandgene^smith.msf.

This is a workaround that I have used for myself and for some experienced users. The typical TB user (non geek) doesn't want to use "hacks and tricks" to create dirs with / . Please consider this

So you knew this was possible all along? It would have been nice if you had told me that. Or maybe I missed it in our previous discussions? All I see is in comment 1 something about "mirroring" the / back to ^.

I know that, but I didn't mentioned that, because this is not relevant to this issue.

Imagine You're working in company with 300+ users. 95% of them are casual users (not IT nor tech geeks). The do their job (accountants, sales representative, personal assistants etc.)
Most of them have use PC with TB (only CEO + top management having use Macs and Mail app).

There is randomly 2 to 10 persons in a month that need to create folder with / in TB. You tried to teach them to use ^ hack in TB. Ok. IT works for 2-3 weeks, because that random person have needed to create one/two folder with / in one time, then have forgotten this "trick", because he/she doesn't needed to create folder with / in next 1-3 weeks.

After 2-4 weeks he/she called You back and complaining that she/he is unable to create folder with / in name ....

And this goes on and on ...

Yes, I see your point. I keep a notebook of tricks to refer back to but often don't write it down and have to research it again.

When your users create a folder are they normally at one level above where they want to create it? What I mean is in my example when I create folder "gene/smith" I am selected on folder 7. That way I enter just "gene/smith". But if I am selected on folder Inbox, I would have to enter "7/gene/smith" to create something under folder 7.

In both cases the imap command sent would, with release tb, be this:

666 create "Inbox`7`gene`smith"

which would of course create new folder gene and subfolder smith under 7.

So if the preference option is to always keep the user entered /'s in the folder name to create, if you are selected on folder 7 it is ok and the imap command sent will be:

666 create "Inbox`7`gene/smith"

But if you are selected on Inbox and create the folder with string 7/gene/smith you will send imap command:

666 create "Inbox`7/gene/smith"

which will create a new single folder under Inbox called "7/gene/smith" which may or may not be expected. So this option would assume the user is creating the new folder containing slash from right above where the new folder is to appear. That's typically how I would create a new folder.

Why not expected?

Replace: 7/gene/smith to 2019/04/14 and user expected to create folder which name is "2019/04/14".

Remember - this fix/feature is for plain users with plain needs (wanting to create folders with / char, because the're loving do that)

Sure, but my point is that folder 7 already exists. In your example folder 2019 probably doesn't already exist. Anyhow, I will try to implement this and check that it really works when I am on my development system. Hopefully not too difficult :).

Please convince me, that You have seen plain user (not geek or IT guy) that using / to create subfoders

For example:
Inbox
-----> 2019

User choosed main folder (Inbox), then New Folder and entered "2019/blablabla-new-folder" in form.

I've never ever seen plain user do these things like this.

But I have seen this structure (created via webmail)

Inbox
------> 2018
--------------> Proposal
--------------> Finished/Completed
------> 2018/Q1
------> 2018/Q2
------> 2018/Q3
------> 2018/Q4

Yes - the / is not folder separator there :-) This is of one of the weirdness folder structure, that I've ever seen.

(In reply to cra3y from comment #41)

Please convince me, that You have seen plain user (not geek or IT guy) that using / to create subfoders

For example:
Inbox
-----> 2019

User choosed main folder (Inbox), then New Folder and entered "2019/blablabla-new-folder" in form.

I've never ever seen plain user do these things like this.

Not sure I know of a "plain" user :). Anyhow, who knows what someone might try to do even if they are a plain user. I just created the folder "30/30" under already existingInbox`7`gene/smith`20/20 while selected on Inbox with this command string in the create folder box:

7/gene^smith/20^20/30^30

Of course, the plain user would probably just select the 20/20 sub-folder and enter 30^30 into the box. (Or with your proposed fix, 30/30).

Still need to look closer at the code and see how difficult this is to do.

(In reply to Alfred Peters from comment #43)

See also: Bug 773579

Not quite the same as this bug helpful info. Thanks. There is also a meta bug that lists many similar problem with unusual chars in folder/mailbox names: Bug 124287. I need to look through these too.

Reporter Mr. c:

I've made some progress on this but not there yet. You can now enter a folder name, e.g., guitar/case and it creates a single new folder guitar/case. You can copy emails to it and open them and delete them, etc. However, if an email has an attachment the internal tb url is somehow messed up and it hangs. Apparently the format of the internal url is wrong and confuses tb. Don't know why as of now.

Edit: Also, you can create the foldera/bwith another client and tb sees it OK on the main tb page. But any folder name containing a slash appears with a ^ in the subscribe screen, e.g., guitar^case and a^b are shown in the list of folders to subscribe.

It looks like that some of TB modules (IMAP, dir listing tree etc.) are incompatible each other in this matter.

Blocks: 1547429

Been working on this off and on for a while now. A problem now seems to be that when I restart tb, any folder with a slash in its name is deleted in tb and re-created and re-downloaded. This is all just internal to tb and the folder (mailbox) on the server is not deleted when tb starts up but tb re-fetches all the emails in the folder again since it thinks it's new. I'm sure the escaping for slash is the problem but don't know why.

See Also: → 1568845

Sometime In the past week, I renamed a tb folder to include slashes. Weird stuff ensued, and I ended up losing subdirectories and saved emails which I couldn't recover. I would suggest at least a band-aid to disallow slashes and other characters that the underlying OS file system can't handle, and if you want to revisit it later and support such characters, that's fine too.

'subdirectories' -> 'subfolders', for clarity.

Hi Mr. cra3y, Sorry it's been so long getting back to this. I now have a possible solution that should work for you so you can enter a slash when you create a folder and any existing folders with a slash should appear OK. If you are still interested I would appreciate it if you would test the "try" build (a patched version of tb 68.9.1) that I have here: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=0a4434686a167b3313e5e0ab0fbc7b73578c2e8e. Click the green B next to linux or windows (I built both since I wasn't sure which you use) and down below you will see Job Details. For window you will find a target.installer.exe or for linux you can just use target.tar.bz2 after unpacking then then just run the internal thunderbird executable.

This still assumes that your server's hierarchy delimiter is something other than slash. I've tested it on Dovecot with '.' as the delimiter but your use of '`' should also work. It also assume that the Dovecot listescape functionality is present in the server. (I don't know if any but Dovecot support this. Do you?)

So with this you should be able to put slashes '/' at any place in the folder name; multiple consecutive slashes will be auto-changed to a single slash.
You can also enter a carets ^ at any place in the folder name; multiple consecutive carets will be auto-changed to a single caret.
If your folder name contains "^/" or "/^" the caret will be changed to -, e.g., these are auto-changed to "-/" or "/-".

Here's an example of a folder name that works with this patch: point^er/yahoo/-com/

These rules apply to creating or re-naming a folder and are needed to avoid big internal tb changes due to tb "escaping" each slash to a single caret and a caret "escaped" to two consecutive carets (but only when the delimiter is other than slash). Also, for now, it avoids adding error messages for invalid folder name content. Note: Apparently usage of slash and/or caret was never tested or no server supported the "listescape" functionality when the code was originally written many years ago.

Hi

I will try tomorrow and will post result

(In reply to stevem from comment #48)

Sometime In the past week, I renamed a tb folder to include slashes. Weird stuff ensued, and I ended up losing subdirectories and saved emails which I couldn't recover. I would suggest at least a band-aid to disallow slashes and other characters that the underlying OS file system can't handle, and if you want to revisit it later and support such characters, that's fine too.

I assume this was an imap account? If so, all imap servers in the wild that I have observed use slash as the "hierarchy delimiter" unless explicitly changed to something else like this bug's reporter "cra3y" has done.

Anyhow, I just tried to rename an imap folder to contain a slash with a typical default imap server (with slash as delimiter) and got an error message: The / character is reserved on this imap server. Please choose another name. I'm not sure when this error message was added to tb but maybe it was after "sometime in the past" when the problem you report occurred?

But if you still see this problem (try it on a temporary and expendable folder) it should probably be entered as a different bug. Also, a different bug if you are not referring to an imap folder.

My issue is related and that's why I brought it up inside this report assuming it might help investigators for the IMAP folder issue.

I'm talking about additional folders one can create to capture filtered mail, which come from a POP server in my case. I couldn't use a / in the folder name without the problems I mentioned happening.

Hopefully this is already has a report for it, as it seems odd I'd be the first to discover it. I'll open a new report if I can figure out a good way to explain how to replicate it.

OK, I now see this is a nearly two-decade-old set of issues, re #124287. No new report necessary. I guess my search last time wasn't extensive enough.

Attached patch Allow-slash-in-folder-name.diff (obsolete) — — Splinter Review

For reference, this is the diff against recent TRUNK for the changes in the "try" build of comment 50. (The try build of comment 50 is 68.9.1 with this diff applied.)

(In reply to stevem from comment #54)

OK, I now see this is a nearly two-decade-old set of issues, re #124287. No new report necessary. I guess my search last time wasn't extensive enough.

Ok, thanks. So above you are referring to the "meta" bug 124287 (FYI, prefixing with "bug" makes a link) that lists many other bugs. Do you know which in the list corresponds to your bug?

Also, FWIW, you can currently name or re-name Local Folders to contain a slash with no problem. I expect you can do the same with POP folder (since they are essentially local) but haven't tried. No idea about "search" folders. Anyhow, for this bug I am focusing on just imap folders.

Assignee: nobody → gds
Status: UNCONFIRMED → NEW
Ever confirmed: true

Re: "FWIW, you can currently name or re-name Local Folders to contain a slash with no problem", no I can't as of a few weeks ago (I'm on Windows 10 so maybe that makes a difference). It led to the problems I stated and I'm not doing that again unless I see it was specifically fixed.

Re bug 124287, I browsed the list quickly and saw that maybe one of them applied to what I saw. I don't see the point of adding more comments to the fray.

stevem, I just tried on win7 (AFAIK, same filesystem as win10) with tb 68.9 and I created the folder gene/smith and then renamed it to gene/d/smith and saw no immediate problem. This is under the default "Local Folders" that show near the bottom of the folder pane.
As a test, you can create the local folder called "temp-test", copy something to it and then rename it to "temp/test". This shouldn't cause any problem with your normal folders I would think.

The "Local Folders" are different animals (stored locally only and works well).

The problem was in IMAP folders

@gene smith, I can confirm, that the "try" build (a patched version of tb 68.9.1) works flawlessly !!!
https://i.imgur.com/1QnwSDj.png
https://i.imgur.com/24odYCI.png

BIG, BIG!! Thank You :-)

(In reply to gene smith from comment #57)

stevem, I just tried on win7 (AFAIK, same filesystem as win10) with tb 68.9 and I created the folder gene/smith and then renamed it to gene/d/smith and saw no immediate problem. This is under the default "Local Folders" that show near the bottom of the folder pane.
As a test, you can create the local folder called "temp-test", copy something to it and then rename it to "temp/test". This shouldn't cause any problem with your normal folders I would think.

My test was a few weeks ago when I lost emails, subfolders and message filters.

(In reply to cra3y from comment #58)

@gene smith, I can confirm, that the "try" build (a patched version of tb 68.9.1) works flawlessly !!!
https://i.imgur.com/1QnwSDj.png
https://i.imgur.com/24odYCI.png

BIG, BIG!! Thank You :-)

Thanks for trying it and glad it works. I have since detected some problems. Tb puts out a lot of errors to the console when a message inside the folder names with slash and/or caret is accessed. Something about "can't get header". Also, I see a crash when I put a space in the folder name and then access its subfolder. So I need to look some more at this.

I've made some more fixes and improvements but still need to take out tons of printf's and other debug related stuff so it can be reviewed. But for now, I'm just posting my current diff for archival purposes. Mr. Cra3y: There is very little functional change from the "try" build.
(Magnus, if your're reading this, I'll finish up the UTF7=ACCEPT related patch tomorrow. Sorry for the delay. There is some overlap with this.)

Attachment #9159032 - Attachment is obsolete: true

The fix for this bug requires some of the changes that have landed for UTF8=ACCEPT feature addition (Bug 1571672). This diff is a subset of the v2 diff but with the changes common to dependent Bug 1571672 removed. This still needs a bit of clean-up before submitting as an official patch.
This is saved mainly for documentation purposes.

Depends on: 1571672

Same as previous ...-v3.diff but cleaned up. Seems to work but not ready for review. Feedback welcome of course.

Attachment #9160579 - Attachment is obsolete: true
Attachment #9180868 - Attachment is obsolete: true

Gene, are there still road blocks?

Flags: needinfo?(gds)
See Also: → 1247379

I'm inclined to let this one drop (WONTFIX). Back when it was reported we found we had to have a special "plug-in" enabled on the dovecot server for a slash in folder names to be possible. So it only works on that one server type with a special setup. So unless there is more popular demand or if the reporter strongly objects, let just mark this as resolved/wontfix.
Also, I fear the fix may touch a lot of unexpected stuff and end up being a can of worms like UTF8=ACCEPT.

Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(gds)
Resolution: --- → WONTFIX

It is not only for dovecot, but for Gmail's IMAP too (bug 1734938).

I think, that this case like that (/) will be posted here from time to time.

Anyway, thank You for your effort.

I'm fine with the slash not being allowed, but at least prevent users from doing it. Give us a basic guard rail.

Oh yes - preventing creation folder with "/" is good idea.

Also a warning should be shown when remote gmail has already folders/labels containing slash to offer the user an explanation why everything is messed up and so that he can fix it. I remember spending days to figure out what the hell was wrong with my account until I realized it was the slash. The average Joe user DOES NOT know how to do troubleshooting, he just uninstalls the software if it ain't working..

I just created the label 3/3/3 at gmail.com. In tb it appears as three nested folders named 3 with only the bottom 3 non-gray and able to hold messages. When tb "discovered" the new gmail folders, gmail imap (per IMAP:5 log) just reports the folder as 3/3/3 so, since gmail's imap hierarchy delimiter is a slash, tb has no way of knowing this isn't 3 nested folders. So I don't see a way to fix this since gmail itself doesn't "escape" the slashes it sends to tb either.

If you create the folder 4/4/4 in tb it appears nested in gmail account. Gmail.com creates and you see there a new single label 4/4/4 with no nesting, so the behavior is consistent in both directions. Maybe not exactly what everyone wants, but consistent.

A possible workaround for this with gmail is to create the folder name (in tb or at gmail.com) with a non-US-ascii unicode slash: e.g., https://www.compart.com/en/unicode/U+2215. Since gmail supports the UTF8=ACCEPT imap capability, it does work. So now 3∕3∕3 is a single folder in tb and a label at gmail.com. (Support for UTF8=ACCEPT appeared starting with tb 91 but requires matching support in the imap server.)

Gene, how to enable UTF8=ACCEPT in TB91? I want to test with dovecot's IMAP (by creating folders with / within name)

(In reply to cra3y from comment #72)

Gene, how to enable UTF8=ACCEPT in TB91? I want to test with dovecot's IMAP (by creating folders with / within name)

Unfortunately, dovecot doesn't support it. It's automatically supported in 91 if the server supports it. Gmail and Courier, among others, support it but dovecot doesn't, AFAIK.

If you see this set to true in config editor, then it's enabled: mail.server.default.allow_utf8_accept

(In reply to gene smith from comment #66)

I'm inclined to let this one drop (WONTFIX).

I can see that as a reasonable solution, if we actively prevent the user from trying to create / rename a folder with these characters. Currently, we still see folk with "lost" mail because they have incorporated various odd things into folder names the / being quite popular. But somewhere along the line it would be good to publish a list of prohibited characters for folder names.

Hold on, actually you CAN create a folder name with the unicode slash (or most any unicode char) in dovecot. It just doesn't get sent out as a true UTF-8 but as a MUTF-7 ascii string like this: SendData: 44 create "threee.a&IhU-b" to produce folder threee.a∕b (created a subfolder of folder threee named a∕b).

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: