Open Bug 148460 Opened 22 years ago Updated 1 year ago
Need new canonical format for IMAP folders, should include information on the namespace of the folder
While working for fixes for bugs #27002 (about making special folders work on cyrus and courier IMAP servers) and #105385 (server directory for these servers) and seeing the mess between the two different concepts of namespaces and server directories, it has become clear in my opinion and that of a few others that to properly fix this, the internal representation - the canonical format in which folder names are stored and are used to build the tree and files on the disk, needs to be reworked to somehow include information on the namespace of the folder. This bug is for discussion on how this can be done.
I will start with my idea: I have been thinking of this for some time now and have asked people to send me NAMESPACE responses from a lot of different IMAP servers they use. From this and earlier ideas which I had in bug #27002, this is my current plan on who I would intend to implement this: The IMAP NAMESPACE command returns namespaces for private, shared and public folders. We should make use of this and have a format like: /<namespacetype>/<namespace>/<foldername> where namespacetype is either "personal", "shared" or "public". This would give a nice representation in the tree as it would group all personal, all shared and all public folders. For IMAP servers that do not publicise a namespace, I would use a neutral string like "folders". The only problem I see is for personal namespaces that are "". For this, the resulting canonical namespace would be "/personal//Sent", which is probably not good. The empty namespace somehow needs to be escaped and then not shown in the tree. Perhaps we can do this by using another namespace type string, for example "personal%" or so. I would also leave the prefs for the 3 different types of namespaces, give them a new role though. Their new role would be to filterthe namespace you are interested in for each of the hierarchies. This would turn their functionality into what today the "server directory" does today. Now to comment on another idea. Henry Jia commented in bug #27002: "4. About the canonical format of the directories, I think we can use the format "<namespace>//<directory>/<directory>...". This can give us flexibility for namespace handling in the future without any information losing. P.S. the delimiter between directories and namespace is '//' instead of '/'." I now agree that the namespace should be part of the canonical format of the directories as you see in my proposal. I do not like the use of // as a delimiter. // is normally only used to destinguish between the URL and the actual protocol. Also, many operations seperate the strings into substrings using / as the delimiter. This would cause an empty string. This could cause problems building the folder tree.
Adding myself to track this issue.
Is there really a need to track whether a namespace is personal, shared, or public? It doesn't really make much difference, does it? I'll try and find some time soon to mock up the UI I was thinking of for handling namespaces better. In general I certainly agree that the current situation is messy and needs to be cleaned up, and that Server Directory and Namespace are to concepts that need to be consolidated (indeed, the IMAP RFC has no concept of "Server Directory").
Adding the type of namespace would give a really nice tree in my opinion. And if we only have one namespace under a namespace type, like I said, I would set the "server directory" for this namespace automatically. This way, all personal folders regardless of server typ would appear maybe under "Personal folders" (coded as /personal), all shared folders (if exists) under "Shared folders" (coded as /shared) and so on. The tree will look the same no matter what type of server you are using. Namespace type necessary? No, not really. Also if we have the namespace we can actually find out what type of namespace it is pretty simply. The reason I want to have it is just to construct a nicer tree and together with the "server directory" (set automatically) for each namespace, then the tree will look the same no matter what IMAP server you are using. I know there is no server director defined in any RFC. But is it a bad thing? The reason I still want it is so that in 99% of all cases we can have /personal/Sent as a folder no matter what type of server we are using. /personal/Sent would be expanded to "/personal/INBOX/Sent" because of the (99% automatically) set Server directory (for this type of namespace), and this would be "expanded" to "INBOX.Sent". I may hack up some code for this next week to illustrate what I mean.
Change the name in my posts of "server directories" to "namespace selection" and you get more what I mean. They are used to specify which namespace(s) we are interested in. If there is only one, then the namespace is removed from the canonical folder string for this type of namespace. To do this absolutely requires having the type of namespace in the canonical diectory name so we can do a reconstruction. For this, the GUI could be left as it is (although we could and should remove the server directory). Call this "namespace selection", keep them in server "namespace" and then we are back to where the RFC is and we have removed the concept sort of of a server directory totally. What I aim at is the following: 1. automatic configuration but manual possible 2. canonical directory names the same on just about every type of server (not possible e.g. when we have multiple personal namespaces. Currently not really supported at all, but this concept will support them). 2 is the reason canonical names are used at all. I know I am developing this idea from post to post. I now think this is an idea we could work with. I like thinking things out before I start coding. Thoughts, suggestions, is this all total b&ll s&%t? I'd like to start coding this, but I won't do so until I've heard some feedback. This is actually something I could implement pretty quickly. I know where to make the changes to support this.
I think we can use the canonical format like this "/namespace/folder" for an IMAP folder. And thus "imap://username@host/namespace/folder" for an IMAP URI. Now, I agree with Michael Klose to not to use "//" between namespace and folder. It's mainly for a protocol. We can just use "/". I think we can omit the "namespacetype". Because this can be easily gotten if we need it. Marking as new and change to 'Normal'.
Severity: enhancement → normal
Status: UNCONFIRMED → NEW
Ever confirmed: true
NAMESPACE command gives us what the server suports. The only useness of this command is to let us have a record for the server's namespacetype and it's prefs. We can use them later then. Compared with server directory, namespace gives us more info about the server. While server directory is only mannully created and only helps us manage the directory tree more easily.(For example,to show the tree in a different format like.)
Henry, if the IMAP URI were in the format "imap://username@host/namespace/folder" as you mentioned comment#6, what will we do with the problem in bug #71383 (where namespace is "Shared Folders/User/")?
'/' is the cononical seperator, server side's '/' seperator should be changed to '^' when it is translated to cononical format.
Mozilla keep canonical formated IMAP folder name instead of real folder name now. When we set IMAP server directory in advanced server setting, the canonical formated folder name will lost some infomation. For example, the exact heirarchy level of the folder in server and the folder's namespace prefix. Mozilla use canonical formated folder name to display folder pane and recreate the tree structure at client side. This means that when we set IMAP server directory, the structure of folders will be different with the structure in server. This brings us many trouble. When we create a new folder, mozilla again only keep the not-real folder name (which may have lost whole path infomations of the folder) into the folder DB. And because mozilla has used canonical formated folder name, we can't get the realname of folder after mozilla has lsubed the folders. This cause many errers on namespace. Thus, we need two kinds of folder name in mozilla now. 1, canonical folder name which can be used to create and display folder pane. 2, real canonical folder name which can be used to do create, rename, and other intercomunicate with server actions.
Now we know that mozilla will convert the realName of a folder into its canonical formate, then only keeps the 'allocatedPathName' of folder. This means that if a folder 'INBOX.a.b' on server side, when it came to mozilla's side, mozilla will convert and keep only 'a/b'. If I want to keep the realName('INBOX.a.b') of the folder in mozilla and use it later when needed. Where should I change the code? Who can give me some suggestions, Thx!:-)
We sure need to keep namespace prefix in url in order to know which namespace a folder are in!
There has been recent discussion of URI handling, but I don't remember where. Did that include URI format?
(In reply to Wayne Mery (:wsmwk) from comment #13) > There has been recent discussion of URI handling, ... You're speaking riddles. Almost everything in TB is addressed via URLs/URIs: Messages in folders, attachments in messages, protocol access, even via SMTP or POP3. So we're talking about URLs *all the time*. Looks like this bug from 2002 is about changing the format of IMAP URLs. Without reading any of it, that looks like a WONTFIX to me after 15 years on the books. We surely won't rework IMAP URLs now, maybe in TB:NG.
(In reply to Jorg K (GMT+1) from comment #14) > (In reply to Wayne Mery (:wsmwk) from comment #13) > > There has been recent discussion of URI handling, ... > You're speaking riddles. Almost everything in TB is addressed via URLs/URIs: > Messages in folders, attachments in messages, protocol access, even via SMTP > or POP3. So we're talking about URLs *all the time*. Certainly. Sorry not able to be more specific because I don't remember the context of what I was reading. SWAG maybe it was bug 922464. But I have conflated that with IMAP urls. > Looks like this bug from 2002 is about changing the format of IMAP URLs. > Without reading any of it, that looks like a WONTFIX to me after 15 years on > the books. We surely won't rework IMAP URLs now, maybe in TB:NG. Perhaps gene or magnus will have some perspective about the usefulness of this
I think it would be useful in general, but it's not a pressing matter so would need someone to step up to do the work. Doesn't sound like very high gain for what would probably be a lot of work.
Not sure I really understand what the original complaint here is. In my recent work with namespaces, I think the main user complaint is "why doesn't my shared folder appear". This has recently been fixed for at least one user. There are two remaining small bug that I have noticed while working with namespaces. Suppose you have multiple namespaces in a given category, e.g., two "personal namespaces" called INBOX and a-bear and INBOX is the "default" namespace (lists first in the imap namespace command response). New folders appear under INBOX when created in tb at the account level without a prefix. If you look in the advanced server setting under personal namespaces you will see: "a-bear.","INBOX." This kind of implies that a-bear is default and not INBOX. The namespaces that show here should be in the same order as the imap namespace command returns, i.e., "INBOX.","a-bear." The same order reversal occurs for user and shared namespaces if there is more than one of a type. The other very small bug is that the order shown on the advanced server page for "Public (shared)" and "Other Users" is in the opposite order from the actual imap namespace command response. Also, the naming for "Public (shared)" doesn't quite match how it is described in the namespace rfc2342. So, I think the page should show like this to match the order and terminology of the rfc: Personal Namespace: Other Users' Namespace: now: Public (shared): Shared (Public) Namespace: now: Other Users: Of course, if anyone wants these namespace changes it should be written up as a new bug.
You need to log in before you can comment on or make changes to this bug.