533337 - [tagging] Need IMAP based fallback method when IMAP server does not support user defined keywords

Status: RESOLVED WONTFIX

(MailNews Core :: Networking: IMAP, enhancement)

Description

[Daniel Kabs, reporting bugs since 2002]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091130 Thunderbird/3.0

Request: 
User should be able to see and share tags between multiple computers connected to a single account on an IMAP server, no matter whether the server supports keywords or not.

Reasoning:
Thunderbird tries to store tags as IMAP meta data on the server using IMAP keywords. In this case all computers connected to that account can see the tags and work on them.

If the IMAP server does not support IMAP keywords, it stores tags locally in the .msf file for the folder. That means that another PC can not see the tags.

Currently the user has no way to find out if his tags are "shareable" or not.


Reproducible: Always

Comment 1

[Daniel Kabs, reporting bugs since 2002]

TB currently does not tell the user where tags are stored.

On some servers sharing does work, on others it doesn't. From the users point of view this seems like pure chance, whether sharing works or not. 

Sharing tags might seem a less important feature. I think sharing tags will gain importance as tagging gets more popular and virtual folders replace real folders (see google mail).

Comment 2

[Kent James (:rkent)]

I'm not really sure how we would go about doing this. IIRC there are some outstanding proposals in IMAP to allow message metadata to be stored on the server, and we have proposed to implement support for that. But that does not really help the current bug, because if someone does not implement shared flags on their IMAP server, they are also unlikely to implement a rare new protocol to save metadata.

The comment "Currently the user has no way to find out if his tags are "shareable" or not" is valid and could be a viable bug, though frankly I think that is best done in an extension.

So the advice that I would give to anyone with this problem would be to switch to a mail provider that supports shared flags.

Comment 3

[David :Bienvenu]

I agree - if the server doesn't support keywords, I can pretty much guarantee it's not going to support message metadata.

Comment 4

[Ludovic Hirlimann [:Usul]]

WONTFIX per comment 3

Comment 5

[Daniel Kabs, reporting bugs since 2002]

oh...not so fast... :-) Let's work this through. I think I did not get my point across.

You write about "IMAP to allow message metadata". I don't know what this "rare new protocol to save metadata" is about.

I was thinking about tagged messages, i.e. the information that a message has been applied one or more tags -> 

How these tags are represented in UI (name, color, priority) and how this information can be shared across computers is none of this bugs concerns. 


I take it that this bug has been closed based on the assumption that I suggest support for sharing the *complete* tagging information but this is not the point I am trying to make. 

So please let me try my case again. -> Reopening.

Comment 6

[Ludovic Hirlimann [:Usul]]

(In reply to comment #5)
 
> You write about "IMAP to allow message metadata". I don't know what this "rare
> new protocol to save metadata" is about.

Imap is not rare. The issue is that all imap server implementation do not support metadata.
 
> I was thinking about tagged messages, i.e. the information that a message has
> been applied one or more tags -> 

That's what we call metadata.
 
> How these tags are represented in UI (name, color, priority) and how this
> information can be shared across computers is none of this bugs concerns. 

So what is ?

Comment 7

[Daniel Kabs, reporting bugs since 2002]

Ludovic, 
 as I understand it TB tries to store the information that a message has been applied one or more tags  using "IMAP keywords" and not using "IMAP metadata". So maybe the term "metadata" has been used in two different contexts, one being message metadata (e.g. a tag applied to a message) and the other being IMAP metadata (which might be a special term coined in IMAP protocol). However, I am neither an IMAP expert nor do I know the intricacies of TB's tagging mechanism. So please correct me if I am wrong. I gathered information from:
> http://www.ii.com/internet/messaging/imap/isps/#keywords


OK, as I mentioned before, TB uses either of two methods to store tags applied to messages:
a) on the IMAP server 
b) locally in the .msf file

We agreed that the user has no way of knowing which of the two methods TB is using.

However using method b) has ramifications in context of roaming: If the user is accessing his mail account from a different computer, he will not see that tags have been applied to messages - even if the tags applied are just the "hard coded" tags like "important" or "personal".

So I am searching for a way to make tags work across different computers (or TB profiles) even if the IMAP server does not support user defined flags.

Suggestions:
- store tagging information in a separate file and allow for sharing that file.
- store tagging information in a "hidden folder" on the IMAP server (would also ease migration).
- store tagging information by manipulating the subject. 

I don't know what is feasible or not using IMAP.

Comment 8

[Ludovic Hirlimann [:Usul]]

(In reply to comment #7)

> 
> I don't know what is feasible or not using IMAP.

The best way to do that would be via a properly configured imap server. If you end up having X locations where tags are stored, you end up with synch issues between the locations. I understand your users point of view, but for the solution to work it needs to be simple and not over complicated - having the information on the server is the way to go in that case. and to do that server side you need to use imap. 

Now if someone want's to get the trouble of writing a solution that's not imap centered and fits your need, in the form of an extension, that would be very fine by us. But for the core set of features that the thunderbird team is working on , your idea doesn't fit, and that's why I had closed it WONTFIX.

Comment 9

[Daniel Kabs, reporting bugs since 2002]

I'd like to see an IMAP centric solution.

However I checked my IMAP server and PERMANENTFLAGS do not support user-defined flags. I don't think this is a configuration issue - it just does not support arbitrary flags. I am not proficient in IMAP but maybe tags could be stored on the IMAP server in hidden "service" messages?

Anyway, if users have to live with tags in the .ms file, how do they migrate this information to a new profile on another computer (e.g. if the old computer is broken down)? Just copy the whole profile (including the .msf file) over?

Comment 10

[Daniel Kabs, reporting bugs since 2002]

I guess, IMAP flags are the most efficient way to store tags. However, user-defined flags are not available on all IMAP servers. 

Why not store user attributed data in the message header if need be?

There are mail applications that do this already, e.g. MailTags for mail on Mac, see http://indev.ca/MailTags.html, storing tags in X-MailTags headers, see
 http://lists.madduck.net/pipermail/mailtags/2007-August/msg00017.html

TB could make use of MailTags formatted headers and allow for interoperability of the two applications.

Comment 11

[Daniel Kabs, reporting bugs since 2002]

What do you guys think about the ideas brought forward?

I like to see this is as a bug.

If the server does not support user defined IMAP flags...
- tags are only visible on the computer on which the user set the tags
- the user is unaware of that fact. He has no way of telling whether his IMAP server is supporting tags or that he is better off using a different IMAP server (e.g. for corporate use).

Solution #1: Do not *silently* fallback to local storage. Instead as soon as the user uses tagging for the first time, tell him that due to limited IMAP server capabilities tags are stored locally and that tags can not be shared across profiles or accessed on different computers.

Solution #2: Implement the server side solution according to MailTags implementation.

Comment 12

[Daniel Kabs, reporting bugs since 2002]

- per comment #2 moved "Currently the user has no way to find out if his tags are
"shareable" or not" to a new bug #536946 
- changed this bug to enhancement request
- set topic accordingly

Comment 13

[Daniel Kabs, reporting bugs since 2002]

added to Bug 432710 ~ tag tracking bug

Comment 14

[Daniel Kabs, reporting bugs since 2002]

Alternative to IMAP keywords:

 rewrite message to store tag data in message header. 

I guess this could be accomplished the way deleting attachments is done.

Comment 15

[Daniel Kabs, reporting bugs since 2002]

Mails stored on IMAP server can be rewritten. Thunderbird does this e.g. when an attachment is deleted. Not only can the attachment be deleted but also the header is rewritten. 

The same "mechanism" can be uses to store tags in messages on an IMAP server.

Comment 16

[Magnus Melin [:mkmelin]]

This isn't something we'd like to do. If you want to share tags between computers, don't use a crippled imap server that doesn't support tags properly.