Messages with tags fail to copy to account on Exchange server due to "APPEND failed: unknown flag"

RESOLVED WORKSFORME

Status

MailNews Core
Networking: IMAP
--
critical
RESOLVED WORKSFORME
11 years ago
6 years ago

People

(Reporter: godfreye@bigw.org, Assigned: Bienvenu)

Tracking

(Blocks: 1 bug, {dataloss, imap-interop})

1.8 Branch
dataloss, imap-interop

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(5 attachments)

(Reporter)

Description

11 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.10) Gecko/20070228 Camino/1.0.4
Build Identifier: version 2.0.0.0 (20070326)

I have mail accounts on 2 servers in diff. cities, both iMAP, call them A & B.  I back each up on the other.  Starting with TB 2.0 copying messages from A-->B fails.  However, copying B-->A works, copying A-->local folders & local folders-->B also works.  Definite cause: use of tags.  Messages arriving untagged always copy; messages with color-coded tags never do, nor do messages from which I've removed the tag.  New bug in 2.0; does not happen in 1.5.0.8 or earlier.

Reproducible: Always

Steps to Reproduce:
1. Set up folder "Server A Inbox Backup" on mail server B.
2. Select all messages in server A inbox. [also applies to individually selected messages]
3. From control-click popup contextual menu, select Copy To - Server B - Server A Inbox Backup.
4. Copying fails UNLESS message never has had tag assigned.
Actual Results:  
Copying fails with this error message (100% of time):  Alert: The current command did not succeed.  The mail server responded: Protocol error: "Specified set of flags is not valid..."
Experimentation shows: (a) this only happens to messages that have NEVER had been assigned a color-coded tag (selecting "Remove all tags" doesn't fix the failure to copy); (b) only happens going from server A ---> server B, and not vice versa.

Expected Results:  
Mail messages should have copied from folder on Server A to corresponding folder on Server B for use as backup.

I apologize, I'm not a programmer (first bug report I've ever filed here); but I'm an experienced user & I've done as much as I know how to do.  I've isolated the cause: the enhanced "tags" feature in TB 2.0.  I sort incoming mail by assigning color-coded tags; useful feature. I back up mail folders from each server to other so my mail is always available if one goes down; and the operator of server B does not back up e-mail accounts (I found out the hard way). A few messages I get aren't assigned a tag, & those do copy.

You probably need to know the mail server software (I asked the operators):
Server A:  Washington University IMAP daemon that comes with Pine; mail delivery agent is Postfix.
Server B:  Microsoft Exchange 2003.
Finally I still have TB 1.5.0.8 on my iBook; this bug definitely does not exist there (i.e., copying mail messages in both directions works).  Thank you.
(Reporter)

Updated

11 years ago
Keywords: interop, mail1
Version: unspecified → 2.0
(In reply to comment #0)
> Server A:  Washington University IMAP daemon
> Server B:  Microsoft Exchange 2003.
> copying messages from A-->B fails. copying B-->A works.
> (a) this only happens to messages that have NEVER had been assigned a
> color-coded tag(selecting "Remove all tags" doesn't fix the failure to copy); 
> (b) only happens going from server A ---> server B, and not vice versa.

Get imap:5 protocol log, and attach log file(mime-type=text/plain) to this bug thru link of "Add an attachment".
  http://kb.mozillazine.org/Session_logging_for_mail/news
  http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

To godfreye@bigw.org(bug opener):
See Bug 353570 Comment #4 for difference between "Specified set of *flags* is not valid..." by Exchange server and "Messages with *tags* fail to copy" by your bug summary. Anyway, see protocol log first. 
(Reporter)

Comment 3

11 years ago
Created attachment 270471 [details]
Requested imap:5 protocol log file to document bug

See comment in this post.
(Reporter)

Comment 4

11 years ago
Re attachment (see Comment #3):  My son created the necessary log file (he's a programmer by profession; sorry for delay, but I had to wait for him to visit me).

In the attached log file, look for BAD Protocol Error - that's where the bug is
occurring (see my previous description).
Attachment #270471 - Attachment mime type: application/octet-stream → text/plain
Assignee: mscott → bienvenu
Component: Mail Window Front End → Networking: IMAP
Product: Thunderbird → Core
QA Contact: front-end → networking.imap
Version: 2.0 → 1.8 Branch

Comment 5

11 years ago
I can verify this problem.  I started a thread on the MozillaZine support forums, but got no responses:

http://forums.mozillazine.org/viewtopic.php?t=573677&highlight=

I was going to open up a bug report but persistent searching through Bugzilla found this one.

Are there any updates?  If any further information is needed, let me know.
Created attachment 276218 [details]
Part of log relates to the error

To godfreye@bigw.org(bug opener):

Will problem occur in next test too? (indirect copy by Tb 2.0)
(1) Copy the mail to a folder in "Local Folders"
(1-1) Move a mail in the folder to other folder, then move back
(1-2) "Compact Folder" (force write of X-Mozilla-Keys:)
(1-3) View Message Source, and confirm that $Lable4 exists
(2) Copy the mail to the MS Exchange server
Confirming, according to log.
Status: UNCONFIRMED → NEW
Ever confirmed: true
RFC 3501 says following. (http://www.faqs.org/rfcs/rfc3501.html)
According to RFC, I think at least response of "BAD(Protocol Error)" is invalid(RFC violation) when "unable to append a flag($Lable4, called keyword in many cases)".

> 6.3.11. APPEND Command
>(snip)
>   Result:     OK - append completed
>               NO - append error: can't append to that mailbox, error
>                    in flags or date/time or message text
>               BAD - command unknown or arguments invalid
>(snip)
>      If a flag parenthesized list is specified, the flags SHOULD be set
>      in the resulting message; otherwise, the flag list of the
>      resulting message is set to empty by default.  In either case, the
>      Recent flag is also set.

David Bienvenu, what is correct behavior in this case? (both IMAP server&Client) 
(Assignee)

Comment 9

11 years ago
we shouldn't be trying to set flags that the server has told us it can't set - I would say that NO would be the appropriate response, but the RFC leaves itself open to interpretation, since an invalid flag could be considered an invalid argument, I guess. 

But in any case, we should fix this in the client.
Product: Core → MailNews Core

Updated

10 years ago
Blocks: 432710
Severity: minor → critical
Keywords: dataloss
Hardware: PowerPC → All

Updated

6 years ago
OS: Mac OS X → All

Comment 10

6 years ago
Is this related to issue 808533 ?
https://bugzilla.mozilla.org/show_bug.cgi?id=808533

It's been going on for few years now, yet it seems like nobody @the TB team is trying to fix this.

Interoperability of IMAP clients is very important. Hope this could be resolved soon !
(Reporter)

Comment 11

6 years ago
(In reply to popo from comment #10)
> Is this related to issue 808533 ?
> https://bugzilla.mozilla.org/show_bug.cgi?id=808533
> 
> It's been going on for few years now, yet it seems like nobody @the TB team
> is trying to fix this.
> 
> Interoperability of IMAP clients is very important. Hope this could be
> resolved soon !

It must have been fixed. I reported the bug 5 years ago, and forgot about the issue, but haven't run into the problem for many versions of TBird now (I still use the same cross-server method of backing up my mail files described in the original bug report above, and it now works fine). I just assumed it had been fixed some years ago, either on purpose or inadvertently.

Updated

6 years ago
Duplicate of this bug: 808533
@godfreye and @popo: What server/version are you using for the destination server currently?

Updated

6 years ago
Summary: Messages with tags fail to copy to account on Exchange server → Messages with tags fail to copy to account on Exchange server due to "APPEND failed: unknown flag"

Comment 14

6 years ago
Mine says: IMAP4rev1 2000.287

popo
(In reply to popo from comment #14)
> Mine says: IMAP4rev1 2000.287

So I can interpret this as Microsoft Exchange Server 2000?

Comment 16

6 years ago
Why ? This IMAP server is running on an old SUN OS 5.8 server. I guess this is not M$.
(In reply to popo from comment #16)
> Why ?

I have a theory that I would like to confirm.

Servers issue a list of PERMANENTFLAGS like:
>  * OK [PERMANENTFLAGS (\11a shraga \* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags

When you copy/move a message, we get this command:
> 2 append "/home/mail/popo/Archives/Nov" (\Seen $label1) "14-Nov-2012 10:00:04 +0200" {1373}

My theory is that only for some servers, but not all, that if one of the labels in the append command is not one of the permanent flags, then the append command fails.

From the Comment 11 above, it sounds like something in Microsoft Exchange Server changed because nothing was ever fixed in Thunderbird.

Comment 18

6 years ago
Created attachment 682881 [details]
Copy file when user flag is not defined on IMAP server

Comment 19

6 years ago
Created attachment 682882 [details]
Manually creating the my_flag tag, using telnet

Comment 20

6 years ago
Created attachment 682883 [details]
3.log - copying after flag was set manually

Comment 21

6 years ago
For some reason only my attachments were uploaded, but not my response to comment #17:

I did 2 tests, to check your theory:

1 (log file in comment 18):
Copy a message, tagged with a "my_flag" tag from exchange server to imap server.
The copy failed ("append failed...)

2 (log in comment 19):
I created manually the "my_flag" tag in the destination folder in the imap server, using telnet

3) (log file in comment 20):
Copied same message. This time the "my_flag" tag exists both in exchange, and in imap destination folder.
Copy filed with same "append failed..." error

Both times I got the same error, which probably means TB is not aware of the IMAP user flag at all.

popo
(In reply to comment 21)

I am not sure that anything can be done in Thunderbird to fix the problem for this server.

Attachment 682883 [details] shows "my_flag" in PERMANENTFLAGS, so my theory was wrong and following the suggestion in comment 9 does us no good.

It believe the best solution is to upgrade your IMAP server.

Comment 23

6 years ago
Close as INVALID then?
Whiteboard: [invalid?]

Comment 24

6 years ago
David,
I thought that if the flag name and also \* appears in the PERMANENTFLAG section, then this means the server supports user made flags(tags). Isn't this the case here ?

-422581360[e4ff3270]: e49a4800:imap1:A:SendData: 12 select "/home/mail/popo/Archives/Nov"
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=12 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: * 1 EXISTS
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=12 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: * 0 RECENT
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=51 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: * OK [UIDVALIDITY 1353245120] UID validity status
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=38 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: * OK [UIDNEXT 11] Predicted next UID
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=60 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: * FLAGS (my_flag \Answered \Flagged \Deleted \Draft \Seen)
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=93 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: * OK [PERMANENTFLAGS (my_flag \* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags
-422581360[e4ff3270]: ReadNextLine [stream=e4915928 nb=37 needmore=0]
-422581360[e4ff3270]: e49a4800:imap1:A:CreateNewLineFromSocket: 12 OK [READ-WRITE] SELECT completed
popo,

I am afraid that the problem is just that your server is just really old. I found this from 2007: http://objectmix.com/imap/201915-mailutil-transfer-fails-unknown-flag.html

That server then was even a more recent version than yours is now. I do not know what else to say besides upgrade your server.

Comment 26

6 years ago
OK
I tried a newer server (dovecot), and things are working OK. I think this issue can be closed.

Updated

6 years ago
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WORKSFORME
Whiteboard: [invalid?]
You need to log in before you can comment on or make changes to this bug.