692768 - Composed and sent mail not removed from Drafts with UW IMAPd 2007f, due to "OK Unable to re-open mailbox for APPENDUID: Mail/Drafts" response to append

Status: RESOLVED WONTFIX

(MailNews Core :: Networking: IMAP, defect)

Description

[Rob Janssen]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20110928 Firefox/7.0.1 SeaMonkey/2.4.1
Build ID: 20110928161145

Steps to reproduce:

IMAP account connected to UW IMAPd version 2007f (most recent version)
Composing a new message and taking longer than the auto-save interval.
Sending message after it has been auto-saved at least once.


Actual results:

A copy of the message remains in the Drafts folder after sending it.


Expected results:

The copy in Drafts should be deleted.

It works OK with UW IMAPd version 2004c and 2006c  (tested).
There appears to be a change after 2006c that causes this behaviour.
There is nothing particular with the saved Draft, it is unread and it can be manually deleted without problem.

Comment 1

[WADA:World Anti-bad-Duping Agency]

Do you see multiple versions of draft mails by next?
 1. Disable auto-save
 2. Compose a very small mail with no attachment
 3 [review]. Execute "Save", click other than Drafts, click Drafts again
 4. Do step 3 multiple times

If only single draft is always kept by step 3/step 4, problem is possibly next;
  Contention between auto-save and mail send.
  There are known two issues.
  - Auto-save is scheduled while mail sending is in progress
  - Mail send is scheduled while auto-save is in progress
  In this case, one of next can happen;
  - last version by auto-save is not removed, because mail sending
    doesn't know last version generated by auto-save. 
  - delete of previous or final version by auto-save and/or mail send may fail
    due to contention. If auto-save generates last version, two copy may remain.

If UW-IMAP 2007 only issue, possibly CONDSTORE relevant Tb's problem, if CONDSTORE is supported from, or enabled by default from, UW-IMAP 2007.
It may be relevant to UIDPLUS. Some servers don't return UIDPLUS in capability response but return response by UIDPLUS to append command. In this case, some of them don't support "search for specific message-id header" which Tb uses to find previous draft when UIDPLUS is not supported.  

Does Tb request "uid xxx store flags \Deleted" to previous version of draft when new version of draft is saved?

Comment 2

[Rob Janssen]

There is no timing aspect in the problem.
(i.e. it happens every time, after the auto-save has been done at least once)
I have set a 15 minute auto-save interval so it is very apparent that no auto-save is
in progress.  I start compose mode with a 1-line message, wait until it auto-saves
(1 unread message present in Drafts folder) then hit send at least 10 seconds later and the
copy in Drafts remains there.
When the compose is left open for more than the drafts interval, only a single copy
of the mail is kept (auto-save mails do not accumulate).

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to Rob Janssen from comment #2)
> When the compose is left open for more than the drafts interval,
> only a single copy of the mail is kept (auto-save mails do not accumulate).

If mail is not altered since last auto-save(first auto-save in your case), next auto-save is not scheduled. Is this case? Or you modified composing mail after first auto-save?

Because your auto-save interval is 15 minutes, you can do manual "Save" multiple times during first 15 minute after start of composition without auto-save. Itentional/explicit "Save" doesn't require modification of mail, but change suffix of subject, like test-001 to test-002, to see latest content of saved draft mail.

Do you see multiple mails in Drafts by multiple manual "Save" within 15 minute?
If yes, can you get IMAP log for it? (timestamp,imap:5)
If no, can you get IMAP & SMTP log for next? (timestamp,imap:5,smtp:5)
  Compose a mail, wait for first auto-save,
  send mail after 10 seconds or more after the first auto-save.

Comment 4

[Rob Janssen]

Attachment: Log of manual save with UW IMAPd 2007f

Comment 5

[Rob Janssen]

Attachment: Log of manual save with UW IMAPd 2004c

Comment 6

[Rob Janssen]

(In reply to WADA from comment #3)

> Do you see multiple mails in Drafts by multiple manual "Save" within 15
> minute?
> If yes, can you get IMAP log for it? (timestamp,imap:5)

Yes.  When I do multiple saves of the message, no matter if I change the subject or
not, I see two copies of the saved message in Drafts.
Interestingly, there always are two copies also when I have saved the third one.  It
deletes the oldest one and puts a new one.

But when using imapd 2004c, I see only a single copy of the message, also when changing
the subject.

I have uploaded 2 logfiles of those tests, hopefully you can see what is going wrong.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to Rob Janssen from comment #4)
> Log of manual save with UW IMAPd 2007f

(A) UIDPLUS, APPENDUID is enabled 
> 2 OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ IDLE UIDPLUS NAMESPACE CHILDREN MAILBOX-REFERRALS BINARY UNSELECT ESEARCH WITHIN SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND] User h.mulder authenticated
(B) UID=9, UID=10 in Mail/Drafts
> 3 select "Mail/Drafts"
> * OK [UIDNEXT 11] Predicted next UID
> 3 OK [READ-WRITE] SELECT completed
> 4 UID fetch 1:* (FLAGS)
> * 1 FETCH (UID 9 FLAGS (\Draft))
> * 2 FETCH (UID 10 FLAGS (\Draft))
> 4 OK UID FETCH completed
(C) First draft save(UID=11). UID is not returned from server.
> 5 append "Mail/Drafts" (\Draft) {2253+}
> 5 OK Unable to re-open mailbox for APPENDUID: Mail/Drafts
(D) Second draft save(UID=12). UID is not returned from server.
> 11 append "Mail/Drafts" (\Draft) {2255+}
> 11 OK Unable to re-open mailbox for APPENDUID: Mail/Drafts
(E) Tb requests delete of UID=10, instead of expected UID=11.
> 13 uid store 10 +Flags (\Seen \Deleted)
> 13 OK UID STORE completed

Is your UW IMAPd 2007f configured properly?

Tb looks confused by bad "OK Unable to re-open mailbox for APPENDUID: Mail/Drafts".
Tb is better not to request delete when UID is not returned correctly.

Comment 8

[Rob Janssen]

(In reply to WADA from comment #7)

> Is your UW IMAPd 2007f configured properly?

The creator of UW IMAPd does not believe in configuration.
The documentation insists all the time that there is no configuration file to be used
outside UW and that the /etc/c-client.cf file that is read should not be used and serves
no useful purpose.

So I don't use it.  The IMAPd is just running as compiled by default (with make lnp, i.e.
linux OS with PAM authentication)

Comment 9

[WADA:World Anti-bad-Duping Agency]

FYI.

(C) First draft save(UID=11). UID is not returned from server.
> 5 append "Mail/Drafts" (\Draft) {2253+}
> 5 OK Unable to re-open mailbox for APPENDUID: Mail/Drafts
(D) Second draft save(UID=12). UID is not returned from server.
> 11 append "Mail/Drafts" (\Draft) {2255+}
> 11 OK Unable to re-open mailbox for APPENDUID: Mail/Drafts

Even if server correctly returns "NO" to append, Tb probably goes ahead to "uid xxx store flas \Deleted" step when "append new draft + delete old draft" case, as done in "append detach'ed mail + delete original mail" case(Bug 693353).
If you can see such problem, post comment to that bug or open separate bug for "NO to append of new draft" case, please.

Comment 10

[Rob Janssen]

I updated my home system to openSUSE 13.1 with UW IMAPD 2007e and SeaMonkey 2.24 (Linux) as
distributed, and now I have the same problem on that system.

Comment 11

[Wayne Mery (:wsmwk)]

If I understand correctly, you are now on Thundrebird.
Do you see the same problem when using a current version of Thunderbird?

Comment 12

[Rob Janssen]

No, at home I still use Seamonkey and this problem still occurs on 2.39

Comment 14

[Rob Janssen]

No longer interested in this