1010711 - Draft message handling - keeping/deleting of wrong mails in IMAP account, if badly configured server with UIDPLUS returns OK to append with incorrect UID information(error message text instead)

Status: NEW

(MailNews Core :: Networking: IMAP, defect)

Description

[H. Sickenberg]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0 (Beta/Release)
Build ID: 2014042500

Steps to reproduce:

I do have a Draft (IMAP) folder with around 40 mails in. When composing a new message, this message will be stored in the Draft folder automatically (autosave). When sending the mail, to my understanding (and it did happen this way in the past), the composed message should be removed from the Draft folder.

Maybe it is important to mention that I do access this folder also with a mail client (K9) from my Android phone.


Actual results:

Sometimes the composed message is not removed from the Draft folder but stays (marked as unread) in that folder. Instead the message gets deleted that was the message before the last message in the list (last message is the composed message).


Expected results:

As mentioned above. My expectation would be that the composed message should be removed.

Comment 1

[WADA:World Anti-bad-Duping Agency]

"Save draft mail/delete old draft mail" is;
 1. Compose a mail, auto-save or manuakl save => append mail to Drafts => UID=(A) is assigned at server
 2. auto-save or manual save => append mail to Drafts => UID=(A+1) is assigned at server.
    if UIDPLUS is supported, UID=(A) is known upon append at step 1 => UID A store Flags(\Deleted)
    if UIDPLUS is not supported, UID=(A) is not known upon append at step 1
       SEARCH for old draft by Message-ID => UID=A is returned => UID A store Flags(\Deleted)
 3. Same actions as step 2 is executed for UID=(A+n) and UID=(A+n-1)
 4. Send or Send Later => mail is sent or saved in Outbox
    => UID (A+n) store Flags(\Deleted) for last saved draft mail

"UID of mail" is shown as "Order Received" column value if IMAP folder.
And all above activities is logged by IMAP logging.

Show "Order Received" column, get IMAP log and check IMAP level flow by Text Editor.
  See bug 402793 comment #28.
  Use timestamp,smtp:5,imap:5,MsgCopyService
Because IMAP activity on all accessed IMAP folders is logged by imap:5, get IMAP log with minimum configuration.
  Disable all automatic new mail check, restart Tb with logging enabled,
  go Work Offline, go Work Online, open one IMAP folder only of relevant IMAP account only,
  compose a mail, save as draft, Send Later.

> keeping/deleting of wrong mails in IMAP account

Which draft mail of what UID was wrongly deleted by Thunderbird?

Comment 2

[H. Sickenberg]

(In reply to WADA from comment #1)
> "Save draft mail/delete old draft mail" is;
>  1. Compose a mail, auto-save or manuakl save => append mail to Drafts =>
> UID=(A) is assigned at server
>  2. auto-save or manual save => append mail to Drafts => UID=(A+1) is
> assigned at server.
>     if UIDPLUS is supported, UID=(A) is known upon append at step 1 => UID A
> store Flags(\Deleted)
>     if UIDPLUS is not supported, UID=(A) is not known upon append at step 1
>        SEARCH for old draft by Message-ID => UID=A is returned => UID A
> store Flags(\Deleted)
>  3. Same actions as step 2 is executed for UID=(A+n) and UID=(A+n-1)
>  4. Send or Send Later => mail is sent or saved in Outbox
>     => UID (A+n) store Flags(\Deleted) for last saved draft mail
> 
> "UID of mail" is shown as "Order Received" column value if IMAP folder.
> And all above activities is logged by IMAP logging.
> 
> Show "Order Received" column, get IMAP log and check IMAP level flow by Text
> Editor.
>   See bug 402793 comment #28.
>   Use timestamp,smtp:5,imap:5,MsgCopyService
> Because IMAP activity on all accessed IMAP folders is logged by imap:5, get
> IMAP log with minimum configuration.
>   Disable all automatic new mail check, restart Tb with logging enabled,
>   go Work Offline, go Work Online, open one IMAP folder only of relevant
> IMAP account only,
>   compose a mail, save as draft, Send Later.
> 
> > keeping/deleting of wrong mails in IMAP account
> 
> Which draft mail of what UID was wrongly deleted by Thunderbird?

@WADA do you ask me to proceed like this? So I did get the NEEDINFO flag for this?

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to H. Sickenberg from comment #2)
> @WADA do you ask me to proceed like this? So I did get the NEEDINFO flag for this?

No. I simply want to say "Here is bugzilla.mozilla.org for developers to resolve Tb's bug".
Here is never support forum nor Help Center. So, even if you are general user, I believe that you have to provide data what is required for problem analysis by developers, because here is bugzilla.mozilla.org.

Comment 4

[H. Sickenberg]

Thanks for the feedback. So the message is: INVALID?

Comment 5

[rsx11m]

No, his request likely was referring to an IMAP log being helpful capturing the event as it happens (which may be tricky if it's hard to reproduce). If you follow the instructions below you should be able to start Thunderbird from the commandline logging all action of interest; then, if and when it happens, shut down Thunderbird and secure the log file. It's end should have the relevant information of the apparent mixup in UID identifiers.

(In reply to WADA from comment #1)
> Show "Order Received" column, get IMAP log and check IMAP level flow by Text Editor.
>   See bug 402793 comment #28.
>   Use timestamp,smtp:5,imap:5,MsgCopyService
> Because IMAP activity on all accessed IMAP folders is logged by imap:5, get
> IMAP log with minimum configuration.
>   Disable all automatic new mail check, restart Tb with logging enabled,
>   go Work Offline, go Work Online, open one IMAP folder only of relevant
> IMAP account only,
>   compose a mail, save as draft, Send Later.

Comment 6

[H. Sickenberg]

What ever I tried - it is not working for me.

I'm using Thunderbird on openSUSE 13.1 - I tried it with changing the thunderbird start script to:

#!/bin/bash
export NSPR_LOG_MODULES=ldap:5,smtp:5,timestamp
export NSPR_LOG_FILE=/tmp/thunderbird.log
/usr/bin/thunderbird

but I just get an empty tmp/thunderbird.log

Any idea?

Comment 7

[WADA:World Anti-bad-Duping Agency]

Did you do LDAP addressbook open and/or mail send?
Do you get log data with following?
> export NSPR_LOG_MODULES=timestamp,all:5

Comment 8

[H. Sickenberg]

Attachment: thunderbird.log

OK, logging was now working. See attached file.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Which draft mail of what UID was wrongly deleted after mail send(copy to Sent is log after SMTP send)?

Comment 10

[H. Sickenberg]

Attachment: thunderbird.log

I added another log file - this is the header of the deleted mail:

FCC: imap://[e-mail]/Sent
X-Identity-Key: id1
X-Account-Key: account1
Message-ID: <[e-mail]>
Date: Tue, 03 Jun 2014 09:29:33 +0200
From: Holger Sickenberg <[e-mail]>
X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0
MIME-Version: 1.0
Subject: This is a drafts folder test
X-Enigmail-Version: 1.6
X-Enigmail-Draft-Status: 512
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Comment 11

[WADA:World Anti-bad-Duping Agency]

Capability response
> 2014-06-03 09:51:45.199331 UTC - 288352000[7fc918683620]:
>   17dc8800:hsimap.dyndns.org:NA:CreateNewLineFromSocket: 
>   1 OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ IDLE UIDPLUS NAMESPACE CHILDREN MAILBOX-REFERRALS
>         BINARY UNSELECT ESEARCH WITHIN SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND]
Upload of draft mail(append)
> 53 append "imap/Drafts" (\Draft) {639+}
>      Subject: This is another test
Response to apend. Server doesn't return UID correctly.
> 53 OK Unable to re-open mailbox for APPENDUID: imap/Drafts

Fetch after append is not seen because UIDPLUS is returned from server. Tb perhaps deletes newest draft mail in Drafts before append. 
Is server correctly configured? If UIDPLUS support is not correctly configured, remove UIDPLUS from capability response.

Comment 12

[H. Sickenberg]

It is a uw-imap-2007e - there is not really anything to configure in that area

Comment 13

[Wayne Mery (:wsmwk)]

(In reply to WADA from comment #11)
> Capability response
> > 2014-06-03 09:51:45.199331 UTC - 288352000[7fc918683620]:
> >   17dc8800:hsimap.dyndns.org:NA:CreateNewLineFromSocket: 
> >   1 OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ IDLE UIDPLUS NAMESPACE CHILDREN MAILBOX-REFERRALS
> >         BINARY UNSELECT ESEARCH WITHIN SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND]
> Upload of draft mail(append)
> > 53 append "imap/Drafts" (\Draft) {639+}
> >      Subject: This is another test
> Response to apend. Server doesn't return UID correctly.
> > 53 OK Unable to re-open mailbox for APPENDUID: imap/Drafts
> 
> Fetch after append is not seen because UIDPLUS is returned from server. Tb
> perhaps deletes newest draft mail in Drafts before append. 
> Is server correctly configured? If UIDPLUS support is not correctly
> configured, remove UIDPLUS from capability response.

can we force thunderbird to ignore UIDPLUS capability ?

Comment 14

[:aceman]

No idea about IMAP commands.