846383 - Auto save creating persistent duplicates in Drafts folder, similar to Bug 382517

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: IMAP, defect)

Description

[mrbi11]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17

Steps to reproduce:

While composing a message, auto-save sometimes creates un-read "sent" messages in the same folder. 
Unlike Bug 382517, these copies do NOT go away by doing a folder repair, they are indeed on the imap server in the same folder.

Added info:
Occurs on an imap server (me.com)
The actual email address being sent from is forwarded to me.com,
that is x@forwarded_domain.com. I don't think its relevant, but not sure.

settings for the me.com account are: 
servers settings
check new messages every 10 minutes
Copies and folders
place copy in sent on me.com, place replies in same folder.
Drafts folder saved in local account (on my disk) 
(I was NOT using drafts, only auto save.)
Synch set to keep messages on this computer, and dont delete messages

x@forwarded_domain.com uses smtp at forwarded_domain.com

composition-general tab
auto save every 5 minutes

If my message filters are being applied, they would be moving the duplicates into this same folder.

extensions
CompactHeader2.0.7true{58D4392A-842E-11DE-B51A-C7B855D89593}
Correct Identity1.3.5true{47ef7cc0-2201-11da-8cd6-0800200c9a66}
Disable DragAndDrop (Thunderbird)[e-mail]
Extra Folder Columns1.1.4trueextra-cols@jminta_gmail.com
Google Contacts0.7true{BDD92442-0534-4D6F-A966-BAB7D561D781}
Show InOut1.0.0 true [e-mail]0.7.4true
[e-mail]
Thunderbird Message Filter Import/Export [e-mail]
Test Pilot for [e-mail]


Actual results:

Every 5 minutes when auto-save happens, i get a new "unread" intermediate copy of my message, as if it had been sent to the recipient.
Copy is actually there, repair folder shows no change.
message times are 5 minutes apart, same as auto save settings.


Expected results:

The auto save should not show in the folder at all, in fact it is not clear why it saves on the imap server at all. If my computer actually self destructs during composing an email, who cares? It is likely to happen 0 times per life-time. 
Putting it on an external mail server wastes more time than it could save.
There have been MANY bugs reported relating to storing auto-save on the mail server.
If it isn't right after a decade, maybe its time for a simpler approach.

Comment 1

[WADA:World Anti-bad-Duping Agency]

(In reply to mrbi11 from comment #0)
> Copies and folders
> place copy in sent on me.com, place replies in same folder.
> Drafts folder saved in local account (on my disk) 
> (I was NOT using drafts, only auto save.)
> composition-general tab
> auto save every 5 minutes

How do you choose the Drafts folder in local account at Copies&Folders?
  (a) Drafts folder of : the local account
  (b) Others :  selected Drafts of local account via folder picker
If (a), does folder named Drafts exist in the local account?

Do you check "Show conirmation dialog ..." at Copies&Folders setting?
If no, check the setting, and do "Save"(save as draft) at composition window.
Where is draft mail saved by Tb?

What Folder View do you use? Folder View=All? Unified Folders?
If non "All" folder view, what is your IMAP delete model setting in Server Settings? Just mark it as deleted? Or other?

What is actual problem of this bug report at B.M.O?

  (c) draft mail by auto-save is not saved in Drafts in local account
      although Draft of the local account is set in Copies&Folders.

  (d) when IMAP Drafts folder is used, old version of draft mail is not deleted
      and multiple draft mails for a compsed mail is shown in the Drafts folder.

      if this case, get protocol log for IMAP, and check IMAP level flow.
      see bug 402793 comment #28 for getting IMAP log.
      is delete of previous version of draft executed normally?      

      draft save in IMAP is roughly;
        append "Drafts", send mail data to server.
        if UIDPLUS, UID of appended mail is returned.
        if UIDPLUS is not supported by server, at Mbox of Drafts,
        serach for message-id of previous(old) draft is requested.
      Is UID of the old mail correctly returned?
        after it, Tb does do "uid xx store +Flag \Deleted" for the old draft.
        the UID for "store flag \Deleted" is;
           if UIDPLUS is supported, UID returned to previous append.
           if UIDPLUS is not supported, UID obtained by search for messae-id.
      Is this "store flag \Deleted" for old draft normally executed?

In any case, please keep "one problem per a bug" at B.M.O.

If you set "Just mark it as deleted" at Server Settings, "mail which has \Deleted flag" is shown with strike-thru line at thread pane.
Is old versions of a composed mail shown in the IMAP Drafts folder with strike-thru line when Drafts folder is viewed with Folder View=All?

Comment 2

[Wayne Mery (:wsmwk)]

mrbi11, please see comment 1 questions for you

Comment 3

[mrbi11]

Replies mixed with questions

(In reply to WADA from comment #1)
> (In reply to mrbi11 from comment #0)
> > Copies and folders
> > place copy in sent on me.com, place replies in same folder.
> > Drafts folder saved in local account (on my disk) 
> > (I was NOT using drafts, only auto save.)
> > composition-general tab
> > auto save every 5 minutes
> 
> How do you choose the Drafts folder in local account at Copies&Folders?
>   (a) Drafts folder of : the local account
>   (b) Others :  selected Drafts of local account via folder picker
> If (a), does folder named Drafts exist in the local account?

I was using B, then switched to A to see if it makes a difference.
The drafts folder appears to exist, at least its listed. 
It was initially empty, but
showed 21k space. Compacting it made space go to 0.


> 
> Do you check "Show conirmation dialog ..." at Copies&Folders setting?
> If no, check the setting, and do "Save"(save as draft) at composition window.
> Where is draft mail saved by Tb?

I checked per request. I am doing a "reply" from folder foo.
(It always happens on replies i think).
I have save sent mail to same folder (foo) set.
The draft went to foo, without prompting me.

> 
> What Folder View do you use? Folder View=All? Unified Folders?
> If non "All" folder view, what is your IMAP delete model setting in Server
> Settings? Just mark it as deleted? Or other?

Folders under view set to all.
under accounts servers, Delete moves to trash.

> 
> What is actual problem of this bug report at B.M.O?
Every 5 minutes i get a persistent duplicate copy of email being composed.
Happens on me.com imap and gmail.com imap.

I don't think i understood this question.

> 
>   (c) draft mail by auto-save is not saved in Drafts in local account
>       although Draft of the local account is set in Copies&Folders.
yes this is happening. 

> 
>   (d) when IMAP Drafts folder is used, old version of draft mail is not
> deleted
>       and multiple draft mails for a compsed mail is shown in the Drafts
> folder.
No this is not happening.
The copies are in the "foo" folder.

> 
>       if this case, get protocol log for IMAP, and check IMAP level flow.
>       see bug 402793 comment #28 for getting IMAP log.
>       is delete of previous version of draft executed normally? 
Will post when i figure out how to do this.
     
> 
>       draft save in IMAP is roughly;
>         append "Drafts", send mail data to server.
>         if UIDPLUS, UID of appended mail is returned.
>         if UIDPLUS is not supported by server, at Mbox of Drafts,
>         serach for message-id of previous(old) draft is requested.
>       Is UID of the old mail correctly returned?
>         after it, Tb does do "uid xx store +Flag \Deleted" for the old draft.
>         the UID for "store flag \Deleted" is;
>            if UIDPLUS is supported, UID returned to previous append.
>            if UIDPLUS is not supported, UID obtained by search for messae-id.
>       Is this "store flag \Deleted" for old draft normally executed?
ditto

> 
> In any case, please keep "one problem per a bug" at B.M.O.
Not understanding this. 
I'm only seeing one bug report, autosave is broken.
Symptom is persistent copies in folder where reply is done.

> 
> If you set "Just mark it as deleted" at Server Settings, "mail which has
> \Deleted flag" is shown with strike-thru line at thread pane.
This is not set.

> Is old versions of a composed mail shown in the IMAP Drafts folder with
> strike-thru line when Drafts folder is viewed with Folder View=All?
The copies are not in the drafts folder.
They are in the folder where the mail being replied to is.

Comment 4

[CP]

I'm seeing this too and it was super confusing. Unsent drafts are periodically saved, it looks like, and then my mail filters are moving them into a folder for me. I thought e-mail I hadn't even finished writing had been sent. Turns out, it's just versions of drafts. 

I'm using Thunderbird 24.6.0 on Fedora 20.
thunderbird-24.6.0-1.fc20.x86_64

I do not have access to our IMAP server, so I can't provide as much info as mrbi11 (nice handle, btw).

Comment 5

[Wayne Mery (:wsmwk)]

Do you still see this issue when using a current version of Thunderbird?

(I did, but no longer, see something like this)

Comment 6

[mrbi11]

Wayne Merry, yes this happens anytime I edit slowly enough for autosave to occur.

I am on http://www.inmotionhosting.com/ which i can set up an email account for someone to use to reproduce and debug.

Sorry it took long time to reply. I turned off auto update on thunderbird because it has frequent problems. 
Such as when i upgraded the folder colums plogin broke.

Anyway, for reasons unclear to me, the admin on inmotion require ALL email folders to be a subfolder of inbox.
I think the act of saving to the drafts folder flags the inmotionhosting email that a new email has arrived,
or some such.

The latest release actually got much worse for me, as all my sorting message filters broke.
I think for the same reason, that moving an email signals that a new email has arrived.
Apparently previously Thunderbird somehow checked for this somehow and no longer does.
So now if email E matches rule to move to folder inbox/X
then Thunderbird gets notice of a new email in inbox/X, runs the rules again
then Thunderbird matches a different rule, moves it to inbox/Y 
then moves it back to inbox/M abd repeat.

essentially, Thunderbird fails to recognize an email it just moved (or stored to inbox/drafts) is NOT
a new email.

As I said I can set an email account for testing for you, but I expect it can be replicated
on any email server by creating a drafts folder under inbox folder.

Ditto for creating subfolders inbox/X and inbox/Y and moving emails to them.

The general solution to both bugs would be to recognize an email that thunderbird 
just stored is NOT a new email, even if flagged as such by the email server.

Comment 7

[mrbi11]

I dont agree it is similar.

Also, the folder where the duplicates occur is not always drafts folder.
If the email is a reply, from folder X, the duplicates go in Folder X.

I believe it may be related to the fact all my subfolders, including drafts, are under "INBOX".
So a likely scenario is that the saved email to drafts is in some way triggering the handling for incoming email, which
also moves the draft email to the X folder.

Another likely scenario is the handling of moving "reply to" email to the X folder as per the settings to keep replies in the same folder as 
the email being replied to.

Should anyone want to work on this, and is having trouble replicating, I can create a test account for you on my domain
and it should be easily fixed with a few minutes debugging.

Is it socially acceptable to offer a reward for bug fixes?
Seriously its worth $ to me to get fixed.

Comment 8

[Wayne Mery (:wsmwk)]

> Is it socially acceptable to offer a reward for bug fixes?
Yes. It's called a bounty.  There are various places you can create one. For example https://www.bountysource.com

Comment 9

[andy]

I am having this issue in thunderbird 38.6.0 on linux. I am using POP and my drafts folder is in the default location. I have auto save for 1 minute, so it can get pretty annoying having to sort through and keep real drafts but delete these duplicate drafts for e-mails that have already been sent. The problem starts occurring randomly. The best correlation that I can find is that it starts happening after I have thunderbird open for about 1 week or more. If I close thunderbird frequently and re-open it, the problem doesn't seem to happen as much. Once it starts happening, it seems to happen to all messages I am composing until I shut thunderbird down and re-open it.

Comment 10

[David Taber]

I have had this issue with TB for several years now, but with one of the releases this summer the symptom seems to have disappeared (at least on Windows).

IMHO we can close this one.

Comment 11

[Wayne Mery (:wsmwk)]

mrbi11, do you still see this when using a current version?

Comment 12

[mrbi11]

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #11)
> mrbi11, do you still see this when using a current version?

Sorry to give vague response, but I don't  know. My experience it was intermittent. 
Something would trigger it, then once triggered one would get a new email message every auto save. 
I think it may have more to do with variations in mail servers behavior than t-bird, which is why it has not yielded to fast solution.
Having said that...
====================================================
There is a fundamental design problem here. Even if you sets it to save drafts on a local machine, it saves them
on the internet draft folder. If that bug/design flaw were not there, this bug would never happen, 
as it requires some interaction with the mail server.

Why is this important? You only need to save something to drafts if you cannot complete and send an email.
The main cause of that is that THE INTERNET IS UNAVAILABLE.
WHEN THE INTERNET IS UNAVAILABLE >>>YOU CANNOT SAVE YOUR EMAIL TO DRAFTS ON THE INTERNET<<< AND THEREFORE 
IT IS IMPOSSIBLE TO SAVE YOUR DRAFT AT ALL!!! THE WHOLE REASON FOR SAVING DRAFTS IS DEFEATED BECAUSE 
YOU CANT SAVE DRAFTS AT ALL.

Sorry for shouting, but I have pointed this out years ago, and the functionality to save drafts locally, 
while it appears as an option does not work, and this whole programming arc including this bug is a huge pointless waste of time.
As long as drafts are not saved locally, draft emails are un-savable the only time it matters,
ie WHEN YOU CANT SEND IT YOU ALSO CANT SAVE IT TO DRAFTS... because the internet is unavailable Have I repeated that enough?

Because of this, there should not be an option to save it to the mail server at all, and always save to the local
computer. Saving locally you can only lose it if your computer actually permanently dies as you type you have lost your email, 
which is about 100,000 times less likely than losing your internet connection. So much less likely that it wont happen
in the average persons life at all.

But internet connections come and go, and in the real world one needs to save drafts without an internet.

I hope someone will step up to the actual problem and fix it. 
I can offer $200 to that person as a reward for fixing it. Not huge, but possibly others would donate as well.
My personal programming days are in the past.
Best wishes all.

Comment 13

[Wayne Mery (:wsmwk)]

Thanks for the update.
What you describe about the imap folder not being availble is covered under bug 457136 and bug 147519.
So we can close your original issue, this bug, as WFM