673400 - Read/Unread status of draft mail should be configurable (due to change by bug 470746, Tb doesn't store \Seen for draft mail, then Android Smart Phone, and needless to say Tb too, shows new mail alert for the draft)

Status: NEW

(MailNews Core :: Backend, defect)

Description

[Tammo van Lessen]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

TB5 with Google Mail via IMAP in conjunction with GMail for Android:
1. Write a reply to a message with auto save turned on. Because of #618553, this results in multiple drafts, which are marked unread since TB5 (#470746).
2. Let TB synchronize folder with IMAP
3. Check the same conversation in GMail for Android
4. Send the mail and sync.
5. Check the same conversation in GMail again.


Actual results:

for 3: GMail notifies about new messages in the conversation (but there are no new messages, only a draft has been changed. Thus the notification is wrong). Also, it shows every single draft that has been created as new mail. Due to #618553 this can be a lot when autosave is on.

for 5: GMail again notifies about a new mail in the conversation, this time because the unread draft is moved to trash and remains marked unread. The conversation is now also labeled "trash"


Expected results:

I expected the behavior from TB 3.1: Drafts and trashed drafts are marked read and will thus not confuse GMail.

I understad this is an interoperability issue and it is not purely TB's fault. However, TB5 introduced this interop regression and thus I report it here. IMO it makes TB almost unusable in this scenario, since a long mail could cause ~10 drafts, which are then a) shown as new mail and b) have to be scrolled through in order to see if there has been a real new message (reply) added to the conversation.

I also skimmed through the related bugs. I understand the reasons for #470746, perhaps there is a different way to bring new drafts to user's attention than marking them unread. What will work in any case is to make the "mark new drafts unread" feature configurable and to think about marking it read before/when trashing it.

Comment 2

[[:Aureliano Buendía]]

xref bug #470746 ?

Comment 3

[Brian Hauer]

I was a voter on bug 671232.  I believe this behavior of marking draft messages as unread is new in Thunderbird 5.  I don't recall it ever happening in 3.x.

I voted on the other bug because I find the behavior bizarre and somewhat disorienting.  I'll be composing a message in a message window and see a bold number come up in the main Thunderbird window.  I think, "Oh, a new e-mail just arrived."  But it turns out it's just my work-in-progress being saved as an unread draft.

So to be clear: I'm not using GMail; not using Android.  I voted on 671232 because I suspect the general behavior of marking drafts as unread wasn't intentional.  I wanted to chime in here just to say that as I see it, this is not actually an issue isolated to GMail and Android.

Comment 4

[Pete Gurney]

I am in agreement with Brian, I originated this bug and am using a standard IMAP server for mail. This behaviour certainly wasnt present in 3.x. I have the same behaviour also while composing mail, as soon as the autosave happens the program reports a new message.

Comment 5

[Tammo van Lessen]

I also agree with you. Not the bug reports a duplicates but their origin, so a "x blocks y" relation would probably fit better. I see the same behavior as you do and in addition it causes the issues with GMail on Android. For the me the latter is more annoying but of course the issue should be fixed at its roots. When reading through related issues, I also found the issue that asked for the behavior we have right now (drafts should be marked unread so that people will notice that there is a draft at all) and there was a discussion whether this behavior could be achieved with some UI foo instead of marking it unread (in IMAP). I guess this would at least help a little against "new mail notifications". In general, I also don't care about the information whether there is now a new draft or not. With autosave on, I usually don't need most of the drafts and if I need one, I'm aware of that and can actively search it. Drafting emails is IMO something people are aware of and do not necessarily have to be reminded about something they know already.

Comment 6

[WADA:World Anti-bad-Duping Agency]

To all problem reporters:

Even when newly saved draft mail is marked as Unread by Tb, if Tb, and if automatic new mail check is disabled for Drafts folder in Tb, and if IDLE command use is disabled at Server Settings/Advanced of Tb, new mail notification won't automatically be passed to Tb from server, then new mail alert on Drafts folder by Tb won't appear automatically.

Why do you need "automatic new mail check of Drafts folder" by your Smart Phone and/or other IMAP clients?
Why do you intentionally enable "automatc new mail check of Drafts folder" in any IMAP clients which you concurrently use, even though you simulteneously use/access same IMAP account's mbox named Drafts(to save draft mail during your intentional mail composition) from multiple your IMAP clients?

Please note that I'm not opossite to request of "Read/Unread status of newly saved draft mail by Tb should be optional or selectable".

Comment 8

[Konstantin Stefanov]

(In reply to WADA from comment #6)
> Why do you need "automatic new mail check of Drafts folder" by your Smart
> Phone and/or other IMAP clients?
And why TB has this feature turned on automatically? I reported bug #676249, which deals with TB itself, only one instance, no special configuration, and TB behaves alike: it produses new message notification on message autosaved by itself in Drafts.

Comment 9

[Konstantin Stefanov]

And must say that 6.0 and 7.0.1 do have the same bug.

Comment 10

[Tammo van Lessen]

(In reply to WADA from comment #6)
> Even when newly saved draft mail is marked as Unread by Tb, if Tb, and if
> automatic new mail check is disabled for Drafts folder in Tb, and if IDLE
> command use is disabled at Server Settings/Advanced of Tb, new mail
> notification won't automatically be passed to Tb from server, then new mail
> alert on Drafts folder by Tb won't appear automatically.

Actually the automatic new mail check is disabled for my Drafts/Trash folders, but I like the IDLE feature. I'll try that as a workaround, but it won't fix this issue in a sound manner IMO.

> Why do you need "automatic new mail check of Drafts folder" by your Smart
> Phone and/or other IMAP clients?

I don't need this, but it's just not configurable in Android's GMail client. It's even pushing changes in these folders. While you can configure the labels to be subscribed to, these 'system folders' are not in this list.

> Why do you intentionally enable "automatc new mail check of Drafts folder"
> in any IMAP clients which you concurrently use, even though you
> simulteneously use/access same IMAP account's mbox named Drafts(to save
> draft mail during your intentional mail composition) from multiple your IMAP
> clients?

This checkbox is not checked in my setup, but the folder is included in Gloda. Could that be a reason?

> Please note that I'm not opossite to request of "Read/Unread status of newly
> saved draft mail by Tb should be optional or selectable".

Puh ;)

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to Tammo van Lessen from comment #10)
> > Why do you need "automatic new mail check of Drafts folder" by your Smart
> > Phone and/or other IMAP clients?
> I don't need this, but it's just not configurable in Android's GMail client.
> It's even pushing changes in these folders.

I see.
If it's applicable to [Gmail]/All Mail too, "one additional new mail alert per a draft save" occurs at Android Smart Phone, because it's Gmail IMAP(Gmail's design).
If it's applicable to [Gmail]/Trash, "a new mail alert per an old draft delete upon draft save" additionally occurs at Android Smart Phone, because it's Gmail IMAP(phenomenon of Bug 562748).
i.e. "Three new mail alerts per a draft save by Tb" is invoked at an Android Smart Phone. It happens every five minutes at all Android Smart Phone you use, if you start mail composition in Tb with auto-save interval=5 minutes.

Some peoples want Unread status of newly saved draft mail for their convenience, and current "Unread status upon draft save" was done by request from such users as the request was reasonable and acceptable.
However, now it's found that "Unread upon draft save" produces problem of this bug.
I think next is practical solution,
  Change to "always set as Read upon draft save",
  until "configurable Read/Unread of newly saved draft" will be implemented, because I think "loss by Unread status due to problem like this bug" is larger than "gain by Unread status in user's convenience",
and because I believe silent majority of users are torelant with any of Unread or Read.

(In reply to Tammo van Lessen from comment #0)
> marking it read before/when trashing it.

"Mark as Read of draft mail upon move to Trash due to phenomenon of Bug 562748", shouldn't be applied to non Gmail IMAP account. Such feature should be lmited to Gmail IMAP account only even if such feature will be implemented.

Some peoples are eager for "change to Read upon move to Trash", and some peoples are eager for "keep Unread mail as Unread upon move to Trash".
(silent majority of users has favarite one, but can be torelant with any of two)
To avoid Read vs Unread war on Trash, "configurable Read/Unread/Not-change" is needed for Trash too.

Comment 12

[WADA:World Anti-bad-Duping Agency]

Possible short/mid-term solution.
  If Gmail IMAP account, set as Read(append with \Seen flag) upon draft save.
Because Read or Unread of newly saved draft won't be changed for non-Gmail-IMAP account(POP3, Local Folders, IMAP other than Gmail IMAP), affected users are Gmail IMAP users only, and affected at his Gmail IMAP folder only.
  Gmail IMAP user who is eager for Unread status of saved draft
  looses convenient "Unread of saved draft" for him at Gmails's Drafts.

Comment 13

[leitl.christoph]

On 2012-01-11 13:42:28 PST 
https://bugzilla.mozilla.org/show_bug.cgi?id=687140 Irving Reid (:irving) wrorte:

 We're working on a single implementation of new mail notification for all platforms right now in bug 715799, so rather than patch the existing Unix-specific code I'll include an equivalent fix in the shared version.

Watch https://bugzilla.mozilla.org/show_bug.cgi?id=715799 for updates.
Nevertheless, windows users could vote for Bug #687140 as well.

Comment 14

[WADA:World Anti-bad-Duping Agency]

(In reply to leitl.christoph from comment #13)
> Watch https://bugzilla.mozilla.org/show_bug.cgi?id=715799 for updates.
> Nevertheless, windows users could vote for Bug #687140 as well.

Thanks for pointing to bugs.
These bugs are for change of new mail notfication behavior of Tb, but this bug is for issue of unwanted new mail alert(s) at Android Smart Phone on draft mail saved by Tb(\Seen flag is not stored by Tb, then Android Smart Phone shows alert for the draft mail at Smart Phone.)
Does your comment posting to this bug imply following?
 Any Android Smart Phone who accesses IMAP server uses Tb as his IMAP client,
 and Android is surely included in "all platform" of fix for bug 715799,
 (note: "Android" exists in drop down menu of Platform: of bug at B.M.O)
 and fix of the bug will be applied to Android Smart Phone by Smart Phone vendor?
Note:
"At least one Android Smart Phone vendor uses Gmail IMAP as his IMAP server" is already known by comment #0.

Comment 15

[Tammo van Lessen]

Attachment: untested git patch

I tried to create a patch that enables the following:
  - The default behavior is not changed.
  - A per identity preference 'mark_drafts_read' is introduced. If this pref is set to true, drafts are marked read, if it is false (default), drafts are marked unread.
  - No UI support for the flag (yet)

Since this is my first contact with the Mozilla code base and I still don't have a working tool chain running to build Thunderbird, this patch is completely untested, I don't even know if it will compile. It is more meant to serve as a starting point for a TB dev. Would be great if someone could give it a try.

Comment 16

[WADA:World Anti-bad-Duping Agency]

Oh, "victim of change by bug 470746 == Tb user" cases, bug 671232 and bug 676249, were already dup'ed to this bug.
I thought enhancement of "configurable Read/Unread of draft" will be done by bug opened earlier for victim=Tb user case, and critical/special case(this bug for victime=Android Smart Phone case) will be fixed by such change, and this bug will be duped to such bug if required.

(In reply to Constantin Stefanov from comment #8)
> And why TB has this feature turned on automatically? I reported bug #676249,

I wasn't aware of your duping to this bug.
If namespace=Inbox, draft folder = Inbox/Drafts or Inbox.Drafts, and Tb may check all subfolders under Inbox upon automatic new mail check, and, as stated in comment #13, Tb currently doesn't exclude drafts folder from new mail alert.

Your report/request is accurate and crispy too.
> Actual results:
> (a) The saved draft appears in 'draft' folder as new (unread) message,
> (b) and 'New message' alert sounds.
> Expected results:
> (A) The message should appear in 'Drafts' folder as read (the preferable),
> (B) or at least 'New message' alert should not trigger.

Initial bug summary of bug 687140 was following. It looked absolutely same as your (a).
> message in Drafts folder announced as new mail
But request by bug opener of that bug was (B). So I added following(complaint by bug opener) to bug summary of that bug.
> (New email alert ping after saving a mail in the drafts folder)
Then, current bug summary looks absolutely same as your (a) + (b) :-)
> bug 687140 : message in Drafts folder announced as new mail
>              (New email alert ping after saving a mail in the drafts folder)

Current status:
  (A) : Request of bug opener of this bug, Tammo van Lessen, in comment #0.
        Because (B) can do nothing for Android Smart Phone case,
        he didn't request (B).
  (B) : Request of bug 687140. 
        Linux : already fixed by bug 661239.
        because fix of bug 661239 is Linux only fix, bug 715799 has been opened.

I'll re-open your bug for ease of tracking of both (A) and (B), and for ease of problem analysis or QA work of future bugs for "annoying new mail alert on draft saved by Tb".
And change bug summary of this bug to explict request for "configurable Read/Unread of draft".

Comment 17

[WADA:World Anti-bad-Duping Agency]

(In reply to Tammo van Lessen from comment #15)
> untested git patch

Owner/patch creator of Bug 470746 is Mark Banner, and reviewer of the patch is David :Bienvenu. Asking review to them is perhaps appropriate action.

> +    // check identity preference to mark draft mails read (Bug #687140, #673400)
> +    aUserIdentity->GetBoolAttribute("mark_drafts_read", &markRead);

(1) When per entity attribute like server.serverX.aaaa, developer frequently introduces default.aaaa like one and puts it in mailnews.js(or all.js) in order to show the default setting at Config Editor panel.
> mail.identity.default.draft_folder = mailbox://nobody@Local%20Folders/Drafts
Is there any plan for such default? 

(2) Currently used setting is one like next.
> mail.identity.idN.drafts_folder
> mail.identity.idN.drafts_folder_picker_mode
> mail.server.serverX.trash_folder_name
> mail.server.serverX.moveOnSpam  <= shown at far upper from spamActionTargetFolder
> mail.server.serverX.spamActionTargetFolder

I prefer draft_folder.mark_read, fcc_folder.mark_read, trash_folder.mark_read, spam_folder.mark_read like one, because options relate to a folder is shown at near location at Config Editor Panel, and common mark_read is used for any folder.
(this implies request for trash_folder.mark_as_read_on_delete at same time :-) )

Comment 18

[David :Bienvenu]

Comment on attachment 588147 [details] [diff] [review]
untested git patch

you would want to add a default value for pref in mailnews.js

Comment 19

[Mark Banner (:standard8)]

I think what might be the better option here is for TB not to mark messages as unread, but still highlight items in the draft folder as existing in a similar style to the unread listing.

@bwinton: any thoughts on this?

Having said that it'd be interesting to do some research on other clients first, i.e. which ones actually mark as unread when you save a draft email, and which mark as read - and see if there's a general trend. I suspect the trend is unread, but I'm not altogether sure.

So far, I've come across K9 which marks as unread, and zimbra web client which marks as read.

Comment 20

[Tammo van Lessen]

Mark, David: First off, thanks for taking care of this issue. 

(In reply to Mark Banner (:standard8) from comment #19)
> I think what might be the better option here is for TB not to mark messages
> as unread, but still highlight items in the draft folder as existing in a
> similar style to the unread listing.
> 

I fully agree. I believe there is, conceptually, no such thing like an unread draft. Either there are drafts or not and thus it makes sense to somehow visualize whether there are drafts in the draft folder, which somewhat indicates that there is some pending writing work to do, or not. But I don't think that the \Seen flag should be used for that. So +1 from me for highlighting the drafts folder if not empty and showing an item count using an alternative approach.

> Having said that it'd be interesting to do some research on other clients
> first, i.e. which ones actually mark as unread when you save a draft email,
> and which mark as read - and see if there's a general trend. I suspect the
> trend is unread, but I'm not altogether sure.

Even if the trend is "unread", the behavior should be configurable because of the annoying side effects with Google Mail's android app.

Comment 21

[Blake Winton (:bwinton) (:☕️)]

(In reply to Mark Banner (:standard8) from comment #19)
> I think what might be the better option here is for TB not to mark messages
> as unread, but still highlight items in the draft folder as existing in a
> similar style to the unread listing.
> 
> @bwinton: any thoughts on this?

Yeah, I think I agree with Myk in bug 198087 comment 21.
"We should be able to have a css rule for the folder pane that says:  "if it's a
drafts folder and it has messages, make it bold (or what ever)".

we should log a spin of bug about fixing this to do it that way, and not
overload unread this way.

one problem with overloading unread is messages could get marked unread, and
then you'd have the same problem."

Later,
Blake.

Comment 22

[leitl.christoph]

So, will the problem "annoying new mail alert on draft saved by Tb" be solved soon?

Comment 23

[Ludovic Hirlimann [:Usul]]

Comment on attachment 588147 [details] [diff] [review]
untested git patch

To have someone look at patches you'll need to set either the review flag or the feedback flag to ? and select a requestee :-)

Comment 24

[David :Bienvenu]

Comment on attachment 588147 [details] [diff] [review]
untested git patch

thx for the patch - at the very least, you need a default value for the pref in mailnews.js, i.e., mail.identity.default.mark_drafts_read, false

Comment 25

[Tammo van Lessen]

Attachment: 2nd version of the patch, added default setting to mailnews.js

As requested, I added the default value to the mailnews.js. Unfortunately I was unable to setup a test environment. I followed the docs @ mozilla but for some reason I was unable to recompile Thunderbird after applying the patch. It does not give a compile error, it smoothly runs through, but when running the test cases, it appears that nothing has changed. Do I need to run a full build after every change? Since this is my first contact with TB's code, it would be great if a core dev could take care of testing the patch, but I'm of course willing to help and to contribute by tweaking the patch.

Comment 26

[Tammo van Lessen]

Attachment: A variant of the same patch, but adds a new field in IDL

I added a variant of the same patch. The first patch uses aUserIdentity->GetBoolAttribute("draft_folder.mark_read", &markRead); Since I'm not sure if this is the way to go, I also created a variant that adds a new field to nsIMsgIdentity.idl.

Comment 27

[Tammo van Lessen]

Attachment: 3rd version of variant A patch. Now it uses 'draft_folder.mark_read' instead of 'mark_drafts_read'

Comment 28

[Mark Banner (:standard8)]

Comment on attachment 597690 [details] [diff] [review]
3rd version of variant A patch. Now it uses 'draft_folder.mark_read' instead of 'mark_drafts_read'

Sorry, but I don't think we should do this.

As discussed in comments 19 to 21, I think we should just revert to not marking as unread and fix the display highlight at the same time.

There's no point in having an intermediate preference - we should be easily able to fix both at the same time.

I can dig out some pointers as to where to fix the UI if you want some.

Comment 29

[David :Bienvenu]

Comment on attachment 597690 [details] [diff] [review]
3rd version of variant A patch. Now it uses 'draft_folder.mark_read' instead of 'mark_drafts_read'

right, forgot about that, clearing review request.

Comment 30

[David :Bienvenu]

Comment on attachment 597689 [details]
A variant of the same patch, but adds a new field in IDL

clearing review request per standard8's comment.

Comment 31

[Robert "Unlogic" Olofsson]

Is there any progress being made on this issue seeing how several patches have already been committed?

Comment 32

[Wayne Mery (:wsmwk)]

changing severity, because this would solve a regression bug

Comment 33

[WADA:World Anti-bad-Duping Agency]

(In reply to Mark Banner (:standard8) (afk until 1st Aug) from comment #28)
> 3rd version of variant A patch. Now it uses 'draft_folder.mark_read' instead of 'mark_drafts_read'
> 
> Sorry, but I don't think we should do this.
> As discussed in comments 19 to 21, I think we should just revert to
> not marking as unread and fix the display highlight at the same time.
> There's no point in having an intermediate preference
> - we should be easily able to fix both at the same time.

Does it mean rejection of following request by this bug?
  Read/Unread status of draft mail should be configurable
If so, I think new bugs for following requests, close this bug as WONTFIX, is better.
(a) Backout of change by bug 470746 as soon as possible.
By this, problem of this bug, Bug 687140, and bug 809513 automatically
disappers, and bug such as bug 676249 won't be opened any more.
(b) implement "highlighting draft mail" for reminding or notification of
new draft mail to other IMAP client who shares IMAP Drafts folder.

Should we open bug for (a)?

Comment 34

[WADA:World Anti-bad-Duping Agency]

For Read or Unread status of saved draft mail.

According to bug 470746 comment #4, Tb's behavior looks;
(1) Initial :                 Unread
(2) After fix of bug 198087 : Read
    => bug 470746 was opened
(3) After fix of bug 470746 : Unread
    => bug 661239 was opened
And,
(4) bug 661239 was fixed on Linux only,
    => "New mail alert on Drafts" was prohibited on Linux only.

Which is Tb's spec on draft mail?
Saved draft=Unread? Saved draft=Read?
Is following Tb way?
  (i)   If bug for complaint on Unread is opened, change to Read,
  (ii)  If bug for complaint on Read   is opened, change to Unread,
  (iii) Repeat (i) and (ii) forever

Comment 35

[WADA:World Anti-bad-Duping Agency]

(In reply to Mark Banner (:standard8) (afk until 1st Aug) from comment #28)
> 3rd version of variant A patch. Now it uses 'draft_folder.mark_read' instead of 'mark_drafts_read'
> 
> Sorry, but I don't think we should do this.
> As discussed in comments 19 to 21, I think we should just revert to
> not marking as unread and fix the display highlight at the same time.
> There's no point in having an intermediate preference
> - we should be easily able to fix both at the same time.

Your comment was posted on 2012-02-16.

And, according to bug 470746 comment #4, change history is;
(1) Initial :                 Unread
(2) After fix of bug 198087 : Read
   (bug 198087 was opened on 2003-03-18, and was FIXED on 2004-04-12) 
    => bug 470746 was opened on 2008-12-22
(3) After fix of bug 470746 : Unread
    Owner of bug 470746 is you. And was fixed on 2010-10-26 by you.
    => bug 661239 was opened on 2011-06-01.
And,
(4) bug 661239 was fixed on Linux only by Mike Conley,
    and the bug 661239 was changed to FIXED by you on 2011-07-04. 
    => "New mail alert on Drafts" was prohibited on Linux only.

What is reason of "bug 470746 is still not backed out yet", even though you posted comment #28 on on 2012-02-16, and despite that you already know well about problem of bug 676249, and request by this bug(bug 673400) & Bug 687140 & bug 809513 in which we are respecting your deciosion on "Unread status of draft mail" what was made in bug 470746 by you?

Comment 36

[Mark Banner (:standard8)]

Wada, backing out bug 470746 would now cause some regression in UI functionality from what we had before (although, that in itself could be called a regression). In any case, I would rather we fixed the UI and backed out at the same time, rather than reverting to the old behaviour before at some later stage re-implementing the fix.

Comment 37

[WADA:World Anti-bad-Duping Agency]

(In reply to Mark Banner (:standard8) from comment #36)
> backing out bug 470746 would now cause some regression
> in UI functionality from what we had before

Change by bug 470746 is following only(essentially one line patch).
>                                                After patch by bug 470746
>     rv = copyService->CopyFileMessage(
>          aDiskFile, dstFolder, aMsgToReplace,
>          aIsDraft,
>          nsMsgMessageFlags::Read,         ==>  aIsDraft ? 0 : nsMsgMessageFlags::Read,        
>          EmptyCString(),
>          copyListener, msgWindow);

What kind of problem did you have in UI functionality before your patch of bug 470746?
What kind of problem in UI functionality was resolved by your patch of  bug 470746?

If the problem is request of bug 470746, problem by "backout of bug 470746" is following only, isn't it?
- Problem of bug 470746 comes back.
- Your automatic test provided by bug 470746 always fails.
If so, and if much severe problem is found by patch for a bug, "backout/re-open of the bug" is normal procedure, isn't it?

Comment 38

[Mic]

I tried to fix this problem with this addon (waiting for a review at the moment): https://addons.mozilla.org/en-US/thunderbird/addon/imap-draft-unread/

It works for me, so now the drafts are unread but doesn't trigger a mail alert.
There is an option also to set the drafts as read.

Comment 39

[Freddy]

The 'imap draft unread' extension worked wonders for a while. Until Thunderbird 68 came along and the old extension no longer work. Is there any chance this becomes a configurable option? I think it is weird to set an unread flag on mails in the drafts folder whenever you (re)edit them.

Comment 40

[Eddie Gerber]

Attachment: same problem

Thanks from litandnesua1976.