Closed Bug 1477557 Opened 6 years ago Closed 6 years ago

Mails marked as deleted are not displayed

Categories

(Thunderbird :: Untriaged, enhancement)

Product:
Thunderbird
Component:
Untriaged
Version:
52 Branch
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: michel, Unassigned)

References

Details

(Whiteboard: [dupme])

Attachments

(1 file)

hybrid-model-can-unwant-expunges-problem.diff
24.35 KB, patch
Details | Diff | Splinter Review 
User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/67.0.3396.99 Chrome/67.0.3396.99 Safari/537.36

Steps to reproduce:

Flag a Mail as deleted and Thunderbird will not Display these Mails, when the config ist set to "When deleting a message" -> "Move to this folder".

That's just wrong. 

If another IMAP client marks the messages like this, then they may not disappear in Thunderbird either. 
The behavior when deleting a mail has nothing to do with the appearance of the mails.



Actual results:

Mails marked as deleted disappear.


Expected results:

They should be crossed out. As if the delete mode is selected in Thunderbird on "When deleting a message" -> "Mark all deleted".
Sounds like a feature change request. So if another client marks a message \deleted, tb will show the message but crossed out regardless of other settings?

If you want to see messages that other clients have marked \deleted then just change the option to "just mark it as deleted" and they will appear crossed out. Of course, this is not as automatic as you suggest.

In any case, I suspect if this were done it would be behind a hidden pref since it would surprise a lot of users.
If you think about it carefully, one has nothing to do with the other!

IMAP is probably comprehensive and designed for the access of several clients.

How Thunderbird should proceed when deleting mails simply has nothing to do with whether it should display messages marked as deleted. Theoretically he would have to move the messages marked as deleted into the paper basket.

It's an error not to display mails just because they don't match a configuration setting that only indirectly has to do with it.

I provide support for many users and can only say that such an approach leads to confusing situations that nobody needs.

In the end, a program should serve the user and be user-friendly and transparent, which Thunderbird is absolutely not here.
> IMAP is probably comprehensive and designed for the access of several clients.

Yes, but this does not mean the imap protocol intended for handling MAIL is intended to be a contact/task/workflow management system (although there are probably one or more bug reports already on file for this).  And we should not alter Thunderbird for a less popular business use case when default behavior is targeted to the average, individual user.

But you can design and implement a workflow process with a combination of folders, settings and perhaps tools outside Thunderbird. Some readily available items:

- "mark as deleted" delete model
- http://kb.mozillazine.org/Mail_and_news_settings offers some expunge settings
- special, perhaps multiple, deleted mail folders as suggested i nhttps://superuser.com/questions/1301505/maintain-separate-read-unread-status-for-multiple-users-in-one-outlook-mailbox
Severity: normal → enhancement
Whiteboard: [dupme]

If another program marks the mails as deleted and Thunderbird just ignores this mail, it's a mistake!

It's not a matter of opinion!

It has basically nothing to do with how Thunderbird should behave when deleting mails.

As long as these two things are not seen by you as separate processes, you do not understand the error.
This isn't about Thunderbird, this is about imap. imap is an approved SPEC, and you (and we) don't get to choose how it operates.
Where the "IMAP SPEC" says that a mail program should not display messages marked as "deleted"?

That's exactly what this is about!

It's not about settings in Thunderbird here nothing has to be changed.

Thunderbird should only behave in conformity to IMAP
We certainly are not going to make "mark as deleted" the default behavior. If that is your request, then we can close this bug.  If it is not, then what do you propose be done?
I'm sorry, maybe I'm making myself misleading.

In the answer I notice that you did not understand at all what the goal is.

These settings, which you talk about all the time, have nothing to do with it.

If there are deleted mails they should simply be displayed and crossed out as it should be.

No matter how Thunderbird is configured when deleting mails! Because it has nothing to do with that!

Because IMAP in the SPECS provides the "/delete" flag and Thunderbird tramples it underfoot.
I understand your goal - you want deleted messages shown in the message list.  Did you test the delete option "just mark it as deleted" under account > server settings?
This program setting is of course known to me!

But that's exactly the point -- it's not about program settings.

Mails that are marked "deleted" in the mailbox must also be displayed and must not simply be ignored.
They are just there and it doesn't matter how this state came about.

Thunderbird behaves wrong at this point and only tries to justify its program settings -- but why ?

Display messages marked as "deleted" ( no matter how Thunderbird should delete mails ) and the IMAP "SPEC" will be more noticed than otherwise. 

Here's an example. ( It has nothing to do with "contact/task/workflow management system")

You are using a mobile phone with Mailapp and Thunderbird on your PC. On the mobile phone, mails should be marked as deleted (there are good reasons for this) on the normal PC in Thunderbird, but this should not be so.

If you delete now a Mail on the mobile phone and go to the PC, you cannot restore the Mail.
Please explain to the user why this must be so. 
There is no explanation and there is also no reason Thunderbird does not behave to what IMAP provides.
(We cannot control what happens on mobile phones, so in this bug please stick to things that we might possibly be able to control.)

> Mails that are marked "deleted" in the mailbox must also be displayed and must not simply be ignored.

Given that you are familiar with "marked as deleted" enabled and have tested it, please tell me 
- messages are listed in Thunderbird?
- you can undelete them, mark as unread, star, tag, etc?
- you can move them?

In other words, what is there that you cannot do in Thunderbird with that setting enabled?
Because I don't like this setting on the PC.
The mails should be moved to the trash.

Because I don't like this setting on the PC.
The mails should be moved to the trash. It's even your default.

I just want a good reason from you why the mails marked as "\deleted" may not be displayed in all setting. Which IMAP "SPEC" does it forbid you ?

You can simply tell me the reason !
So that I also understand it and explain it to others.
Sorry, but you have not answered my questions.  

> why the mails marked as "\deleted" may not be displayed in all setting

You have a potential solution in the described mark as deleted setting.  And we are not going to display deleted mails in the Inbox as default behavior.  - so => WFM

If you want further explanations and solution that works for you, then I think you need to visit a forum.  https://support.mozilla.org/en-US/kb/get-community-support
Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Resolution: --- → WORKSFORME
If crossed out mode configured it might be possible to copy deleted mail to trash. New feature or enabled by hidden pref. Might be useful. (Entered on phone)
(In reply to michel from comment #0)
> 
> Steps to reproduce:
> 
> Flag a Mail as deleted and Thunderbird will not Display these Mails, when
> the config ist set to "When deleting a message" -> "Move to this folder".
> 
> That's just wrong.

Well, not really wrong. Notice the word "Move". When an imap move command occurs it actually copies the message to the destination folder (Trash or whatever) and then the server automatically expunges the message from the source folder. This is specified in the special imap "Move Extension" rfc6851.

Based on the imap log the reporter supplied in Bug 1477556, his Dovecot server definitely has MOVE capability so tb uses the imap move command when "deleting" messages to another folder. The server will report that the message was expunged from the source folder.

If the server does not support the MOVE capability, in that case the deleted message is imap COPY'd to the destination (Trash or whatever). If the server doesn't support MOVE and UIDPLUS, the message in the source folder remains marked with \deleted. If UIDPLUS capability is supported by the server, a imap "UID EXPUNGE" is done by tb on the message in the source folder, so only that message is expunged (not all \deleted messages in the folder).

Assuming that reporter Michel has control of his dovecot imap server, a possible workaround is to override the server's capability response so that neither MOVE nor UIDPLUS capability is advertised by the server. This will cause any message deleted to still be copied to Trash or wherever, but the message will remain in the source folder with flag \deleted and not be expunged.

With these changes (either in dovecot or tb) to view the deleted messages, just set the delete option from "move to folder X" to "just mark as deleted". All deleted but not expunged messages will appear crossed out.

Specifically in dovecot's /etc/dovecot/conf.d/20-imap.conf, set imap_capability, to this string (based on the capability response from the imap.log in Bug 1477556):

imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY SPECIAL-USE

This overrides the after-login capability response by removing only UIDPLUS and MOVE capabilities so tb never sees them. Doing this with my localhost based dovecot server, the deleted message(s) remain unexpunged. (Note: with this dovecot config change, messages copied or moved using the tb menus will also remain unexpunged in the source folder.)

A fix for this only in tb would require a special pref that would disable usage of MOVE and UIDPLUS capabilites, similar to how CONDSTORE can be disabled.
(In reply to michel from comment #10)
 > 
> Here's an example. ( It has nothing to do with "contact/task/workflow
> management system")
> 
> You are using a mobile phone with Mailapp and Thunderbird on your PC. On the
> mobile phone, mails should be marked as deleted (there are good reasons for
> this) on the normal PC in Thunderbird, but this should not be so.
> 
> If you delete now a Mail on the mobile phone and go to the PC, you cannot
> restore the Mail.
> Please explain to the user why this must be so. 
> There is no explanation and there is also no reason Thunderbird does not
> behave to what IMAP provides.

When you delete a mail with the phone client, are you sure it only marks the email as \deleted? I think it is probable that some (or most?) phone clients actually do the same as tb always tries to do when deleting: move the message to trash and expunge the original message. I haven't checked this so I could be wrong. But if I am right, tb would not be able to see the expunged emails no matter what.

Maybe I misunderstood your Comment 0 bug report:

>Flag a Mail as deleted and Thunderbird will not Display these Mails, when the config ist set to "When deleting a message" -> "Move to this folder".

I thought you mean after *tb* deletes an email. I think you actually mean when another client flags the message as \deleted without expunging it? If so, maybe you don't have a problem with how tb handles emails it deletes (by default, move to trash, expunge from original folder)? You are only wanting to see emails crossed-out that were marked \deleted by another client (like a phone app or the clean-up tool mention in the other bug) or if tb is not able to expunge the message from the source folder (server has no MOVE and no UIDPLUS capability). If so, you can probably ignore most of my suggestions in long comment 15.
(In reply to gene smith from comment #16)
> (In reply to michel from comment #10)
>  > 
> > Here's an example. ( It has nothing to do with "contact/task/workflow
> > management system")
> > 
> > You are using a mobile phone with Mailapp and Thunderbird on your PC. On the
> > mobile phone, mails should be marked as deleted (there are good reasons for
> > this) on the normal PC in Thunderbird, but this should not be so.
> > 
> > If you delete now a Mail on the mobile phone and go to the PC, you cannot
> > restore the Mail.
> > Please explain to the user why this must be so. 
> > There is no explanation and there is also no reason Thunderbird does not
> > behave to what IMAP provides.
> 
> When you delete a mail with the phone client, are you sure it only marks the
> email as \deleted? I think it is probable that some (or most?) phone clients
> actually do the same as tb always tries to do when deleting: move the
> message to trash and expunge the original message. I haven't checked this so
> I could be wrong. But if I am right, tb would not be able to see the
> expunged emails no matter what.
> 
> Maybe I misunderstood your Comment 0 bug report:
> 
> >Flag a Mail as deleted and Thunderbird will not Display these Mails, when the config ist set to "When deleting a message" -> "Move to this folder".
> 
> I thought you mean after *tb* deletes an email. I think you actually mean
> when another client flags the message as \deleted without expunging it?
Yes!
 If
> so, maybe you don't have a problem with how tb handles emails it deletes (by
> default, move to trash, expunge from original folder)? You are only wanting
> to see emails crossed-out that were marked \deleted by another client (like
> a phone app or the clean-up tool mention in the other bug) or if tb is not
> able to expunge the message from the source folder (server has no MOVE and
> no UIDPLUS capability). If so, you can probably ignore most of my
> suggestions in long comment 15.

Thunderbird should only show the messages marked as "deleted"!
That's it!  the subject is "Mails marked as deleted are not displayed"

The other tool is to move between different providers. Therefore the possibilities of the single IMAP server do not play a role, there is no move to another server.

Thanks
> If

Typo?

> Thunderbird should only show the messages marked as "deleted"!

I think you mean "Messages without the \deleted flag are displayed normally but Thunderbird should show messages with the \deleted flag set with the special markings (Red X and crossed out). This would be true when delete model configured is "move to trash". It is already true when delete model is "mark as deleted". It would never be true when delete model is "delete immediately" since messages with \deleted set are explicitly hidden in this mode but they are still present and marked with \deleted unless the folder is "compacted"."

I could see this being enabled with an optional pref setting. Not sure how hard this would be to do or if it would be acceptable to the project leaders (Wayne, Jorg, Magnus etc.).

But with tb as-is, if you really want to see the \deleted messages in a folder you can switch to "mark as deleted", move to another folder and then back to the folder of interest and you will see the \deleted messages (does not take effect until you switch folders). The emails marked as \deleted can be un-deleted now. This is probably not as automatic as you want, but it is a reasonable workaround and I think this is what Wayne was describing in Comment 11.
(In reply to gene smith from comment #18)
> > If
> 
> Typo?
> 
> > Thunderbird should only show the messages marked as "deleted"!
> 
> I think you mean "Messages without the \deleted flag are displayed normally
> but Thunderbird should show messages with the \deleted flag set with the
> special markings (Red X and crossed out). This would be true when delete
> model configured is "move to trash". It is already true when delete model is
> "mark as deleted". It would never be true when delete model is "delete
> immediately" since messages with \deleted set are explicitly hidden in this
> mode but they are still present and marked with \deleted unless the folder
> is "compacted"."

True

> 
> I could see this being enabled with an optional pref setting. Not sure how
> hard this would be to do or if it would be acceptable to the project leaders
> (Wayne, Jorg, Magnus etc.).

No, there's no need for a new option.

> 
> But with tb as-is, if you really want to see the \deleted messages in a
> folder you can switch to "mark as deleted", move to another folder and then
> back to the folder of interest and you will see the \deleted messages (does
> not take effect until you switch folders). The emails marked as \deleted can
> be un-deleted now. This is probably not as automatic as you want, but it is
> a reasonable workaround and I think this is what Wayne was describing in
> Comment 11.

You should take your time and think about the problem.
Your thoughts always seem to revolve around these configuration options.
No matter how Thunderbird is configured, it can always display mails marked as "/deleted" and then "red crossed" !

This is always correct and existing mails marked as "deleted" not to show is always wrong.

On the other hand, TB even delete the mails it doesn't display, that's of course the absolutely worst ever.
See this https://bugzilla.mozilla.org/show_bug.cgi?id=1477556

But I do realize that you are not in a position to rethink these things.

Much could be simplified in TB by simply deleting code that contains a much too complicated logic that is completely superfluous.


All you ever care about is these options:

1. "Move is to this folder:"
This is my default. 
Why does this option prohibit TB from displaying existing messages that are marked as "deleted" if they exist? 
In normal working only with tb this could not be happen, but if another program is attached to the same mailbox, then you can leave this to the user.

Why don't you show me the messages marked "deleted"? 
Please give a reasonable reason for this! 

2. "Just mark it as deleted"
Then tb deletes mails by marking them as "deleted".

3. "Remove it immediately"
Then they deleted right away.

!!! However, all this refers to the way what should happen when deleting a mail and not to what may be displayed !!!
As bad as I find it even "Outlook" behaves just like that here!

   Please correct that in Thunderbird !
That's really the last thing I'm gonna say!
(In reply to michel from comment #19)
> 
> You should take your time and think about the problem.
> Your thoughts always seem to revolve around these configuration options.
> No matter how Thunderbird is configured, it can always display mails marked
> as "/deleted" and then "red crossed" !
> 
> This is always correct and existing mails marked as "deleted" not to show is
> always wrong.

It's *not* always wrong, if only for historical and traditional reasons. Say I am a long-time tb user running with default setting and I delete an email on my phone. Say the app doesn't expunge the message but just marks it \deleted so with current tb version I will not see the message. Then I update tb and it now has the new feature you want enabled by default. Now I see a message I thought I had deleted with my phone marked out in tb with a big red X in front, "...what the...!", I say, "what does this mean? I better file a bug report".

> 
> On the other hand, TB even delete the mails it doesn't display, that's of
> course the absolutely worst ever.
> See this https://bugzilla.mozilla.org/show_bug.cgi?id=1477556

I'm still working on this too.

> 
> But I do realize that you are not in a position to rethink these things.
> 
> Much could be simplified in TB by simply deleting code that contains a much
> too complicated logic that is completely superfluous.

There's a lot of code and hard to say what is superfluous since I've only been working on tb (on my own time) for a couple years.

> 
> 
> All you ever care about is these options:
> 
> 1. "Move is to this folder:"
> This is my default. 
> Why does this option prohibit TB from displaying existing messages that are
> marked as "deleted" if they exist? 
> In normal working only with tb this could not be happen, but if another
> program is attached to the same mailbox, then you can leave this to the user.
>
 
Actually it could happen with an imap server that doesn't support MOVE and doesn't support UIDPLUS. This causes any message "moved to trash" to still be copied to trash but remain in the source folder marked \deleted. (I am simulating this right now by removing MOVE and UIDPLUS from my Dovecot server's capability response.)

> Why don't you show me the messages marked "deleted"? 
> Please give a reasonable reason for this! 

See above historical reason above.

> 
> 2. "Just mark it as deleted"
> Then tb deletes mails by marking them as "deleted".

Of course, this mode also shows the existing messages marked with \deleted to be visible but crossed out, not just the ones deleted by tb. That's why this seems like a reasonable workaround for your concerns.

> 
> 3. "Remove it immediately"
> Then they deleted right away.

This mode is the same as mode 2. except the \deleted messages are not visible. By "removed immediately" it really means they are removed from the list and not copied to trash folder. Any message marked with \delete is still there unless the folder is expunged (compacted). If you set back to mode 2 you will see the \deleted messages crossed out.

> 
> !!! However, all this refers to the way what should happen when deleting a
> mail and not to what may be displayed !!!

The modes *do* affect the way messages are displayed, as I described above.

>    Please correct that in Thunderbird !

I have made some experimental changes in my development tb build to "show messages marked with \deleted in all delete models (modes) except remove immediately". First I was able show the \deleted messages but they weren't crossed out. Then I found where the cross-out is enabled and set that and it worked. The difficult part though is getting the "undelete" function to work right. It seems to be working now for the most part but I ran into an existing and unfixed tb bug that can causes problems with the undelete and delete right-click functions: Bug 393523. This only occurs if you select a mixture of messages marked and not marked with \deleted.

> No, there's no need for a new option.

Again, for historical (and tb policy) reason cited above, if my change is incorporated into tb it would have to be an optional feature that would not change default tb behavior.
See Also: → 1644130

For possible future reference, this is the now bit-rotted work I did on this almost 2 years ago.

Assignee: nobody → gds
Assignee: gds → nobody
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: