Closed Bug 1856536 Opened 8 months ago Closed 3 months ago

IMAP folder which cannot contain messages cannot be deleted anymore. Thunderbird seems to send an "unsubscribe" command rather than a delete command to the server

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Version:
Thunderbird 115
Type:
defect

Tracking

(thunderbird_esr115 affected, thunderbird124 fixed)

Status:
RESOLVED FIXED
Milestone:
125 Branch
Tracking Flags:
Tracking Status
thunderbird_esr115 --- affected
thunderbird124 --- fixed

People

(Reporter: forum.news, Assigned: gds)

References

Details

Attachments

(4 files)

IMAP Log
7.62 KB, text/plain
Details
TB_imap_log.txt
7.68 KB, text/plain
Details
subscribe_dialog.png
21.05 KB, image/png
Details
Bug 1856536 - Allow NoSelect imap folders to be deleted as well as un-subscribed. r=mkmelin
48 bytes, text/x-phabricator-request
wsmwk
: approval-comm-beta+
Details | Review

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/118.0

Steps to reproduce:

I created an IMAP folder which can contain only folders (no messages). Such a folder cannot be deleted anymore. Thunderbird seems to send an "unsubscribe" command rather than a delete command to the server. With the account server setting "only show subscribed folders" deactivated, such a deleted folder shows up again after a Thunderbird restart.

This is the IMAP log sequence recorded by the server for such a deletion (folder: TestToDelete):

...
09:45:20.201: << 59 OK AUTHENTICATE completed.<cr><lf>
09:45:20.232: >> 60 capability<cr><lf>
09:45:20.232: << * CAPABILITY IMAP4rev1 AUTH=PLAIN X-MERCURY-1<cr><lf>
09:45:20.232: << 60 OK CAPABILITY complete.<cr><lf>
09:45:20.263: >> 61 list "" "Archiv/TestToDelete/*"<cr><lf>
09:45:20.263: << 61 OK LIST completed.<cr><lf>
09:45:20.294: >> 62 unsubscribe "Archiv/TestToDelete"<cr><lf>
09:45:20.294: << 62 NO That object is not subscribed.<cr><lf>
...

It is cearly visible that there is an

unsubscribe "Archiv/TestToDelete"

sent to the server.

Actual results:

Folder does not get deleted but just unsubscribed.

Expected results:

Folder should get deleted for good instead of just getting unsubscribed.

Component: Untriaged → Networking: IMAP
Product: Thunderbird → MailNews Core
Summary: IMAP folder cannot be deleted → IMAP folder which cannot contain messages cannot be deleted anymore. Thunderbird seems to send an "unsubscribe" command rather than a delete command to the server

Any updates on this? This is a really annoying issue. Any btw: renaming such folders does not work either.

Nothing?! This is really causing a lot of problems here...

Thanks!

With inbox selected, I create the folder structure t/gd/s. This ends up with un-selectable (grey) folder t under Inbox, un-selectable gd under t and selectable (black) s under gd. I can only copy a message into s, which I do.
I then delete s, then delete gd and then delete t, which all get moved to trash.
So this works for me.

I repeat the steps above but this time I delete the top un-selectable folder t. The whole structure t/gd/s gets moved into Trash.

In your case Archiv/TestToDelete, I'm not sure which is the un-selectable folder (grey) or if there are more folders below TestToDelete. I suppose you are saying TestToDelete is grey and it won't delete.

If you could record and attach an IMAP:4 level log it would help to correlate it with your server log and show exactly what TB is doing. You can find how to record an IMAP:4 log here: https://wiki.mozilla.org/MailNews:Logging

Well, the problem here is very simply. Lets forget about a folder "structure or hierarchy" at the moment. If I create a folder (one which you call un-selectable (grey)), I cannot delete it anymore. No matter if there is something underneath it. Thunderbird sends a unsubscribe command to my server rather that a delete command. I will try to create the requested logs later...

Konrad

Attached file IMAP Log — 
Hi, here is the requested log which also shows an "unsubscribe" command for my action to delete the Folder "TestToDelete3" which is such a "grey" folder:



Also, keep in mind that my server does not support folders in folders so deleted folder does not show up in the trash folder. Its gets deleted completely...

Hi, attached is the requested log which also shows an "unsubscribe" command for my action to delete the Folder "TestToDelete3" which is such a "gray" folder

Also, keep in mind that my server does not support "folders in folders" so deleted folder can not show up in the trash folder. They get deleted completely...

Attached file TB_imap_log.txt —

Or is the unsubscribe command executed always if a folder gets deleted to make sure that it is fully cleaned up?! If that is the case, what is the real delete command? Maybe the actual delete command and problem is this here:

I/IMAP 17afbcee600:xxx.xxx.xxx:A:ProcessCurrentURL:imap://yyy@xxx.xxx.xxx:993/delete%3E/Archiv/TestToDelete3: = currentUrl
...
D/IMAP URL failed with code 0x80550021 (imap://yyy@xxx.xxx.xxx:993/delete%3E/Archiv/TestToDelete3)

Ok, I'm seeing it too.
Some questions:

Also, keep in mind that my server does not support "folders in folders" so deleted folder can not show up in the trash folder. They get deleted completely...

So I suppose you are not deleting to trash but are using either "just mark as deleted" or "delete immediately" under "server settings?

What exactly is your server type that doesn't support (or has disabled) folder in folder?

If your server doesn't support folder in folder, how does it allow Archiv/TestToDelete?

Or is the unsubscribe command executed always if a folder gets deleted to make sure that it is fully cleaned up?! If that is the case, what is the real delete command?

Yes, I think it is alway done after the folder is "deleted". But a true imap "delete" only occurs with "delete immediately" or "just mark as deleted". With "delete to Trash" it just renames the deleted folder (or folder tree) to be under trash using imap "rename" command.

If you subscribe the "gray" folder, does it allow you delete it and not just unsubscribe it? That seemed to work for me but I need to check again.

Hi,

I'm actually using the delete option "move to trash" under server settings and I disabled the advanced server settings option "server supports folders which can contain folders and messages" since my server does not support that. This is what I meant in my last post instead of "my server does not support folders in folders". Sorry for that misunderstanding.

Reading through your post makes me think that the "move to trash" delete option could actually cause this problem. Since the trash folder is a "normal" folder which can contain messages, it cannot contain folders (as my server does not support that). So the deleted folder cannot be renamed and put into the trash folder, right? Maybe Thunderbird should figure that out and case of a server not supporting that, it should initiate a true imap "delete" command rather than just renaming it to be under trash. Does that make any sense?

Thanks
Konrad

btw: changing to "delete immediately" does not solve the problem (just did a test) and I think that Thunderbird actually knows my server does not support it because the popup confirming the deletion says something like "are you sure you want to delete this folder permanently". Anyhow, it still does not send that imap delete no matter what I try.

About your suggestion to subscribe the folder before deleting it: Such folders cannot be subscribed. There is no checkbox shown for such "gray" folders. Only regular folders (which can only contain messages) can be subscribed here. Screen shot attached...

Konrad

Attached image subscribe_dialog.png —

Wrote this before seeing comment 11:
Well, I see the issue even with "just mark as deleted" and it just unsubscribes the folder. But when I re-subscribe it, I can actually delete it.
Yes, if you have that advanced setting set, TB should not allow delete of folders by rename into Trash, it does seem.
Still curious, if you can say, what is your imap server type that doesn't allow creating subfolders? Is it an intentional server configuration setting or just a limitation of that particular server?

About your suggestion to subscribe the folder before deleting it: Such folders cannot be subscribed. There is no checkbox shown for such "gray" folders. Only regular folders (which can only contain messages) can be subscribed here. Screen shot attached...

Ok, interesting. When I create the "gray" folders I do see subscribe checkboxes. Were yours created via TB or some other way?
Also, I think there is supposed to be a \noselect attribute on those folder from the imap LIST command but I don't see that in your attached logs. I need to check my logs closer to see if I see \noselect in gray folder.

With both cyrus and dovecot imap servers, those gray folders still show up even after they are deleted when accessed purely with telnet (no TB involvement). They show up using the LIST-EXTENDED command:

. list (subscribed) "" "*" return (special-use)

like this as subscribed and "non-existent":

:
* LIST (\Subscribed) "." test-delete.gmail-inbox.test-move
* LIST (\Subscribed) "." test-delete.sub2
* LIST (\Subscribed) "." test-delete.sub1
* LIST (\Subscribed \NonExistent) "." foo
* LIST (\Subscribed \NonExistent) "." foo.bar
. OK List completed (0.044 + 0.000 + 0.043 secs).

Both foo and foo.bar show up in TB subscription list as gray with no checkbox to subscribe (even though listed as "subscribed). After folder discovery at TB startup they appear as grey in tb.
I think a way to delete them in the server is to re-create them as normal folders in TB, i.e., foo and then foo.bar and then you can delete them. After doing this in TB for dovecot account, I don't see them in TB's subscription list or with the above "list" command via telnet.

Reading the RFC for imap and list extended, it seems the base "list" command should show them as \noselect but with the extended list, they show up \NonExistent so \noselect and \nonexistent mean the same thing.

I don't know if your server supports LIST-EXTENDED capability or if that even matters.
Of course, to delete folders you will need to switch off of "delete to trash" mode if your server truly can't create subfolders (of Trash).

P/S: I still don't understand how you have folder like Archiv/TestToDelete if your server doesn't support subfolders. Or are you saying you are trying to prevent TB from creating subfolder by unchecking the advanced setting "Server supports folders that contain sub-folders and messages"?

Well, your responses are very technical ;-) and I have a hard time following the part with the "list command" section. To me, Thunderbird should send a true IMAP "DELETE" command if I want to delete a folder. Isn't it that simple?

About my server: I'm using "Mercury Mail Server" (https://www.pmail.com/overviews/ovw_mercwin.htm).

And to clarify more detailed: This server does support folders in folders. But it does not support folders which can contain sub folders AND messages at the same time. So while creating a (sub-)folder, I need to make a choice if the new folder is "a folder which can only contain sub folders (would be a gray/italic one)" or if it is "a folder which can only contain messages (would be a normal/black one)". There is this advanced setting you mentioned -> "Server supports folders that contain sub-folders and messages". I have deactivated it since my server does not support that. Thunderbird can delete these normal/black folders without any problems here. Only these gray folders cannot be deleted.

FWIW: Other mail clients like Roundcube happily delete such gray folders. It's only Thunderbird who wants to unsubscribe them rather than actually deleting them.

Konrad

You asked how I created these gray folders: I created them with Thunderbird

Hi,

Not sure if it would help at this point but the developer responsible for the Mail Server "Mercury" offered to help us in case there are any questions or doubts about how the server is behaving in this case. Let me know if you think that this would make sense or if it would help us to solve this issue.

Thanks
Konrad

About my server: I'm using "Mercury Mail Server" (https://www.pmail.com/overviews/ovw_mercwin.htm).

...the developer responsible for the Mail Server "Mercury" offered to help us in case...

Yes, contact with the server developer would be helpful.

Another possibility is to provide me with a temporary test imap account on a "Mercury" server so I can see exactly what transpires using TB's logging and a network sniffer (wireshark).

P/S: Long ago (maybe 1990s) where I worked they first used a "Pegasus Mail" system before they "standardized" on Microsoft Exchange. I didn't know Pegasus Mail was still around! Since I've been supporting TB imap (>5 years), you are the first I've seen using their "Mercury" server.

Konrad,
Another piece of helpful information would be to record TB's IMAP:4 log again and from the log show me the last "CAPABILITY" response you see. It will look something like this example:

2024-02-16 18:51:16.594106 UTC - [Parent 323597: IMAP]: I/IMAP 7f48aaf04c00:mail.your-server.de:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE QUOTA

The length and content will, of course, be different with your Mercury server. I don't need to see the full log file, just this small part of it.
Thanks.

From Comment 18:

Another possibility is to provide me with a temporary test imap account on a "Mercury" server so I can see exactly what transpires using TB's logging and a network sniffer (wireshark).

Reporter forum.news provide the test account so I could see exactly what is going on.

It looks to me like it would be OK for TB to just delete the "container" folders (marked \NoSelect by Mercury). Right now TB just looks at them and says, "it doesn't really exist so I'll just unsubscribe it". This seems to work OK for the 3 "popular" servers I tested with (Dovecot, Cyrus and Courier) but not for Mercury. But I don't see anything in the RFC that saying deleting \NoSelect folders is illegal and, in fact, RFC 3501 shows an example of doing just that, deleting a \NoSelect folder: https://datatracker.ietf.org/doc/html/rfc3501#section-6.3.4 , specifically:

               C: A686 LIST "" *
               S: * LIST (\Noselect) "/" foo
               S: A686 OK LIST completed
               C: A687 DELETE foo
               S: A687 OK DELETE Completed

I can fix this in TB so it goes ahead and deletes the \NoSelect folder and then also unsubscribes (which it always does after deleting a folder). When I made this change my examples of the reporter's problem folders (gray/italic) were permanently deleted, even after a restart:

diff --git a/mailnews/imap/src/nsImapProtocol.cpp b/mailnews/imap/src/nsImapProtocol.cpp
--- a/mailnews/imap/src/nsImapProtocol.cpp
+++ b/mailnews/imap/src/nsImapProtocol.cpp
@@ -6778,23 +6778,27 @@ nsresult nsImapProtocol::SetFolderAdminU
   return rv;
 }
 // returns true is the delete succeeded (regardless of subscription changes)
 bool nsImapProtocol::DeleteMailboxRespectingSubscriptions(
     const char* mailboxName) {
   bool rv = true;
   if (!mailboxName) {
     return false;
   }
-  if (!MailboxIsNoSelectMailbox(nsDependentCString(mailboxName))) {
+// temp test gds: what happens if you go ahead and delete it?
+//  if (!MailboxIsNoSelectMailbox(nsDependentCString(mailboxName))) {
     // Only try to delete it if it really exists
     DeleteMailbox(mailboxName);
     rv = GetServerStateParser().LastCommandSuccessful();
-  }
+    if (!rv) printf("gds: failing here for folder = %s\n", mailboxName);
+    if (MailboxIsNoSelectMailbox(nsDependentCString(mailboxName)))
+      rv = true;  // ignore error if NoSelect
+//  }
 
   // We can unsubscribe even if the mailbox doesn't exist.
   if (rv && m_autoUnsubscribe)  // auto-unsubscribe is on
   {
     bool reportingErrors = GetServerStateParser().GetReportingErrors();
     GetServerStateParser().SetReportingErrors(false);
     Unsubscribe(mailboxName);
     GetServerStateParser().SetReportingErrors(reportingErrors);
   }

I need to test this some more and possibly provide the reporter with a try build to test with.

Reporter Konrad/forum.news,
Here's the win64 installer for my "try" build which is my patch above applied to the current beta, 123.0b5 code. You can run this along side your existing TB install.
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/GKGr0lnxSIGHD42vSIeEow/runs/0/artifacts/public/build/install/sea/target.installer.exe
This is the "optimized" build and not a slower "debug" build.

You can see the complete "try" build here: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=77eca4ff7939662d7bcd35d5987428592ab7a3ff&selectedTaskRun=GKGr0lnxSIGHD42vSIeEow.0
which includes debug and optimized builds for window and linux (both 64 bits).

Note: When/if you run this, it will say "Daily" in Help | About but will show version 123.0b5 (5th beta of version 123). Also, AFAIK, "try" builds are always en-US localized.

Hi,

Tested the "try" build and it looks like your fix it worked as expected!

Thanks a lot!

Konrad

Although, I have to admit that in my mind these "popular" servers do respond to an "unsubscribe" command initiated by an IMAP client in a very interesting way if they really delete the folder during that command. Why such an unsubscribe should result in a delete operation is beyond my understanding but what Mercury is doing in this case feels "more correct" (even though the unsubscribe command is most likely not specified in detail).

Konrad

See Also: → 1882013
Assignee: nobody → gds
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

(In reply to forum.news from comment #23)

Although, I have to admit that in my mind these "popular" servers do respond to an "unsubscribe" command initiated by an IMAP client in a very interesting way if they really delete the folder during that command.

Unsubscribe command, by itself, doesn't delete a normal (selectable) folder on any server, AFAIK. What "delete" and "unsubscribe" does on a NoSelect folder seems to be server dependent.

So all the patch does is allow whichever folder you select to receive the imap "delete" command no matter its type (NoSelect or selectable). And even if the "delete" fails, it goes ahead and does the unsubscribe (which typically fails too but doesn't matter).

FYI, if you have a 3 level tree like this:

t1
  t2
     leaf

Even if t1 and t2 are NoSelect (gray/italic) and if you select t1 to delete, tb does this:

imap delete t1/t2/leaf
imap unsub t2/t2/leaf
imap delete t1/t2
imap unsub t1/t2
imap delete t1
imap unsub t1

TB doesn't support deleting only the top or an intermediate folder. E.g., you can't delete just t2 in the above tree. Imap RFC requires that deleting just t2 removes the messages from t2 and (server?) sets t2 NoSelect and leaf stays the same. TB doesn't support this operation so with TB when t2 is deleted, t2 and leaf are both deleted (and unsubscribed) leaving t1 as is.

Re: https://datatracker.ietf.org/doc/html/rfc3501#section-6.3.4

(In reply to forum.news from comment #11)

... I think that Thunderbird actually knows my server does not support it [deleting folders to Trash] because the popup confirming the deletion says something like "are you sure you want to delete this folder permanently".

Yes, with your server (Mercury) an attempt to delete a folder will always give that prompt even though you have configured TB to delete to Trash. That's because the server reports all folders, including Trash, as having the \NoInferiors flag which tells TB that sub-folders of any folder can't be created, including, of course, Trash.

Keywords: checkin-needed-tb
status-thunderbird124: --- → affected
status-thunderbird_esr115: --- → affected
Target Milestone: --- → 125 Branch

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/07c555c861a3
Allow NoSelect imap folders to be deleted as well as un-subscribed. r=mkmelin

Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Keywords: checkin-needed-tb
Resolution: --- → FIXED

It is interesting that a server can decide what a "delete" or "unsubscribe" does on a NoSelect folder but anyhow, I 'm really glad for this fix. With which release will it be shipped?

Thanks
Konrad

Wayne, Could this also be lifted to beta the same as the related bug 1882013? They were reported at about the same time by the same user, forum.news.
-gene

Comment on attachment 9382746 [details]
Bug 1856536 - Allow NoSelect imap folders to be deleted as well as un-subscribed. r=mkmelin

[Triage Comment]
Approved for beta to go with bug 1882013

Attachment #9382746 - Flags: approval-comm-beta+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: