547455 - Unable to delete folders from my IMAP server

Status: RESOLVED DUPLICATE of bug 301714

(Thunderbird :: Folder and Message Lists, defect)

Description

[Ian Graham]

I am trying to delete an IMAP folder - but it doesn't go away. 
Steps to recreate (for me):
1. Access IMAP server via my webmail account, and create a new 
   folder (e.g. "test-mail")
2. Access IMAP server using TB. new "test-mail" folder appears in 
   folder list associated with that server.
3. Delete folder by dragging it to the Trash folder for that IMAP server.
4. Select 'empty trash"  -- folder is removed from the TB trash 
   folder list.
5. Shutdown and restart TB.
6. Folder 'test-mail' is still listed in the Trash folder. 
   If I select that folder I get the popup error message: "The 
   current command did not succeed.  The mail server responded:
   Can't open mailbox Trash/test_folder: no such mailbox"
7. Verified using Webmail account that the folder is still 
   present on the IMAP server.

My IMAP server settings:
   [*] Check for new messages at startup
   [*] Check for new messages every "5" minutes
   [*] When I delete a message, move it to the "Trash" folder.
   [*] Empty Trash on Exit
  Advanced:
   [*] Show only subscribed folders
   [*] Server supports folders that contain sub-folders and messages.\
   [*] Use Idle command if server supports it.

I hesitate to experiment further as I have built up a rather long list of IMAP folder names I can't get rid of ;-/

Comment 1

[David :Bienvenu]

An imap protocol log would give some clues as to what's happening. My guess is that you're subscribed to the folder, but it doesn't exist.

https://wiki.mozilla.org/MailNews:Logging

Comment 2

[Ian Graham]

Attachment: IMAP log upon startup -

Thanks David. I have attached the IMAP log. For clarity I will also attach a screen shot of the TB folder view (showing the IMAP folders), and also from my Webmail client (which gives  an independent and accurate view of the IMAP folders).

Comment 3

[Ian Graham]

Attachment: Screen capture of Folders view (smart folders)

Comment 4

[Ian Graham]

Attachment: Screen capture of Webmail client IMAP folders view

Comment 5

[David :Bienvenu]

Ian, can you go into the imap subscribe ui (File | subscribe) and unsubscribe to the folders that don't exist?

Comment 6

[Ian Graham]

Thanks David - that does the trick removing the non-existent folders. 

FYI, I was not aware of the subscribe UI. I figured visual editing of the folder tree would do what I wanted.

It would seem that visual editing / deleting of folders in the IMAP folder view UI is not triggering appropriate unsubscribe/subscribe actions??

I see this can be an interesting design problem: should deleting folders on TB by moving them to the trash 'unsubscribe' them or 'delete' them from the IMAP server? Or, rather, how to I allow both unsubscribe and delete actions in a way that is easily understood by users and not likely to lead to disaster. 

Having used it, I am not fond of the subscribe UI -- the subscribe / unsubscribe buttons seem unnecessary, and indeed confusing, given the way the UI works. (just unselect the ones you don't want to subscribe to and press 'OK').

Comment 7

[David :Bienvenu]

Thunderbird does try to unsubscribe when emptying the trash or moving folders (moving folders does a rename and a subscribe/unsubscribe, iirc). But it seems to have failed in your case.

Comment 8

[Ian Graham]

Ok. I will try repeating the exercise - creating the new folder, dragging to delete it, etc. and will log all the IMAP traffic during that to see what happens.

Comment 9

[Shane Caraveo (:mixedpuppy)]

(In reply to comment #7)
> Thunderbird does try to unsubscribe when emptying the trash or moving folders
> (moving folders does a rename and a subscribe/unsubscribe, iirc). But it seems
> to have failed in your case.

FYI, I am working on an extension that creates folders, and in testing I need to repeatedly delete/recreate them, I am running into this issue constantly.  One side effect I notice is that listening for folder delete notifications via nsIMsgFolderNotificationService shows that I do not get a delete notification, even though the folder is deleted on the imap server.  

Also, often, when deleting via the context menu on a folder results in the folder remaining outside the trash as though it is not deleted, though it is actually moved to trash on the imap server.  In that situation, empty trash does delete it from the imap server.

Comment 10

[Ian Graham]

Attachment: IMAP log for startup and drag 'n drop deletion of IMAP folder

So I did the followin test by reproducing the original test case.
1. Access IMAP server via my webmail account, and create a new 
   folder (e.g. "test-folder")
2. Start up TB with IMAP logging enabled. 
3. Access IMAP server using TB. new "test-folder" folder appears in 
   folder list associated with that server.
4. Delete folder by dragging it to the Trash folder for that IMAP server.
5. Select 'empty trash"  -- folder is removed from the TB trash 
   folder list.
6. Shut down TB.
7. Viewed folder list using webmail account - "Test-folder" is still
   present as a top-level folder.
8. relaunched TB. after logging into IMAP server TB thinks the "test-folder"
   is in the remote server "Trash" folder. I am unable to view that
   remote folder content: TB responds with a popup error, saying the 
   folder does not exist.

Comment 11

[David :Bienvenu]

As WADA has said in an other bug, the issue has to do with the trailing / on the folder name. I wonder if this part of the log indicates the problem, but I'd need to compare it with a server that delete works on:

71780]: 66af400:mailbox3.utcc.utoronto.ca:S-Trash:CreateNewLineFromSocket: 40 OK LIST completed

3876[5771780]: 66af400:mailbox3.utcc.utoronto.ca:S-Trash:SendData: 41 list "" "Trash/test-folder/*"

3876[5771780]: ReadNextLine [stream=193b9c8 nb=34 needmore=0]
3876[5771780]: 66af400:mailbox3.utcc.utoronto.ca:S-Trash:CreateNewLineFromSocket: * LIST () "/" Trash/test-folder/

3876[5771780]: ReadNextLine [stream=193b9c8 nb=22 needmore=0]
3876[5771780]: 66af400:mailbox3.utcc.utoronto.ca:S-Trash:CreateNewLineFromSocket: 41 OK LIST completed

3876[5771780]: 66af400:mailbox3.utcc.utoronto.ca:S-Trash:SendData: 42 delete "Trash/test-folder/"

3876[57

Comment 12

[Ian Graham]

Is there any way, in TB, of identifying the server software TB is connected to?

Comment 13

[David :Bienvenu]

No, though the server greeting is in the log strongly indicates it's a UW server.

Comment 14

[WADA:World Anti-bad-Duping Agency]

Ian Graham, thanks for log. Check next case with logging enabled.
  1. Via Tb, create folder abc/  (with trailing /) => OK
  2. Tb probably issues "subscribe abc/" instead of "subscribe abc",
     because server returns "abc/" in create or list response.
It's known bug.
> Bug 301714 subscribe, unsubscribe and delete add extra '/' to mailbox name

> S-Trash:SendData:                40 list "" "Trash/*"
> S-Trash:CreateNewLineFromSocket: * LIST () "/" Trash/
> S-Trash:CreateNewLineFromSocket: * LIST () "/" Trash/test-folder
> S-Trash:CreateNewLineFromSocket: 40 OK LIST completed
> S-Trash:SendData:                41 list "" "Trash/test-folder/*"
> S-Trash:CreateNewLineFromSocket: * LIST () "/" Trash/test-folder/
> S-Trash:CreateNewLineFromSocket: 41 OK LIST completed
> S-Trash:SendData:                42 delete "Trash/test-folder/"
> S-Trash:CreateNewLineFromSocket: 42 OK DELETE completed

Server should return "NO no such mailbox", if absolutely RFC compliant.
Why "OK is returned" is explained by bug 301714 comment #7.

> S-Trash:SendData:                43 unsubscribe "Trash/test-folder/"
> S-Trash:CreateNewLineFromSocket: 43 NO Not subscribed to mailbox Trash/test-folder/

If UW-IMAP has quirks for Tb's bug like Dovecot and the quirks is enabled, this error will be avoided. See bug 301714 comment #5.

DUP of Bug 301714.

Comment 15

[Ian Graham]

Ok, I will do that test.

Some other info. The IMAP server and version is: UW IMAP 2006g.

My webmail account UI will not let me create an IMAP folder with a trailing slash in the folder name: it ignores the slash entirely. 

That is, if I create a folder "test" and then try to create a second folder named "test/" the Webmail client responds with the message "Unable to create folder "test" - folder already exists."

Comment 16

[Ian Graham]

Attachment: IMAP log for creation of remote IMAP folders.

Here is what is captured in the IMAP log:

1. On TB, select IMAP server folder, right click menu -> "New Folder"
2. enter name "abc/" and press enter.
3. New folder with name "abc" is listed.
4. repeat 1-3 for a folder name without a trailing slash ("def")
5. folder named "def" is listed.
6. If I now use my webmail client, it lists two folders (abc, and def) - but 
   abc is listed as if it contains subfolders (I will attach a screen capture
   "create-imap-folders.png" to show what I mean)

Comment 17

[Ian Graham]

Attachment: Webmail client view of folders created by attachment 429334

This attachment shows the webmail client view of the folders created by the test case in the previous comment. Note "abc' is displayed as an expandable folder (notionally with subfolders, although there are no subfolders there), while 'def' does not.

Comment 18

[Ian Graham]

Attachment: IMAP log for 'drag and drop' deletion of folder abc/ and def

Here's the log created when I try and delete these folders.
1. Start TB (logging to "delete-abc+def.log") 
2. Delete folder 'abc' by dragging to trash.
3. Delete folder 'def' by dragging to trash.
4. Empty trash folder - folders are removed.
5. Shut down TB. 

6. Check with webmail client - both folders are now gone from the 
   IMAP server.

7. Restart TB (new log file - restart-after-delete-abc+def.log)
8. On startup and after login to the IMAP server, TB  shows 
   'abc' and 'def' folders in the IMAP trash folder.
9. Shut down TB.

Comment 19

[Ian Graham]

Attachment: IMAP log for session subsequent to deleting abc and def folders

IMAP log: TB restart after drag and drop deletion of 'abc/' and 'def' folders.

Comment 20

[Ian Graham]

Correction to comment #1 - TB actually does delete the folder from the IMAP server - I just have to be sure to log out from the webmail client to flush the file delete operations. Here is a scenario description I just ran to clarify this:

1.  Login to webmail-based IMAP server client 
2.  Create new folder named 'test'.
3.  Login to IMAP server using TB -- new 'test' folder is listed
    in IMAP server folder view.
4.  In TB, Delete 'test' folder by dragging to IMAP trash folder.
5.  Go back to webmail client and refresh folder view (may need to
    log out and log back in). 'test' folder is now found in the
    'trash' folder. 
6.  Go back to TB and empty trash on TB client.
7.  Go back to webmail client and refresh folder view (may need to 
    log out and log back in).  On  webmail client,
    'test' folder is gone - no longer found in 'trash' folder
    (or anywhere else).
8.  Go back to TB and look to the IMAP server 'subscribe' window: this 
    still lists a subscribed 'test' folder as a subfolder of 'Trash' 
9.  shut down TB and restart TB
10. on TB, 'test' folder magically re-appears in IMAP trash folder.

Comment 21

[WADA:World Anti-bad-Duping Agency]

> 6.  Go back to TB and empty trash on TB client.
>(snip)
> 8.  Go back to TB and look to the IMAP server 'subscribe' window: this 
>     still lists a subscribed 'test' folder as a subfolder of 'Trash' 

It may be next.
- Tb issues "delete Trash/xxx" first, and issues "unsubsribe Trash/xxx"
  at step 8, probably.
- IMAP server doesn't remove Trash/xxx from subscription list,
  because Trash/xxx doesn't exist any more by delete folder step of
  "Empty Trash" operation.
Following may be relevant.
- Tb issues "rename xxx Trash/xxx" if "delete of a folder xxx with IMAP delete
  model==move to trash".
As rename operation, unsubscribe by Tb at step 6 may produce problem at server, especially when user logins via Web interface(similar to other mail client via IMAP like Tb generally, but user of Web Mail is special and different from interface of IMAP supported by Web Mail).

Ian Graham, does your server return "OK" to "unsubscribe" by Tb at step 6?

Comment 22

[Ian Graham]

Attachment: IMAP log corresponding to comment 20, items 3 through 8

Sorry for the delay - was on vacation :)
I've attached my IMAP log of steps 3. through 8. from comment 20. That should tell you if the server returns OK at step 6 (I do not know enough to interpret the log file).

Comment 23

[WADA:World Anti-bad-Duping Agency]

RENAME rename "test" "Trash/test" was used.
  (1) You looks to check with "test" instead of "test/" this time.
      (Treatment of "test/" depends on IMAP server)
  (2) You looks to use IMAP delete model of "move to trash".
> 30 rename "test" "Trash/test"
> 30 OK RENAME completed
> 31 subscribe "Trash/test"
> 31 OK SUBSCRIBE completed
> 32 unsubscribe "test"
> 32 OK UNSUBSCRIBE completed
I can't see any problem around "delete folder" operation on "test" folder.

Your server returns "* NO Checkpoint ignored on readonly mailbox" to STATUS command for any folder. Following is one after above delete folder oeration.  
> 34 STATUS "Trash" (UIDNEXT MESSAGES UNSEEN RECENT)
> * STATUS Trash (MESSAGES 0 RECENT 0 UNSEEN 0 UIDNEXT 404776)
> * NO Checkpoint ignored on readonly mailbox
> 34 OK Check completed
Is this "* NO Checkpoint ignored on readonly mailbox" seen even when access by Tb only? (no access via Web Mail Interface, no access by other mail client)

Comment 24

[Ian Graham]

I don't quite understand the question - I have no way of checking my webmail client to know / see what IMAP responses are being sent to it....

Comment 25

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #24)
> I don't quite understand the question - I have no way of checking my webmail
> client to know / see what IMAP responses are being sent to it....

I wanted to say 'Is this "* NO Checkpoint ignored on readonly mailbox" seen in IMAP log?'. "Checkpoint request from IMAP server invoked by STATUS command which Tb issued" may be locked out by concurrent access from Web Mail client and/or other IMAP mail client.

Comment 26

[Ian Graham]

I see. For the log file attached at comment 22 I disconnected / shut down the webmail client (and shut down the web browser) after completing steps: 2, 5, and 7. So the webmail client was completely shut down before every TB access of the IMAP server. Does that help?

Comment 27

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #26)
It sounds that "Mail receiver + Web mail system" itself is other client for IMAP. As Web mail system accesses/updates DB for mail data, access from IMAP to the DB may be forced "readonly mailbox" in the message, upon checkpoint by IMAP server which is requested by STATUS command.
> http://www.washington.edu/imap/documentation/mixfmt.txt.html
> UW IMAP Server Documentation
> 5. Checkpoint and expunge
> Checkpoint is identical to expunge, however it skips the step of expunging
deleted messages.
> The index and status files are locked exclusive.
> If expunging, all deleted messages are expunged from the index and announced
> via mm_expunged().  The message data is notremoved at this time.

I think the "* NO Checkpoint ignored on readonly mailbox" won't produce any problem. It simply says "checkpoint by your STATUS command was temporarily impossible because other one is also using the mail box", although it occurs always instead of sometimes if IMAP system on Web mail system or Web mail system on IMAP system.

Comment 28

[Jason Oster (:Parasyte)]

(In reply to comment #23)
> RENAME rename "test" "Trash/test" was used.
>   (1) You looks to check with "test" instead of "test/" this time.
>       (Treatment of "test/" depends on IMAP server)
>   (2) You looks to use IMAP delete model of "move to trash".
> > 30 rename "test" "Trash/test"
> > 30 OK RENAME completed
> > 31 subscribe "Trash/test"
> > 31 OK SUBSCRIBE completed
> > 32 unsubscribe "test"
> > 32 OK UNSUBSCRIBE completed
> I can't see any problem around "delete folder" operation on "test" folder.

I think this is kind of strange for Thunderbird (and the IMAP server, probably) Shouldn't it unsubscribe from "test" before the rename?  (Strange the IMAP server responds OK to unsubscribing to "test" after it's already been renamed!)

Similarly, Thunderbird should unsubscribe from "Trash/test" before the delete, to avoid this:

> 40 delete "Trash/test/"
> 40 OK DELETE completed
> 41 unsubscribe "Trash/test/"
> 41 NO Not subscribed to mailbox Trash/test/

A change to the order-of-operations would fix this bug?

Comment 29

[Jason Oster (:Parasyte)]

Also, the trailing slash in the delete and unsubscribe; oops!  See: Bug 537626 Comment #8

Comment 30

[Wayne Mery (:wsmwk)]

Ian, do you stillll see thisnow that bug 301714  is fixed?

Comment 31

[Ian Graham]

I repeated the test case described in Comment #1 and can reproduce this behavior. Note also my Comment #20 - turns out that step 3 in comment #1 does delete the folder from the Imap server: but TB seems to 'forget' this the next time it starts up :(

Comment 32

[WADA:World Anti-bad-Duping Agency]

Bug 301714 is already fixed. So closing as dup of bug 301714. For problem other than "Tb doesn't remove trailing / in LIST response from delete command", open separate bug, please.

Comment 33

[Priya Sharma]

The folder "Trash" was not created. This is what the server said: Client tried to access nonexistent namespace. (Mailbox name should probably be prefixed with: INBOX.).......http://mytecharticle.com/?p=668