537626 - imap maildir folders are undeletable

Status: RESOLVED DUPLICATE of bug 301714

(MailNews Core :: Networking: IMAP, defect)

Description

[Harald Dunkel]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2b5) Gecko/20091218 Firefox/3.6b5
Build Identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091208 Thunderbird/3.0

If I move a imap maildir folder to Trash and try to flush it, then it is back on the next restart


Reproducible: Always

Steps to Reproduce:
1.move a mail folder to Trash
2.click on [Empty Trash] for this account
3.terminate Thunderbird to disconnect from the IMAP server and let it do its job

Actual Results:  
When I am back in Thunderbird the deleted folder and all EMails inside are still there.

Expected Results:  
the folder and all EMails inside should be gone

I reproduced this using Dovecot. Configuration:

# 1.2.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32.2 x86_64 Debian squeeze/sid 
log_timestamp: %Y-%m-%d %H:%M:%S 
protocols: imap imaps pop3 pop3s managesieve
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_executable(managesieve): /usr/lib/dovecot/managesieve-login
mail_privileged_group: mail
mbox_write_locks: fcntl dotlock
mail_executable(default): /usr/lib/dovecot/imap
mail_executable(imap): /usr/lib/dovecot/imap
mail_executable(pop3): /usr/lib/dovecot/pop3
mail_executable(managesieve): /usr/lib/dovecot/managesieve
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve
namespace:
  type: private
  separator: /
  location: maildir:~/Maildir:LAYOUT=fs
  inbox: yes
  list: yes
  subscriptions: yes
auth default:
  passdb:
    driver: pam
  userdb:
    driver: passwd

Comment 1

[Harald Dunkel]

Attachment: imap log file 1

This IMAP log file shows how the folder was created, some EMail was put inside, and then how the folder was removed.

Comment 2

[Harald Dunkel]

Attachment: imap log file 2

This is the imap log file of the second Thunderbird session, showing that the folder is still there.

Comment 3

[Harald Dunkel]

Please note the "Invalid mailbox name" replied by the imap server in log file 1. Thunderbird does not show this to the user on a click on [Empty Trash].

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #1)
> imap log file 1

> 10 list "" "Trash/*"
> * LIST (\HasChildren) "/" "Trash/" <== trailing / shouldn't be returned 
> * LIST (\HasNoChildren) "/" "Trash/subfolder2delete"
> 11 list "" "Trash/subfolder2delete/*"
> * LIST (\HasNoChildren) "/" "Trash/subfolder2delete/" <== trailing / shouldn't be returned
> 11 OK List completed.
> 12 delete "Trash/subfolder2delete/"
> 12 NO [CANNOT] Invalid mailbox name

Bug of IMAP server. IMAP server has to return valid folder name which can be used by other command such as delete.

> I reproduced this using Dovecot. Configuration:
>   separator: /
> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN] Dovecot ready.

Can you reproduce problem with default separatoe of "." of Dovecot?

AFAIR, I saw bug(s?) for same problem. I don't remember report for Dovecot or not.
Quirks by Tb is mandatory for tolerance with server's bug?

Comment 5

[Timo Sirainen]

Dovecot is behaving correctly. See bug 517054 for a similar problem reported against UW-IMAP. Note that UW-IMAP is created by Mark Crispin, who created the IMAP protocol. The relevant reply from him about this issue:

> 5 list "" "Trash/emu2000/*"
> * LIST (\HasNoChildren) "/" "Trash/emu2000/"
(B) This response means folder named <null> under emu2000 under Trash.
    Folder name in response has to be correct name which can be specified in
    SELECT commant etc. So this is apparently server side bug.

There is no such thing as a "folder named <null>".  The LIST response here
simply confirms that Trash/emu2000 exists as a directory.  Otherwise, there
would be no way to distinguish "directory empty" from "directory does not
exist".

Comment 6

[Timo Sirainen]

BTW. In Dovecot there's tb-extra-mailbox-sep imap_client_workaround that deals with Thunderbird's bugginess of adding these extra slashes to mailbox names all around. I'm not sure if it fixes this specific issue, it was mainly created for subscription/unsubscription issues (bug 301714, which has only been open for 4 years now). I guess I could always add more functionality to it...

Comment 7

[WADA:World Anti-bad-Duping Agency]

Bugs I saw were bug 497096 and bug 517054.

(In reply to comment #5)
> There is no such thing as a "folder named <null>".

It's not IMAP rfc level thing. It's for meaning of "/abc/def/" in internal URL like path notation used by Tb.  
"imap:/.../abc/def/ghi" means ghi under def under abc. "abc/def/*" means any folder name including delimiter if IMAP list/lsub. "abc/def/%" means any folder name excluding delimiter if IMAP list/lsub.
Considering "abc/def/ corresponds to <ghi or * or %" part> == Null" is natural, isn't it?
It's also an analogical extension of "default index.html by Apache for http://.../abc/def/". 

> The LIST response here simply confirms that Trash/emu2000 exists as a directory.

I couln't find any description about "folder hierarchy delimiter" at end of folder name.
Where can I see RFC description for it?

Following is log in your bug 301714. 
> 20 create "helloworld/"
> 20 OK Create completed.
> 21 subscribe "helloworld/"
> 21 NO Invalid mailbox name: helloworld/
> ..
> 9 delete "helloworld/"
> 9 NO Invalid mailbox name

Does Dovecot or UW-IMAP behaves same for "separator /" and "separator ."?
Does IMAP servers on Win or Mac behaves like Dovecot or UW-IMAP on *nix?

(In reply to comment #6)
> In Dovecot there's tb-extra-mailbox-sep imap_client_workaround that deals
> with Thunderbird's bugginess of adding these extra slashes to mailbox names
> all around.

Are we better to implement next option of Tb?
  mail.quirks.dovecot_uw_imap.remove_imap_extra_mail_box_sep = true

Comment 8

[WADA:World Anti-bad-Duping Agency]

"Suffix of delimiter in CREATE command" was written in RFC 3501 7.3.3, and it was applicable to LIST response. LISTt response itself refers as "name", and "name" doesn't inhibit "Suffix of separater".
> 6.3.3 CREATE command
> http://tools.ietf.org/html/rfc3501#section-6.3.3
> If the mailbox name is suffixed with the server's hierarchy
>    separator character (as returned from the server by a LIST
>    command), this is a declaration that the client intends to create
>    mailbox names under this name in the hierarchy.
> 7.2.2 LIST response
> http://tools.ietf.org/html/rfc3501#section-7.2.2 
> Contents:   name attributes
>             hierarchy delimiter
>             name
>(snip)
>      The name represents an unambiguous left-to-right hierarchy, and
>      MUST be valid for use as a reference in LIST and LSUB commands.

(I wanted to say by "Null folder name" that abc/def/ should be treated like "directory in http:" by Tb. RFC 3501 said it as "mailbox name suffixed with separater.)

However, "DELETE Command" says explicitly "mailbox name".
> 6.3.4. DELETE Command
> http://tools.ietf.org/html/rfc3501#section-6.3.4
> Arguments:  mailbox name

bug 301714, bug 497096, bug 517054, and this bug(bug 537626) is for same issue, and cause is RFC violation by Tb.
Sorry for my misunderstanding.
Setting dependency to bug 301714 for ease of tracking, before duping.

To Harald Dunkel(bug opener):

Can "tb-extra-mailbox-sep imap_client_workaround" of Dovecot be a workaround of your case?

Comment 9

[Harald Dunkel]

(In reply to comment #6)
> BTW. In Dovecot there's tb-extra-mailbox-sep imap_client_workaround that deals
> with Thunderbird's bugginess of adding these extra slashes to mailbox names all
> around. I'm not sure if it fixes this specific issue, ...

Yes, it does. I am happy.

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to WADA from comment #8)
> "Suffix of delimiter in CREATE command" was written in RFC 3501 7.3.3, and
> it was applicable to LIST response. LISTt response itself refers as "name",
> and "name" doesn't inhibit "Suffix of separater".
> > 6.3.3 CREATE command
> > http://tools.ietf.org/html/rfc3501#section-6.3.3
> > If the mailbox name is suffixed with the server's hierarchy
> >    separator character (as returned from the server by a LIST
> >    command), this is a declaration that the client intends to create
> >    mailbox names under this name in the hierarchy.
> > 7.2.2 LIST response
> > http://tools.ietf.org/html/rfc3501#section-7.2.2 
> > Contents:   name attributes
> >             hierarchy delimiter
> >             name
> >(snip)
> >      The name represents an unambiguous left-to-right hierarchy, and
> >      MUST be valid for use as a reference in LIST and LSUB commands.
> 
> (I wanted to say by "Null folder name" that abc/def/ should be treated like
> "directory in http:" by Tb. RFC 3501 said it as "mailbox name suffixed with
> separater.)
> 
> However, "DELETE Command" says explicitly "mailbox name".
> > 6.3.4. DELETE Command
> > http://tools.ietf.org/html/rfc3501#section-6.3.4
> > Arguments:  mailbox name
> 
> bug 301714, bug 497096, bug 517054, and this bug(bug 537626) is for same
> issue, and cause is RFC violation by Tb.
> Sorry for my misunderstanding.
> Setting dependency to bug 301714 for ease of tracking, before duping.

should this, and the other bugs, be duped to bug 497096

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to Wayne Mery (:wsmwk) from comment #10)
> > Setting dependency to bug 301714 for ease of tracking, before duping.
> should this, and the other bugs, be duped to bug 497096
And, ...
> Flags: needinfo?([e-mail])

Why needinfo from me?

> Bug 301714 == Target Milestone: Thundebird 13.0
>   i.e. already fixed by Tb 13.

If this bug(bug 537626) and similar bug 497096/bug 517054 won't occur in Tb 13 or later(where the bug 537626 is already fixed), these bug can be said "same as Bug 301714".
However, if this bug(bug 537626) and similar bug 497096/bug 517054 still occurs even after fix of Bug 301714, it simply means that other problem than problem of the Bug 301714 is involved in these bugs.

What is reason why you request additional info from me instead of bug opener?

Comment 12

[Wayne Mery (:wsmwk)]

I guess, to be more direct, my question is why is this bug still unconfirmed?

Comment 13

[:aceman]

Wayne, this bug seems to predate Thunderbirds maildir implementation. Does it really block bug 845952? I see no mention of TB's maildir here. The discussion here seems to be about commands with folder names sent to the IMAP server.

Comment 14

[WADA:World Anti-bad-Duping Agency]

Closing as dup of bug 301714. If wrong, re-open, please.