537298 - sub-sub folders from Zimbra servers not showing in Thunderbird

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: IMAP, defect)

Description

[Alan Bell]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15
Build Identifier: 3.1

I set up a three level folder structure in Zimbra and Gmail and set up the accounts in 
TB3 Thunderbird on Windows Vista
TB2 on Ubuntu Karmic
TB3.1 on Ubuntu Karmic
Evolution on Ubuntu Karmic

the folder structure was
Leads
Leads/2010 Leads
Leads/2010 Leads/Important

In all the Thunderbird versions sub-sub folder (Important) on Zimbra did not display. They did in Evolution.
Having done an IMAP trace of the connections to Gmail and Zimbra I can see a different response to the LSUB IMAP command, I have verified this with telnet to the IMAP ports.
Zimbra responds like this:
* LSUB () "/" "Leads/2010 Leads"
Gmail responds like this:
* LSUB (\HasChildren) "/" "Leads/2010 Leads"

Zimbra responds to the LIST command like this:
* LIST (\HasChildren) "/" "Leads/2010 Leads"

so when doing LSUB Zimbra does not include the hierarchy information.
RFC 3501 states in 7.2.3 "The data is identical in format to the LIST response." thus one might expect the different response to be a server bug, however
RFC3348 states in section 3:
"   The \HasChildren and \HasNoChildren attributes might not be returned
   in response to a LSUB response.  Many servers maintain a simple
   mailbox subscription list that is not updated when the underlying
   mailbox structure is changed.  A client MUST NOT assume that
   hierarchy information will be maintained in the subscription list.
"
I think Thunderbird is assuming the hierachy information is provided in LSUB which is a MUST NOT therefore I think the bug is in the client side.

Reproducible: Always

Steps to Reproduce:
1. set up three level hierarchy on Zimbra server
2. connect with Thunderbird
3. observe third level (and beyond) are missing.

Comment 1

[WADA:World Anti-bad-Duping Agency]

What is displayed at subscription list?
("+" for expand is displayed before "Leads/2010 Leads"? Or not displayed?)

Tb issues "LSUB *". No "* LSUB ... Leads/2010 Leads/Important" from server means "Leads/2010 Leads/Important" is not subscribed yet".
Default is "Show only subscribed folders" checked. So, if "Leads/2010 Leads/Important" is not subscribed yet by anyone, "Leads/2010 Leads/Important" won't be displayed at folder pane by Tb.
Did you subscribe "Leads/2010 Leads/Important"?
Do you mean you are not able to subscribe "Leads/2010 Leads/Important" because expand button is not displayed at subscription list?

Workaround if you want to show all folders always:
  Uncheck "Show only subscribed folders". 

> They did in Evolution.

Evolution may work like Tb with "Show only subscribed folders unchecked".

Comment 2

[Alan Bell]

The same folder list was displayed in the subscription list, with no expand indicator.
I unchecked the show only subscribed folders thing and they did turn up after a while - however when talking to a gmail back end there was no need to do this. I will try and set up an account on a public Zimbra server

Comment 3

[Alan Bell]

I reproduced this on the http://www.testzimbra.com server

E-mail Address: [e-mail]
Servername/Hostname: testzimbra.com
Username: zm1231qalvi
Password: inqrfc

this is just a demo account that expires after 2 weeks.

Comment 4

[Alan Bell]

Attachment: screenshot showing lack of visibility of sub sub folders on Zimbra back end

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #2)
> The same folder list was displayed in the subscription list, with no expand
> indicator.

Tb still has problem like Bug 514106, "expand indicator at subscription list" is not always displayed properly. Check whether your problem is similar problem to Bug 514106 or not, please.

As you say, I'm suspecting that Tb relies on "* LSUB (\HasChildren/\HasNoChildren)" response. If it's true, I think Tb should merge garanteed "LIST response" with non-garanteed "LSUB response" for (\HasChildren) / (\HasNoChildren) / ().

Comment 6

[Alan Bell]

Attachment: Zimbra web UI showing third level folder below 2010 hot leads

Comment 7

[Alan Bell]

I don't think it is the same as bug 514106 because that one occurs against gmail which does include (\HasChildren) in the LSUB response.

Comment 8

[WADA:World Anti-bad-Duping Agency]

Can you attach minimum IMAP log file(never paste, please) for diagnosis by developers?
 - Define new profile, define the IMAP account
   - Disable unwanted IMAP access by Tb
     - Disable "Global Serch and Indexer"(Gloda), Disable auto-sync
     - Disable check for software-update, Disable check of RSS feed
     - Disable new mail check at startup/every NNN minute
     - etc.
     - Enable "Show only subscibed folders"
 - Open smallest IMAP folder(no mail folder is better), and terminate Tb.
 - Start Tb with NSPR logging enabled for IMAP.
   Get IMAP log for account's property/subscribe.
> https://wiki.mozilla.org/MailNews:Logging
> http://www.mozilla.org/projects/nspr/reference/html/prlog.html#25328
> SET NSPR_LOG_MODULES=imap:5,timestamp

Comment 9

[Alan Bell]

Attachment: imap log file from 3.0.1Pre

running Ubuntu Karmic installed Thunderbird-3.0 package from ubuntu-mozilla-daily ppa. set up account against testzimbra.com, login details as above. It did show all the folders this time, but not in the file-subscribe dialog.

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #9)
> imap log file from 3.0.1Pre

> It did show all the folders this time, but not in the file-subscribe dialog.

(1) Nothing is returned to lsub "" "*" & lsub "" "/home/*" by your server.
    So, I couldn't see log like next(no \HasChildren, \HasNoChildren). 
> * LSUB () "/" "Leads/2010 Leads"

(2) Following log is seen.
> 5 namespace
> * NAMESPACE (("" "/")) (("/home/" "/")) NIL
> 5 OK NAMESPACE completed
>(snip)
> 14 list "" "/home/%"
> 14 NO LIST failed: wildcards not permitted in username
> 15 list "" "/home/%/%"
> 15 NO LIST failed: wildcards not permitted in username

Comment 11

[WADA:World Anti-bad-Duping Agency]

> Zimbra responds like this:
> * LSUB () "/" "Leads/2010 Leads"

In this case, I think one of next is displayed at subscription list.
(a)  + Leads (+ = expand button)
(b)  - Leads
         2010 Leads (no expand button)
Is it right?

If "+ Leads" is expanded, or if "- Leads" is collapsed and expanded again, I think Tb issues "LIST Leads/%" and "LIST Leads/%/%".
Is it wrong?

If "collapse/re-expand of Leads" is executed several times at subscription list, "+ 2010 Leads" is displayed  at subscription list, isn't it?

Comment 12

[Vincent (caméléon)]

I don't reproduce this bug with Thunderbird 7 and Zimbra 7.1.4. (with a free.fr account).

Can someone confirm if this (old) bug is still valid?

Comment 13

[Den Milu]

I'm using thinderbird 12.0.1, but till now I could not get new emails in subfolders from my zimbra inbox. So how can I fix this?

Comment 14

[David :Bienvenu]

(In reply to Den Milu from comment #13)
> I'm using thinderbird 12.0.1, but till now I could not get new emails in
> subfolders from my zimbra inbox. So how can I fix this?

Den, are you actually subscribed to the sub-folders? Or are you saying that new mail doesn't automatically get checked for the sub-folders?

Comment 15

[Den Milu]

Hi David, 

Thanks for your support, I used filter to send all my income emails that have @mydomain.com to my subfolder, and after that, my thunder bird only get messages from inbox, NOT in my subfolder :(
I dont know how to tell Thunderbird subscribe to my sub folder on zimbra.

Comment 16

[David :Bienvenu]

Is Thunderbird showing you the subfolder? If so, you can right click on it in the folder list in Thunderbird, pick properties, and check the box that says "when getting new messages for this account, always check this folder". If you're not seeing the folder at all in Thunderbird, you can use the file menu, pick subscribe, and make sure there's a checkmark next to the subfolder.

Comment 17

[Den Milu]

Dear David,

I added 2 emails account (Gmail and Zimbra) in Thunderbird, but, When i clicked on File menu, then subscribe, the box only appear acc Gmail, and the Zimbra acc is did not appear. So what is my problem? Can you help me? Thanks

Comment 18

[WADA:World Anti-bad-Duping Agency]

(In reply to Den Milu from comment #17)
> I added 2 emails account (Gmail and Zimbra) in Thunderbird, but, When i
> clicked on File menu, then subscribe, the box only appear acc Gmail, and the
> Zimbra acc is did not appear.

Are you looking bug 534994?
If so, click an IMAP folder(force login and Mbox open) before subscribe in order to bypass that bug.

Comment 19

[David :Bienvenu]

(In reply to Den Milu from comment #17)
> Dear David,
> 
> I added 2 emails account (Gmail and Zimbra) in Thunderbird, but, When i
> clicked on File menu, then subscribe, the box only appear acc Gmail, and the
> Zimbra acc is did not appear. So what is my problem? Can you help me? Thanks

Den, what happens if you click on a folder in the zimbra account, and then bring up the subscribe UI? Or click on the zimbra account line in the folder pane, and click on the subscribe link?

Comment 20

[Ludovic Hirlimann [:Usul]]

Den ?

Comment 21

[Den Milu]

@David: the same result when I clicked on Zimbra acc then clicked subscribe. There only appear subscribe for Gmail acc.  :(
I can get all email on my gmail sub folder acc

@Ludovic Hirliman: ??? :P

Comment 22

[Wayne Mery (:wsmwk)]

Do you still see this problem?

There are a number of bug reports

https://bugzilla.zimbra.com/show_bug.cgi?id=85327 Unable to subscribe to IMAP folder with thunderbird
fixed Release 8.7.0.BETA1.1035.UBUNTU12.64 UBUNTU12_64 NETWORK edition..

https://bugzilla.zimbra.com/show_bug.cgi?id=61435  dangling IMAP folder subscriptions resulting in mail.NO_SUCH_FOLDER error
fixed Release 7.1.3_GA_3342.UBUNTU10_64 UBUNTU10_64 

bug 1013386 - On Zimbra 8.0.7_GA_6021 , Thunderbird 24.5.0 does not appear to show you subscribed folder

Bug 680174 - Thunderbird should automatically subscribe to all folders in my mail.mozilla.com / Zimbra account

Comment 23

[Den Milu]

@:Wayne Mery 
I have remove my sub folder to complete get all my email. using sub-folder on zimbra is not a good idea :( . Only get all sub-folder content on gmail acc.

Comment 24

[Wayne Mery (:wsmwk)]

Dave, is this something you can reproduce with your zimbra server?

Comment 25

[Ludovic Hirlimann [:Usul]]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #24)
> Dave, is this something you can reproduce with your zimbra server?

We don't have it anymore.

Comment 26

[Wayne Mery (:wsmwk)]

Justdave had indicated earlier this year he was running zimbra on a personal server

Comment 27

[Dave Miller [:justdave]]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #26)
> Justdave had indicated earlier this year he was running zimbra on a personal server

This is correct, I do.  I cannot reproduce this.  I do have 3-level deep folders in my personal account, and both Postbox 4 and Thunderbird 41 can see all three levels.  I didn't try any packet tracing yet to see what the actual IMAP responses look like.

A folder named "Financial/Archived/SBC" (and several of its siblings) show up in the folder tree.

I'm running Zimbra 8.6.0_P4 Community Edition on the server end.

Comment 28

[Wayne Mery (:wsmwk)]

WFM per comment 27. If anyone sees differently please update the bug

Comment 29

[Den Milu]

Because some of reasons, my company still using Zimbra 7.2.1_GA_2790 :( So can not get any sub directory.

Comment 30

[luca.g.martini]

My company is using Zimbra 7.1.3_GA_3346 but the problem with the Thunderbird subfolders is still present.

I found the following workaround to fix the synchronization problem, hope it helps:
1 - In Zimbra, move the missing subdirectories directly under "Inbox"
2 - In Thunderbird, Manage folder subscriptions --> Refresh --> Subscribe the directories moved at the previous step
3 - The missing directories should now appear in Thunderbird under Inbox, move them to their original place

By doing so, the directiories should now be synced.