1244779 - nsImapProtocol::DiscoverAllAndSubscribedBoxes() lists only 2 levels of IMAP folders - causing "Subscribe" to ignore sub-(sub)-folders of shared IMAP folders

Status: RESOLVED DUPLICATE of bug 1221038

(MailNews Core :: Networking: IMAP, defect)

Description

[R.Koerner]

Attachment: TB_IMAP_communication_subscribe_refresh.txt

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0
Build ID: 20160126223146

Steps to reproduce:

Two different IMAP users exist.
1. User "owner" shares a folder "a_level1" which is located in the INBOX of "owner".
2. User "reader" starts Thunderbird and subscribes to the shared folder "a_level1" - this works.
3. User "owner" shares the sub-folder "a_level1/a_level2" (and exists Thunderbird).
4. User "reader" restarts Thunderbird, selects menu "File > Subscribe...", in the "Subscribe" dialog, clicks button "Refresh".


Actual results:

The "Folder List" of the "Subscribe" dialog does not show the newly created, shared sub-folder "a_level1/a_level2". This is reproducible.


Expected results:

The "Folder List" of the "Subscribe" dialog shows the newly created, shared sub-folder "a_level2" (after expanding the listed "a_level1" folder). It is possible to subscribe to that sub-folder.

Note:
This happens with two different Linux IMAP servers.
Setting the sharing ACls was done on the IMAP servers by using the commandline. (Tests with the Thunderbird ACL Addon gave the same result.)

Analysis:
A network packet capture (see attached file for the extracted IMAP communication) shows that Thunderbird does not query the full list of folders. It seems to ignore the "\HasChildren" at the folder "a_level1", which probably should cause another query.

Comment 1

[R.Koerner]

Exact Thunderbird version:
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1

Comment 2

[R.Koerner]

Attached an experimental patch, which seems to fix the problem. (No extensive testing was done.)

The patch uses the IMAP command
 list "" "shared/*"

instead of the original two commands
 list "" "shared/%"
 list "" "shared/%/%"

to obtain all available folders within an arbitrary depth of levels (asuming the IMAP server provides it in the response).
Note: "shared" is the actual name of the IMAP namespace, which is given by the IMAP server.

I do not know, why the original code retrieves the folders level by level (but only two levels max.).
So I cannot say whether there is a special reason for that, which might cause my patch to break something.

At least, the patch gives a direction where to start for further analysis.
Hint: the affected file "mailnews/imap/src/nsImapProtocol.cpp" contains some more code which makes use of a similar pattern (i.e. "/%/%" from "%" and '%') for accessing IMAP folders. Maybe that code should be considered for a revision, too.

Comment 3

[R.Koerner]

Attachment: Thunderbird_38.5.1_fix_ImapProtocol_Subscribe_any_depth_of_shared_folders_EXPERIMENTAL.patch

Comment 4

[Kent James (:rkent)]

Can you try setting the preference mail.imap.hide_unused_namespaces to false, and see if you get the same results?

Comment 6

[Kent James (:rkent)]

I believe this is a TB 38 regression. The first report that I see is bug 1221038, so I will comment further there.

Comment 7

[Kent James (:rkent)]

Comment on attachment 8715360 [details] [diff] [review]
Thunderbird_38.5.1_fix_ImapProtocol_Subscribe_any_depth_of_shared_folders_EXPERIMENTAL.patch

Review of attachment 8715360 [details] [diff] [review]:
-----------------------------------------------------------------

Just for the record, I looked over this patch, and it is not the right thing to do, because it does not work well when there are lots of folders at many levels. From my comments in bug 1221038 the solution to this problem is in the UI, to ask people to request each folder be opened to display subfolders.

Comment 8

[Kent James (:rkent)]

Here's an update.

I've tried restoring the UI behavior of pre-38 releases, namely starting with all folders collapsed in the subscribe dialog. This mostly works for me, but unfortunately there is some sort of race condition that prevents it from working in some cases. Christian from Bug 1264724 sees this always, but I have seen it occasionally in my environment as well. Also, unfortunately I am not seeing it now so I cannot debug it. This is all made worse by the fact that this is an old RDF template implementation that has been mostly removed from Mozilla code, precisely because it is so darn hard to debug any issues that occur there such as this one.

So I am tempted to just accept the patch in this bug, and live with he consequences for environments where there are large numbers of unsubcribed folders. That is better than not working at all. But still looking at it all.

Comment 9

[Kent James (:rkent)]

Sorry, I should have made this comment in the duped-to bug.