Closed Bug 1419735 Opened 7 years ago Closed 7 years ago

User shared imap folders does not show

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(thunderbird_esr52 wontfix, thunderbird59 fixed, thunderbird60 fixed)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 60.0
Tracking Flags:
Tracking Status
thunderbird_esr52 --- wontfix
thunderbird59 --- fixed
thunderbird60 --- fixed

People

(Reporter: kire.dyfvelsten, Assigned: gds)

Details

Attachments

(4 files, 2 obsolete files)

tb_error_shared_imap_folder.PNG
11.32 KB, image/png
Details
Fully subscribed test account screenshot
20.04 KB, image/png
Details
list-all-pattern.png
104.31 KB, image/png
Details
1419735-review-changes1.patch
2.40 KB, patch
jorgk-bmo
: review+
jorgk-bmo
: approval-comm-beta+
Details | Diff | Splinter Review
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0 Build ID: 20171024165158 Steps to reproduce: connect to an imap mail account and have someone on the same mailserver share a folder make sure it works on the server webmail server (Tested with Alt-N Mdaemon 17.5.1 and Smartermail 16 Actual results: the "Shared Folders" can not be found Expected results: a folder named "Shared Folders" and containing folders should be listed!
5.0 is not a current version. Which version are you running?
Flags: needinfo?(kire.dyfvelsten)
A case for Gene.
Flags: needinfo?(gds)
Thunderbird: the latest in Release Channel 52.4.0 32-bit
Flags: needinfo?(kire.dyfvelsten)
same result in 52.5.0 32-bit
Users that allready uses shared folders (from earlier versions of TB) can still see the shared folders.. but a user connecting to shared folders for the first time can not see them
current list of shared folders issues https://mzl.la/2A6nlrB Does it work in versioin 45? https://archive.mozilla.org/pub/thunderbird/releases/45.8.0/win32/en-US/Thunderbird%20Setup%2045.8.0.exe
Flags: needinfo?(kire.dyfvelsten)
Version: 5.0 → 52 Branch
Please i am not an developer so dont ask me to reinstall, also im using swedish version. all i know it has been working and does not work in the later versions in release channel
Flags: needinfo?(kire.dyfvelsten)
Has it ever worked in prior versions?
yes it has worked before. and still does for those who earlier connected to another user's share. I am not sure if those users can connect to another new share... i have noticed that users that did not earlier connected to a share, are unable to connect now.
If it works for them with https://archive.mozilla.org/pub/thunderbird/releases/45.8.0/win32/sv-SE/Thunderbird%20Setup%2045.8.0.exe then we have a regression. Installing an old version is as simple as downloading and running. No other changes.
i found an unused computer that had 45.8.0 installed but it does not show there either. i found even older versions and tried on that computer.. but it never showed in the Subscribe window... but strangley i now have some of the imap folder shared using the smartermail server on my folder list... it still does not shows under Subscribe... the folder gives an error when i tries to open it in 52.5.. screenshot attached (swedish error message translated by me Current operation on "foldername" did not succed. Mailserver for account "my smartermail account" answered: SELECT mailbox does not exist.
Will probably need to see the imap log produced by tb to really know what the problem is. I have a local Zimbra server with public and shared folders that work as expected. Instructions for imap log production are here: https://wiki.mozilla.org/MailNews:Logging The main tb configuration settings that affect public or shared/user folders is in the advanced server namespace settings. You might check that and see if the default setting are still set, especially the "Allow server to override these namespaces" checkbox that defaults to checked. (On old tb version that still work, maybe you have some special non-default namespaces settings?) Also, just to be clear, is the folder you are not seeing a public or a user folder? In the namespace settings tb uses the term "Public (shared)" for truly public folders (everyone can see) and "User" for folders shared between selected users. Also, does the missing folder(s) just not appear or are they grayed-out?
Flags: needinfo?(gds)
i have no been in contact with the alt-n server support: this is their report: ------------------------------ Hej, Alt-N har återkopplat och bekräftar att detta är en bugg i Thunderbird och inget som de kan göra något åt: This appears to be a bug within Thunderbird. When I tested this, I experienced the same problem. No matter what I did, I could not get those subfolders to display in either the folder list or in the "Subscribe" window. I checked my IMAP log, and it showed that the Thunderbird client received those subfolders whenever it issued the LIST command, but it would not display them. Han jag pratade med säger att han lyckades visa katalogerna, men jag lyckades inte återskapa det han förklarar: In the end, I went to the folder list in Thunderbird's main window where it only showed the following: Shared Folders User1 Folder1 In then double-clicked "Folder1". Once I did that, "Folder2" revealed itself. After that, I double-clicked "Folder2", and then "Folder3" revealed itself. In the end, it displayed them all like this: Shared Folders User1 Folder1 Folder2 Folder3 I am not sure why Thunderbird behaved this way, but this was not something that MDaemon could control. -----------------
Kire, Thanks for the info from alt-n (your email provider I assume). However, I am a bit confused by their response. They initially say "it would not display them". Then toward the end they indicate "it displays them like this". So they do seem to be displayed but just with some additional double-clicks required. Does it work like this for you? Also, if not working for you, I will need more info: 1. The folder structure you are trying to create. Is problem only for "nested" folder like alt-n's example? 2. Which are "user" and which are "public" folders (see comment 6). Also, which are "personal" not-shared folders? 3. Record and attach the imap log to record tb start up and "shared" folder access attempts (also see comment 6). Or maybe you can ask alt-n to provide me with the log they recorded. 4. As an alternative to 3, provide me a temporary guest account on your imap server so I can personally observer the problem. -gene
Oops, my reference to comment 6 above should actually be comment 13.
Also, I see alt-n is the owner of mDaemon server, not your isp/email providers. I will also try to create nest "shared" folders on my local imap server. Currently I only have root level shared folders and see no problem with that.
gene - a more refined list https://mzl.la/2j7bBxw
Flags: needinfo?(kire.dyfvelsten)
1. I can now confirm that if i share a root imap folder everything works nice... But most of our users wanna share subfolders.. infact most of our users has all their folders as subfolders to inbox 2. Public folders works as normal. its just "nested" user shared folders that does not show 3. i'll attach some logs
Flags: needinfo?(kire.dyfvelsten)
3. it just creates an emty file?!
4. you got mail i have shared my root folder "Delad mapp" and a sub-folder to that named "delad med hoffman & MU" i dont wanna share the root folder "Delad mapp" at all..
(In reply to Kire Dyfvelsten from comment #21) > 4. you got mail > i have shared my root folder "Delad mapp" and a sub-folder to that named > "delad med hoffman & MU" > i dont wanna share the root folder "Delad mapp" at all.. Thanks, I have configured the account and I see this under account "martin.upstream@thage.com" Inbox -- 2 mails inside Drafts Sent Items Deleted Items Folder SubFolder Junk Email Shared Folders - kire dyfvelsten -- this is grey delad mapp -- 1 email inside Should I see "delad med hoffman & MU" under "delad mapp". Is that the problem? I haven't yet produced the imap log so it might answer my question.
If I subscribe "delad med hoffman & MU" I see it under "delad mapp". It is initally unsubscribed. : : Shared Folders - kire dyfvelsten -- this is grey delad mapp -- 1 email inside delad med hoffman & MU -- 3 emails inside What do you mean by "i dont wanna share the root folder "Delad mapp" at all.." ? The mDeamon server is reporting in as shared. Here is a XLIST response from mDaemon: TB: 4 xlist "" "%/%" mD: * XLIST () "/" "Folder/Sub-folder" mD: * XLIST () "/" "Public Folders - Bayesian Learning/Non-Spam" mD: * XLIST () "/" "Public Folders - Bayesian Learning/Spam" mD: * XLIST () "/" "Shared Folders - kire.dyfvelsten/delad mapp" mD: 4 OK XLIST completed This indicates that "delad mapp" is shared and tb reacts correctly. Also, I can see the Public Folders after I subscribe them. So, other than the fact that I have to subscribe the shared and public subfolders, I don't see a problem. Am I missing something? Also, I can copy a message to the shared folders. I can't copy to pubic folders since I assume they are read-only.
(In reply to Kire Dyfvelsten from comment #14) > i have no been in contact with the alt-n server support: this is their > report: > <SNIP> > > In then double-clicked "Folder1". Once I did that, "Folder2" revealed > itself. After that, I double-clicked "Folder2", and then "Folder3" revealed > itself. > > In the end, it displayed them all like this: > > Shared Folders > User1 > Folder1 > Folder2 > Folder3 > > I am not sure why Thunderbird behaved this way, but this was not something > that MDaemon could control. Alt-n indicated they had to double-click to see the folders. For me, double click did nothing and I had to subscribe the lower level folders to reveal them.
i cant find the "Delad mapp med hoffman & MU" folder at all.. i have now stoped sharing of the "Delad mapp" (the most common share among our users are "Folder2" och "Folder3" in the above example) And i shared another folder for the martin.upstream account
since i stoped shareing of the "Delad mapp" the following is happening.. if i now unsubscribe the "Delad mapp med hoffman & MU" i can not find it again
Same here, I don't see any shared folders that are present in webmail in a newly setup account in tb, only see public folders (except for Thage.com public folder). Also, they don't show up in the subscription list. Will look closer...
(In reply to Kire Dyfvelsten from comment #26) > i cant find the "Delad mapp med hoffman & MU" folder at all.. > i have now stoped sharing of the "Delad mapp" I still see "delad mapp" under "Shared" in the martin.upstream webmail page. Have you shared it again? > (the most common share among our users are "Folder2" och "Folder3" in the > above example) Not sure what example you are referring to...? I see Folder and Sub-Folder under Personal and in tb (not shared or public). Should I see Folder2 or Folder3 in webmail? I currently don't. > And i shared another folder for the martin.upstream account Would this be "gary_found_it" shared folder? I see that on webmail (but not on tb, just like all shared folders :( ). Still need to look at imap log / wireshark traces.
Ok, I think I see what the problem is. Assuming your user has a folder structure f1/f2/f3/f4/f5 and they want to share just f4, mD names the shared folder f4 "Shared Folders - f1/f2/f3/f4", not just f4 or "Shared Folders - f4". To determine what folders are there, tb uses xlist "" "%/%" to list any folder that has a 2nd level and returns no shared folders. mD does not return "Shared Folders - f1/f2" like you might expect; so you can't subscribe f4 and it never appears. Why it returns nothing shared, I'm not sure. However, I think if folders f1, f2 and f3 also were shared, it would work. (Last week when you had folder "delad mapp" shared I could see the shared folders below it, now I can't, see comment 23 above.) I compared the sharing method of mD to Zimbra (which I have running locally). With Z, if you share just f4, it names it just f4. If you also share f5 it provides you with just shared f5 and also f5 under f4: f4/f5. The higher level unshared folders are never part of the name. You can keep both or just one. I deleted the f5 and kept the f4/f5 shared pair. In this case xlist "" "%/%" returns f4/f5 so I can subscribed either or both of them and they appear in tb. Note: your public folders still subscribe and display OK because the structure is p1/p2 and all are are configured as public so xlist "" "%" will return p1 and xlist "" "%/%" will return p1/p2 So all work correctly. If only p2 was configured as public, xlist "" "%" would probably return nothing and probably no public folder would display (but I can't really verify this -- maybe it would require 3 levels to mess up, e.g., make public only p3 in p1/p2/p3). I tried to setup sharing with the mDaemon webmail in the martin.upstream account that you provided but was unable to. Maybe I need a 2nd test account to actually test sharing on mD? Would that be possible?
Thanks for the 2nd "mail.tester" account. I've done some testing with just the webmail using the existing folders. I think the problem is that tb can't "discover" shared folders unless all upper to lower contiguous levels are shared with at least level0 and level1 shared. Doing some experimenting in the tb code. More to come...
Attached patch list-asterisk.patch (obsolete) — — Splinter Review
This proposed patch seems to fix the problem. When the subscription dialog is opened, it was doing LIST "" "%" and then LIST "" "%/%" to find subfolders in the user and public namespaces as well as personal. This caused shared folders at a lower level that did not have the top two levels also shared to be missed and not display in the subscription dialog. This does LIST "" "*" instead so everything is listed and appropriate folders (mailboxes) at all levels appear in the subscription dialog and can be selected. I originally thought the problem was caused by XLIST when folders are displayed on the main screen. I forced tb to ignore capability XLIST so it only used LIST and it made no difference. I have backed out this "ignore XLIST" change. Also, when folders on the main screen are displayed, LIST "" "*" occurs even when XLIST is used. This has always occurred and my fix did not cause this. So doing a LIST of folders at all levels during subscription dialog is no different and occurs only on demand (when user activates the subscription dialog).
Attached image list-all-pattern.png —
Kire, this shows how the folder list and subscription list look with the changed code. Is this how you would expect the shared folders to display in tb? Using the mail.tester account, I shared only the 2 highlighted sub-folders. It also shows the folders you shared from your account.
A few more things. The fix still actually works around a possible bug in mDaemon in that, in the mail.tester account example, LIST "" "%/%" does not list "Shared Folders - mail.tester/folder_in_root" as existing mailboxes (folders) like I think it should. If mDaemon did return this string in response to LIST "" "%/%" there would be no need for a code change; the lower level shared folder(s) would be found. Also, if mDaemon handled sharing like Zimbra, and excluded the higher level un-shared folders from the share name, there would also be no problem with the existing code, since shared folder would always exist at the top and lower levels. Regarding the proposed fix, there are several warning about using LIST "" "*" (google: imap list wildcards). The main concern is that 1000's of folders may be listed. However, since I am doing this only when displaying the subscription list, you actually want to see *all* the folders, it seems. Also, as I pointed out in comment 32, LIST "" "*" also occurs when tb starts up and the main folder pane is updated, so the warnings on the net about not using LIST "" "*" have already been ignored by tb. About 6 months back we had a problem with another mDaemon user. Looking at it again, I don't think it is really related to this. However for reference it is Bug 1376299.
Hi Kire, I haven't received a response from you about the folder and subscription display I show in the attachment with comment 33. Also, in revisiting this today I think the test accounts you provided are rejecting my connection attempts now. I haven't submitted a formal patch proposal like in comment 32 since I was waiting for your response. Or maybe you have pointed out the possible issues I found with mDaemon (see comment 34) and Alt-N is working on a fix? Just wondering where things stand. -gene
Flags: needinfo?(kire.dyfvelsten)
Please go ahead and submit the formal patch proposal... :) Alt-n already pointed out that mdeamon sends the whole list but Thunderbird does not show it.. (atleat that how i understand comment 14)
Flags: needinfo?(kire.dyfvelsten)
Attached patch 1419735-review-changes0.patch (obsolete) — — Splinter Review
This patch fixes the reporter's (Kire) bug but it actually fixes a more general bug that is the root cause of his issue (the use of shared folders is not the cause). If a folder is created below the 2nd level, e.g., f0/f1/new-folder it is created and typically auto-subscribed and appears in the folder tree on the main screen. If the right-click Subscribe... action is done, new-folder appears and is shown as subscribed, which is good. But if new-folder is unsubscribed (uncheck its box and quit subscribe screen) and then later an attempt is made to re-subscribe to new-folder, it does not appear on the subscribe screen and remains hidden on the main screen. This patch fixes this by causing the subscribe screen to do just lsub "" "*" // show subscribed folders at all levels list "" "*" // show all folders at levels instead of lsub "" "*" // show subscribed folders at all levels list "" "%" // show all 1st level folders list "" "%/%" // show all 2nd level folders (but not below!) to obtain the data tb needs to build the Subscribe screen. This problem has been touched on in bug reports such as Bug 529244 and others. Even for servers that support the \HasChildren flag, the subscription code in tb does not use it to find folders at lower levels; so they remain hidden. As I pointed out in previous comments, the use of the somewhat frowned upon list "" "*" is already used in tb for servers that "support subscription" when displaying the folders on the main screen. So it should be OK to use this when displaying the folders for subscription too. I have test accounts setup with several popular servers (gmail, aol, outlook, yahoo etc) and some others not so popular and they all work fine when displaying the subscribe page with this patch applied.
Attachment #8934753 - Attachment is obsolete: true
Attachment #8938239 - Flags: review?(jorgk)
Sorry about the delay in reviewing this, I've been busy with other IMAP issues, for example a long string of trash handling bugs. Sadly the subscription dialogue is currently busted (bug 1425962), so I'll get back to this in due course (which may still be a couple of weeks).
Comment on attachment 8938239 [details] [diff] [review] 1419735-review-changes0.patch Review of attachment 8938239 [details] [diff] [review]: ----------------------------------------------------------------- OK, let's go with this. I had a real life case today where TB didn't find the shared folder, yet with the patch it did. It might be worth to go through https://mzl.la/2j7bBxw and close all bugs fixed by this. r+ with the issue below fixed. ::: mailnews/imap/src/nsImapProtocol.cpp @@ +7312,3 @@ > if (!m_imapServerSink) return; > > if (!allPattern.IsEmpty()) This logic can be simplified, allPattern can't be empty here (since it receives at least a '*').
Attachment #8938239 - Flags: review?(jorgk) → review+
Assignee: nobody → gds
Gene, would you mind to address the review issue, or do you want me to do it?
Flags: needinfo?(gds)
Gets rid of useless check for allPattern as empty string since it always contains at least "*".
Attachment #8938239 - Attachment is obsolete: true
Flags: needinfo?(gds)
Attachment #8944478 - Flags: review?(jorgk)
Comment on attachment 8944478 [details] [diff] [review] 1419735-review-changes1.patch Thanks, I'll get this landed.
Attachment #8944478 - Flags: review?(jorgk) → review+
Status: UNCONFIRMED → ASSIGNED
Component: Untriaged → Networking: IMAP
Ever confirmed: true
Keywords: checkin-needed
Product: Thunderbird → MailNews Core
Version: 52 Branch → 52
https://hg.mozilla.org/comm-central/rev/836e8ded762fa62d4d802f297b439b29f7be8515 Subscribe does not find unsubscribed folders below the 2nd level. r=jorgk
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 60.0
Comment on attachment 8944478 [details] [diff] [review] 1419735-review-changes1.patch Missed the branch date by a few hours.
Attachment #8944478 - Flags: approval-comm-beta+
Attachment #8944478 - Flags: approval-comm-esr52?
Comment on attachment 8944478 [details] [diff] [review] 1419735-review-changes1.patch It's not worth it at TB 52.7 when TB 60 ESR is coming out six weeks later.
Attachment #8944478 - Flags: approval-comm-esr52?
I noticed on by local build with several uncommitted changes in place, including the patch to fix the master password prompt, that all my subscribe lists for every account are totally flat. The only thing I could remember that affects subscriptions was this bug. When I reverted the above attached patch, it didn't help. Also, turning off the master password didn't affect the displayed subscribe screen structure. Then I did a bugzilla search for any recent subscription problems and found Bug 1244779 which is marked as duplicate of bug 1221038 that contains a similar patch that that was never accepted that tries to fix the same problem as this bug. So this bug and the two older bugs are all probably duplicates of the same problem. Anyhow, I also noticed that I now couldn't subscribe or unsubscribe a folder on accounts with more than one level of folders. I will keep trying to determine what that problem is and probably submit a new bug report since it does not seem to be a regression of this bug.
Thanks, I saw that bug but the symptom I saw was mentioned way down in the comments. I should have added "flat" to my bugzilla search. With more testing, I see the same subscribe problem with and without the patch attached to this bug. Most accounts work OK but I sometimes don't see folders automatically appear/vanished when subscribed/unsubscribed, especially on the gmail accounts and sometimes restart tb doesn't help. Its seem that the folder tree is not always re-discovered after a change is made in the (now flat) subscribe dialog and maybe not completely on startup. So I think this patch is still OK since it is not the cause of this new problem. However, there are some interesting comments by Kent James in the above two seemingly duplicate reports. He thinks the fix should be in the UI but seems to conclude that the proposed fix (similar patch attached above) is "ok for now".
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: