Closed Bug 1714572 Opened 3 years ago Closed 2 years ago

Office365: Subscribing to folders is not successful.

Categories

(Thunderbird :: Untriaged, defect)

Product:
Thunderbird
Component:
Untriaged
Platform:
Desktop
Windows 10
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: clamb, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0

Steps to reproduce:

Updated to 78.11 yesterday, restarted TB today.
Restarting/safe mode/resubscribing doesn't resolve it

Actual results:

No folders are shown other than Inbox and Trash w/ Office365 account.
Subscribing to folders is not successful.
On re-start of TB, sometimes they appear briefly, but always disappear.
No errors observed.
Safe mode does not resolve it.

Expected results:

100's of folders w/ many years of messages should have appeared

I was able to resolve it by unselecting the 'Only show subscribed folders' option.
However, it seems to be a significant bug that perhaps has existed since 2016 based on the reports I have seen. Not sure what happened with the 78.11 upgrade that trigger that option to become checked.

Thanks Chris, for this ticket. YES, i can confirm that there is a serious issue on subscribing IMAP folders with office365.
Even unselecting the 'Only show subscribed folders' option (and restarting) doesn't give me access to my folder structure. Hopefully, someone can fix this, soon.

Update: unselecting the 'Only show subscribed folders' option does help.

Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
Resolution: --- → INVALID

I don't understand why this has changed to invalid.
Folder subscriptions don't work and updates change settings in ways that make the product unusable. Seems like 2 features are broken and several people have spent considerable time working around them, just this week. Perhaps it's a duplicate issue, but it's not Invalid.

Just to clarify, selecting "Only show subscribed folders" works, but it take some time until this becomes active. Not sure what the inner working in TB are that cause the delay or even require a restart. Anyway, this could make people thing that this setting also does not work and then folder are back magically suddenly. However, subscribing to singe folder does not work, regardless of how lone one waits of restarts TB.

Current state is that things get worse. Subscribing still does not work, but now with "Only show subscribed folders" unchecked, only folders on level 1 and 2 show up, any levels below this are gone. So not things are seriously broken. Maybe this is not even TB to blame, but Microsoft is changing things on the outlook365 side that cause all this. Since there are no logs shown about this it's really hard to say.

For me, it looks like 3 levels at least show up:
2021->BusDev->Marketing, I don't know of any deeper nests I have than this.

I also changed this to back to unverified as I don't think it is resolved per my comments above.

Status: RESOLVED → UNCONFIRMED
Resolution: INVALID → ---
OS: Unspecified → Windows 10
Hardware: Unspecified → Desktop

I'm getting this too on the beta channel. Only started happening after updating to v90.0 Beta 1 (from 89.0 Beta 4)

Maybe something on their server changed... xref bug 1668833 though, it was always odd wrt. subscriptions.

Magnus, I would like to look at this again but it seems the o365 testing access with imap/oauth2 you provided me a while back (see email you sent to me on 10/2/2020) at gene@mozthunderbird.onmicrosoft.com is no longer available. I tried to setup again using info here, bug 1668834, but no luck.

The account is still available. I reset the pwd now - check you email.

This sounds very similar to a bug raised a few years ago - bug 1241926

Magnus, I still can't get the account to work. I can sign in OK with my gene@mozthunderbird.onmicrosoft.com email addr and password at the office365 site but I can't see anything there and any attempt to setup the account in tb fails. I assume you are the account administrator so maybe you can check if it is valid and/or reset the password one more time. Or maybe the account should just be deleted and then re-created under a new user name?

I sent you an email on 6-11 about this but never heard back.

Specifically, when I go to the office365 site and sign in and see nothing, I then check my security settings. It then asks me to confirm my sign in using the "Authenticator App". I run the M/S authenticator app on my phone and the app asks me to add the account. I enter the email address and password and then the Authenticator apps says to open the Authenticator app to verify it account. So it's a vicious cycle that gets nowhere.

This may be somewhat related to the fact that my old phone broke that had the authentcator app on it and I used it when you originally set up the account but now I'm on a new phone. Also, I'm not even sure if I should have used the authentictor app.

Flags: needinfo?(mkmelin+mozilla)

I'm having the same issue on an older version 68.10.
Unable to subscribe to folders, but unchecking only show subscribed folders shows all folders.

I have been following this bug because after starting to use TB just recently (78.11 on Windows 10) the folders in my old outlook.com mailbox were only being shown partly. After digging deeper I found an issue that may not solve what's been found here - but may also affect one or the other: If the folder in the Exchange-mailbox does not have the FolderClass-property set to IPF.Note they are not processed properly when accessing from an IMAP-client (here TB). Fixing this with some Powershell-magic (i.e recursing over all folders and setting missing FolderClass-property to IPF.Note) made all folders appear - and I could properly unsubscribe from those I didn't want (i.e. I do not have to deactivate "Show only subscribed folders". More details on request ...

...the folder in the Exchange-mailbox does not have the FolderClass-property set to IPF.Note...

Are you referring to a folder on the Exchange server itself or a folder on the Win10 client? I think you mean the server but don't know how you are accessing the internals of outlook.com.

Yes, I mean the folders on the exchange-server under the message-folder root. What you need to understand is that the FolderClass property does not deter Outlook, OWA, Windows 10 Mail, evolution (with evolution-ews), KMail (with akonadi-ews) from accessing all folders and items in those folders - but if you access it via IMAP (from the builtin IMAP-service of the exchange-server) folders and mail-items in them are presented (at least as visible in TB) only if the FolderClass is IPF.Note. So, yes, that is "internals" - but only by correcting it there (I used Powershell and the EWS Managed API) makes folders and mail-items accessible to TB via IMAP.

(In reply to gene smith from comment #17)

...the folder in the Exchange-mailbox does not have the FolderClass-property set to IPF.Note...

Are you referring to a folder on the Exchange server itself or a folder on the Win10 client? I think you mean the server but don't know how you are accessing the internals of outlook.com.

Yes, I mean the folders on the exchange-server under the message-folder root. What you need to understand is that the FolderClass property does not deter Outlook, OWA, Windows 10 Mail, evolution (with evolution-ews), KMail (with akonadi-ews) from accessing all folders and mail-items in those folders - but if you access it via IMAP (from the builtin IMAP-service of the exchange-server) folders and mail-items in them are presented (at least as visible in TB) only if the FolderClass is IPF.Note. So, yes, that is "internals" - but only by correcting it there (I used Powershell and the EWS Managed API) do invisible folders (and their mail-items) become accessible to TB via IMAP.

Ok, I think I see what you mean. When I google for imap ipf.note I see articles describing changing folders from IPF.IMAP to IPF.NOTE so they work with proprietary M/S (non-IMAP) protocols like maybe EWS. This seems the opposite of what you describe, but I guess it works to fix the problem you see. Maybe setting the folders to IPF.IMAP would fix the problem too?

Unless I'm missing your point completely, this is not due to any bug in TB or TB's IMAP implementation.

I agree, the specific condition I describe can't be fixed in TB - and in fact with the work-around I found TB is working flawlessly with my huge mail-collection in outlook.com. My point was really to prevent unnecessary work by TB-devs in this case!
Regarding your suggestion of setting FolderClass to IPF.Imap: That would lock out all clients accessing Exchange with a native protocol - and IMO that property is meant to mark folders in an Exchange-mailbox (or .PST file) that are retrieved from and managed in an external IMAP-server (not the one belonging to the Exchange-server of the mailbox).

My point was really to prevent unnecessary work by TB-devs in this case!

Much appreciated!
Hopefully other reporters and commenters on this bug can try your solution and see if it helps.

(sent gene new details)

Flags: needinfo?(mkmelin+mozilla)

Has there been a fix found for this? I have a user that is having this problem. We are unable to subscribe to folders in Thunderbird but we can see they exist and Thunderbird sees them in the Subscribe list. We are currently doing the work around of showing all folders but that isn't very useful when they have so many folders.

bcomp, if you select the folder you want to see in the "subscribe" dialog and don't select the ones you don't care about there, and restart TB, do you see the folders? You should not have to uncheck the advanced setting to show all folders regardless of subscription status.
This seems to work for me. I can also subcribe to or unsubscribe folders that are subfolders of any depth and it works too.
This is on the mozthunderbird2 account on office365 that Magnus provided (Thanks, Magnus!).

The only "issue" I see is that subscribe and unsubscribe does not immediately show up with the office365 account. Also, collapse and then expand the full folder tree from the account name is supposed to cause folder "re-discovery" but for o365 account doesn't so a restart is needed to see the changed subscription effect. This works for some account types and not others (not just o365) and the reason is unclear but needs to be determined.

This isn't bug 1714870 is it?

(In reply to Magnus Melin [:mkmelin] from comment #27)

This isn't bug 1714870 is it?

I don't think so. That shows a folder in the tree just going blank when selected

I think the problem is what I describe here: bug 1668833 comment 2.
With the function return due to empty password commented out, the newly subscribed folders appear and newly unsubscribed folders disappear. No collapse/expand of folder tree at the account root level is needed, and, of course, a TB restart isn't needed.
But I have no idea why a password string is obtained, checked for empty and otherwise ignored in this code.

Also, if account activity is idle for more than 30 minutes and then a subscribe or unsubscribe is attempted, the password is non-empty this one time until another idle interval elapses. This is probably because the connection times out and a new connection is established with the oauth2 password or token.

After subscribing and then closing Thunderbird as you suggested the folders do appear briefly but then disappear and aren't checked as being subscribed to the folder.

(In reply to gene smith from comment #28)

(In reply to Magnus Melin [:mkmelin] from comment #27)

This isn't bug 1714870 is it?

I don't think so. That shows a folder in the tree just going blank when selected

I think the problem is what I describe here: bug 1668833 comment 2.
With the function return due to empty password commented out, the newly subscribed folders appear and newly unsubscribed folders disappear. No collapse/expand of folder tree at the account root level is needed, and, of course, a TB restart isn't needed.
But I have no idea why a password string is obtained, checked for empty and otherwise ignored in this code.

Also, if account activity is idle for more than 30 minutes and then a subscribe or unsubscribe is attempted, the password is non-empty this one time until another idle interval elapses. This is probably because the connection times out and a new connection is established with the oauth2 password or token.

Yes, this sounds very similar to the issue we are seeing also.

Yes, this sounds very similar to the issue we are seeing also.

I'm not sure if you are referring to bug 1714870 or bug 1668833 comment 2 ??
I think bug 1714870 only applies to Mac and doesn't really affect subscriptions, so can I assume you are not referring to bug 1714870?

After subscribing and then closing Thunderbird as you suggested the folders do appear briefly but then disappear and aren't checked as being subscribed to the folder.

By "disappear" I think you mean they become not present in the list. I don't think you mean there is a blank spot where the folder should be as described in bug 1714870, right? Also the missing folder loses its subscription when you look at the subscription screen, correct?

I can't seem to duplicate this with my o365 account with tb 78.11.0 or trunk version. When I subscribe to a folder it stays subscribed and visible in tb.

How long do the newly subscribed folders remain visible? Does it matter if they contain messages?

I've mostly just been subscribing and unsubscribing to o365's built-in folders like "Journal" and "Conversation History" and they don't lose their subscription. Also, to lose a subscription the o365 server would have to not include the folder in the LSUB response, so maybe this is a server issue, not sure. Maybe an IMAP:5 log would show something. See https://wiki.mozilla.org/MailNews:Logging for how to produce one and attach it above. Thanks!

Summary: 78.11.0 update looses all my folders w/ Office365 → Office365: Subscribing to folders is not successful.

Support Forum is suddenly reporting issues. For example:
https://support.mozilla.org/en-US/questions/1346467
https://support.mozilla.org/en-US/questions/1346059
https://support.mozilla.org/en-US/questions/1346193

Workaround: Currently advising to uncheck 'Show only subscribed folders' to force a display and then after a restart accessing the 'Synchronisation & Storage' and making sure the 'Advanced..' button has checkmarks fo folders.

See comment 28 for the only problem I see. My o365 account seems to work fine subscription wise. I never see folders go away or disappear. But I only look at it occasionally like when a (this) bug is reported.

As I mentioned above, capturing something in the IMAP:5 log is probably necessary since I can't duplicate this
https://wiki.mozilla.org/MailNews:Logging

Anyhow, FWIW, if I comment out the check for password in the code that handles the "expand" (which is also called when a folder discover URL is required) it fixes the need to restart TB to see oauth2 based server subscription changes:

diff --git a/mailnews/imap/src/nsImapIncomingServer.cpp b/mailnews/imap/src/nsImapIncomingServer.cpp
--- a/mailnews/imap/src/nsImapIncomingServer.cpp
+++ b/mailnews/imap/src/nsImapIncomingServer.cpp
@@ -862,19 +862,19 @@ NS_IMETHODIMP nsImapIncomingServer::Rese
 }
 
 NS_IMETHODIMP
 nsImapIncomingServer::PerformExpand(nsIMsgWindow* aMsgWindow) {
   nsString password;
   nsresult rv;
   rv = GetPassword(password);
   NS_ENSURE_SUCCESS(rv, rv);
 
-  if (password.IsEmpty()) return NS_OK;
+  // if (password.IsEmpty()) return NS_OK;

I have no idea why password is requested here. If you are not authenticated already the discovery won't happen anyhow since the commands to do them require imap authenticated state.

Does problem reproduce when using newer version (91 for example) with Help > Troubleshoot mode?

Whiteboard: [closeme 2021-11-20]

Resolved per whiteboard

Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago → 2 years ago
Resolution: --- → INCOMPLETE
Whiteboard: [closeme 2021-11-20]
See Also: → 1759883

Current report in Support Forum
https://support.mozilla.org/en-US/questions/1397215

User says:

I am using Godaddy that now uses Office365. This only happened after the last update. All folders that I was subscribing to were showing prior to the update.

User has just updated to 102.5.0
Imap Office 365
Subscribe list shows all expected folders

In Folder Pane, Imap account only shows, Inbox, Sent, Trash
Subscribing to see folders does not display in Folder Pane.
Going back to the Subscribe list then shows all previously selected folders are now unchecked.

Should this bug get reopened OR would a new bug report be prefered ?

Sounds like bug 1798685

You need to log in before you can comment on or make changes to this bug.