Closed Bug 1880014 Opened 1 year ago Closed 1 year ago

When "IMAP Server Directory" is set, "Archive"-Button does not always work

Categories

(Thunderbird :: Mail Window Front End, defect)

Product:
Thunderbird
Component:
Mail Window Front End
Version:
Thunderbird 115
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: nico, Unassigned)

References

Details

Steps to reproduce:

  • Use an IMAP mail server with the "Archive" mailbox a subfolder of "INBOX"
  • Set Edit -> Account Settings -> Server Settings -> Advanced -> IMAP Server Directory to "INBOX"
  • Restart TB
  • Mark a message and hit "Archive" button, repeat

Actual results:

  • Sometimes messages correctly get moved to the Archive folder
  • Most times, TB goes into a state where it tries to move the mail but can't and goes into a loop of some kind visible in the status line trying

Expected results:

  • All messages should be moved to the Archive folder consistently

Use an IMAP mail server with the "Archive" mailbox a subfolder of "INBOX"

Does this server put ALL folders under Inbox or just Archive?
I have a few account that put Inbox at the top level by default (courier and dovecot) so everything is under Inbox. When I try to archive a message it tries to create the "year" folder and, as you describe, goes into an endless loop until I shutdown TB and remove the "server directory" of Inbox and restart. Then it seems OK and the archive message is present in the new "year" folder.
So something is definitely wrong with this.

Status: UNCONFIRMED → NEW
Ever confirmed: true

(In reply to gene smith from comment #1)

Does this server put ALL folders under Inbox or just Archive?

All folders are subfolders of "Inbox".

I have a few account that put Inbox at the top level by default (courier and dovecot) so everything is under Inbox. When I try to archive a message it tries to create the "year" folder and, as you describe, goes into an endless loop until I shutdown TB and remove the "server directory" of Inbox and restart. Then it seems OK and the archive message is present in the new "year" folder.

I did see that problem but I have set the option "When archiving messages, place them in:" to "A single folder". So that should not be the problem. Or rather: maybe the same bug is responsible for the problems in both cases.

I did see that problem but I have set the option "When archiving messages, place them in:" to "A single folder". So that should not be the problem. Or rather: maybe the same bug is responsible for the problems in both cases.

Yes, I see the problem no matter how archiving is configured.

Well, this seem to be (randomly?) conditional. Today at first it didn't work at all. Then after a while it worked fine. Then suddenly it stopped working again.

Reporter Nico, I'm curious as to exactly what your imap server is.
I've only tested so far with Courier. I'll try it on dovecot next.

I don't know that it matters, but are you setting the imap server directory to "INBOX\" or maybe, depending on delimiter, "INBOX." or maybe just INBOX without the quotes and delimiter? Also, how is your "personal namespace" set exactly?

With your setting, are you seeing the other folders at the same level as Inbox? In comment 2 you said All folders are subfolders of "Inbox" so I'm not sure how your folders appear in the UI (flat or everything appearing under Inbox).

(In reply to gene smith from comment #4)

Reporter Nico, I'm curious as to exactly what your imap server is.
I've only tested so far with Courier. I'll try it on dovecot next.

I looked it up: it's dovecot.

I don't know that it matters, but are you setting the imap server directory to "INBOX\" or maybe, depending on delimiter, "INBOX." or maybe just INBOX without the quotes and delimiter? Also, how is your "personal namespace" set exactly?

The imap server directory is set to "INBOX" without the quotes and without slash or dot. The personal name space is set to "INBOX." with quotes and with a dot. This is the only way I managed so far to get the result in the next paragraph below:

With your setting, are you seeing the other folders at the same level as Inbox? In comment 2 you said All folders are subfolders of "Inbox" so I'm not sure how your folders appear in the UI (flat or everything appearing under Inbox).

Exactly. The result is all folder being shown on the same level as the Inbox folder rather than the default where all folders appear as subfolders of Inbox. I think that is the goal of setting the imap server directory in the first place, correct? It's not a big thing and TB can be used perfectly without it but I just noticed the possible bug so I reported it.

So, again, to be clear:

  1. With empty imap server directory:
Inbox
  Drafts
  Sent
  Archive
  ...
  1. With imap server directory set to "INBOX" (no quotes):
Inbox
Drafts
Sent
Archive
...
See Also: → 697706

Wow this is a really old bug :) I don't understand even half of it but thank you, Gene, for working on it!

I found way to duplicate the bug but haven't found a fix.
Start with a fresh install of an account for a server that puts everything as a subfolder of Inbox as shown in comment 5, item "1." This is default for dovecot, courier and cyrus.
Create a new folder under Inbox called archive. Don't create the folder Archives which is the default name, at least for en-US localization.
Go to Settings / Copies and Folders and select Other under archiving and with the folder "picker" choose folder Inbox/archive.
At the main screen, verify that archived messages get copied into Inbox/archive folder (or the year subfolder if that is wanted).
Now go to the Setting / Server Setting / Advanced... and enter into Imap Server Directory just INBOX (uppercase, no trailing spaces). Click OK.
At main screen, select Inbox and then restart TB.
On startup, all folders should be at same level as Inbox (nothing now appears as subfolder of Inbox, but to the server they still are).
Now try to archive a message.
The infinite loop is visible in status bar. Just let it keep trying...
While it tries, go back to Settings / Copies and Folder and you will now see that archiving Other is still selected but it says "Choose folder...".
Choose the folder "archive" (now the same level as Inbox) from the list and restart TB.
After restart, archiving to the non-standard "archive" folder should work OK.

I don't think the problem occurs if you just stay with the default name Archives and don't choose Other but instead use the default selection Archives folder on:.

I think the problem is caused by the preference mail.identity.idX.archive_folder. The default setting looks like this example (with and without the imap directory set to INBOX):
imap://gds%40tana.it@fwmail.tana.it/Archives; but when Other is selected the folder Inbox/archive is originally set, you see this:
imap://gds%40tana.it@fwmail.tana.it/Inbox/archive which works OK as long at the Imap server directory setting is empty. But when imap server directory is set to INBOX, which "flattens" the paths, the URL is wrong until you fix it in the Other setting (or maybe also with config editor?) so that it becomes
imap://gds%40tana.it@fwmail.tana.it/archive (with no Inbox) and then the TB is no long confused when you try to archive a message into the "non-standard" named folder, e.g., archive.

So I think an easy workaround when the "infinite loop" occurs is to just go to the Settings / Copies and Folders and make sure the Other selection for archiving is properly selecting the correct target folder for archiving (Inbox must not be in the path). Then restart TB.

Small additions from my side:

  1. I was not able to reproduce your solution, because here "choose folder" does not appear and even if I set the archive folder again manually, it doesn't work for me. I also tried switching back and forth and restarting inbetween and I also tried using the folder "Archives" (plural) - without luck. Fun fact: I never noticed the plural! On my iPhone it seems to be singular even with US l10n.

  2. One other thing I noticed is that once I created the folder "Archives", I wasn't able to remove it anymore even if it's not set at the archive folder. Is that an additional bug I should file? And I also noticed that when changing the imap server directory entry and restarting, all subfolders of Inbox are treated as completely new folders, have to be reloaded and also loose their settings (columns shown, sorting, etc.).

  3. The infinite loop after hitting "Archive" with the imap server directory set to "INBOX" consists of "downloading new" messages for the folder recognized as new and "looking for folders" alternating quickly. To me it looks more like TB somehow can't finish folder discovery for some reason.

(In reply to Nico from comment #8)

Small additions from my side:

  1. I was not able to reproduce your solution, because here "choose folder" does not appear and even if I set the archive folder again manually, it doesn't work for me. I also tried switching back and forth and restarting inbetween and I also tried using the folder "Archives" (plural) - without luck. Fun fact: I never noticed the plural! On my iPhone it seems to be singular even with US l10n.

Well, I am now able to duplicate the problem even without re-installing a fresh account (or even restarting). I thought for sure my fix would fix it for you too. When it is stuck in the loop and you are seeing "archive" (not "Choose folder...) in the bottom selection for archiving, could you look at this parameter in config editor and tell me what it says: mail.identity.idX.archive_folder, where x might be 0, 1, 2... depending on how many accounts you have. For me, when it is working right, I see imap://gds%40tana.it@fwmail.tana.it/archive, but when not working right I see this: imap://gds%40tana.it@fwmail.tana.it/Inbox/archive, i.e., it still has the Inbox in the url.

  1. One other thing I noticed is that once I created the folder "Archives", I wasn't able to remove it anymore even if it's not set at the archive folder. Is that an additional bug I should file?

I've noticed that too over the years that the "special" folder flag sometimes gets stuck on a folder even when it it no longer being used as a "special" folder. But what I've seen is that after a couple restarts usually it removes the flag and you can delete the folder.

And I also noticed that when changing the imap server directory entry and restarting, all subfolders of Inbox are treated as completely new folders, have to be reloaded and also loose their settings (columns shown, sorting, etc.).

That's probably true too since all URLs to access the folders get altered. In my tests it just with a 'throw-away" account with few messages and I set it up to only download headers so I haven't noticed this issue. If things were working right, it would be one-time re-download and not a big deal, but since you are having this issue I see it could be a problem if you have a lot of messages.

  1. The infinite loop after hitting "Archive" with the imap server directory set to "INBOX" consists of "downloading new" messages for the folder recognized as new and "looking for folders" alternating quickly. To me it looks more like TB somehow can't finish folder discovery for some reason.

I don't think it's a folder discovery failure or hang. I see it stuck in an infinite loop in the javascript code that handles archiving. It just keeps asking the imap protocol code to check if the "archive" folder exists and then subscribes it over and over. I have been trying to figure out why.
If you run an IMAP:4 log as described here (https://wiki.mozilla.org/MailNews:Logging), when stuck in the loop you should see repeating "ensureexists" URLs followed by subscribe to Archive. At least that's what I see.
If you don't see the repeating "ensureexists" and subscribes to archive in the log, I would be curious to see what your IMAP:4 log says while stuck in the loop. You can attach it above.

Nico,
When it's not working right, it might be useful to also see all the archive_* parameters for the problem account, e.g.,

user_pref("mail.identity.id2.archive_enabled", true);
user_pref("mail.identity.id2.archive_folder", "imap://gds%40tana.it@fwmail.tana.it/gmail-inbox");
user_pref("mail.identity.id2.archive_granularity", 0);
user_pref("mail.identity.id2.archive_keep_folder_structure", false);
user_pref("mail.identity.id2.archives_folder_picker_mode", "1");

I took this from the prefs.js file located at the top level of the profile. These should match what you see in config editor. I'm just using a random folder on tana.it server called "gmail-inbox" as my archive folder for testing (has nothing to do with gmail).
Note: the first item, archive_enabled, may not be present for your account since it defaults to true if not present.

Gene, I will do that and post the results but might need until Monday. Thanks for the instructions :-)

(In reply to Nico from comment #11)

Gene, I will do that and post the results but might need until Monday. Thanks for the instructions :-)

That's fine.

However I may have found another way to make it work for you. Note: When I say "check the parameter" below, those steps are optional.

Since just choosing or re-choosing "Archive" doesn't work for you, check if the parameter mail.identity.idX.archive_folder contain Inbox in the path. My bet is that it does, e.g., "..../Inbox/Archive". Then try choosing another random folder, doesn't matter which one. Then check that that folder now appears at parameter mail.identity.idX.archive_folder in config editor without "Inbox" in the path. Then re-select "Archive" and check that mail.identity.idX.archive_folder now does not contain Inbox in the path and, of course, ends with "Archive". Now see if archiving works without going into the infinite loop.

I set up a more complex account on a Cyrus server (more messages and folders) and the first time I changed from "Choose folder" to "Archive" the parameter archive_folder still contained Inbox. So I selected another random folder and then re-select "Archive" and archive_folder did not contain Inbox and archiving worked OK into "Archive" folder.

If that works, probably won't need more info like the IMAP:4 log or the parameter details I asked for in comment 10.
Thanks and have a good weekend!

Hi Gene,

I just took some time and I think I found the problem(s) by looking at the Settings Editor and the key mail.identity.idX.archive_folder. There are two ways of setting the folder preferences:

  1. Edit -> Account Settings -> Copies & Folders
  2. Edit -> Account Settings -> <account_name> -> Manage Identities -> <identity> -> Copies & Folders

I have one IMAP/SMTP account with two identities: one main identity for private mails and a second identity for a mail alias for work mail. I always used method 1 which only changed the archive folder for the main identity. That is why it sometimes worked and sometimes not! When I checked out mail.identity.idX.archive_folder I saw that there were two archive folder settings, one for each identity. Now I used method 2, renamed the folder "Archive" to "Archives" (mostly for standards compliance ;-), restarted TB and it works perfectly.

So I guess the only technical "bug" would be that you have to restart TB in order for the change to take effect and that isn't mentioned in the UI. But in addition I guess I would expect method 1 to change the folder preferences for all identities of the account or at least state which identity gets changed. And maybe TB automatically adapting the folder preferences in case of a new IMAP server directory setting. Also, maybe TB could even auto set "INBOX" for the mail servers that use that structure as other mail clients do but those points are surely debatable.

Is it feasable to file change requests?

Thank you for your help and the tip with using the Settings Editor!

Ok, glad you found a solution.
It didn't occur to me that you were using two identities. It did occur to me the other day that one can also set the archive folder in the identity settings. Not sure why you would need a different archive folder for a different identity, but I guess some do.
If you always use the default "Archives" folder on the server and don't try to set a special one, it should always work OK, even with INBOX set as the "server directory". Not so good if you want a custom folder for archiving and INBOX server directory is set.

Also, maybe TB could even auto set "INBOX" for the mail servers that use that structure as other mail clients do but those points are surely debatable.

I don't know if you are familiar with gmail in TB but TB now does something similar with it. There is a special [gmail] folder containing most of the folders that before 115 were always visible in light gray. Now [gmail] is hidden and the folders are all now in a straight line (flattened). Something similar could be done for accounts with Inbox at the top level, e.g., (cyrus, dovecot, courier etc) without the user having to know about "server directories" or "namespaces". But not sure it should be the default setting like it now is for gmail since some users don't mind seeing Inbox as the root folder.

you have to restart TB in order for the change to take effect

Yes, that is true, especially for things set in config editor and "advanced" imap parameters. For efficiency some parameters are read once at startup and saved to RAM and not re-read during runtime each time they are used. So I usually suggest a restart when changing any low-level parameters since you never know when it takes effect.

Is it feasable to file change requests?

Yes, in bugzilla there is an "enhancement" category for the bug report. If you do that, I would recommend starting a new bug.

Alright I will do some research and might then make some suggestions. I'm somewhat puzzled that this wasn't really a bug, because it does feel like one :o) Thanks so much again!

I'm somewhat puzzled that this wasn't really a bug ...

If by "this" you mean the bug you reported in comment 0, it is definitely a bug. However, some bugs are very difficult to fix without maybe re-writing a ton of code. I have a feeling this may be one. That's why you can see at the bottom the possible resolution as "WONTFIX".

But maybe you are referring to your proposed enhancements?

As for the "identity" issues with archive folder, I've been working on TB for about 7 years off and on and only a few months ago even became aware of what is meant by identity in TB.

As for auto-flattening under INBOX, it's not something I would really be in favor of. I didn't agree with the [Gmail] flattening either but have only heard one or two complaints and I think those were resolved. (I try to stay out of UI issues.)

Thank you for your thoughts on this, interesting! Regarding the "bug" in comment 0 I'm not so sure whether I follow. It did work in a few cases and it now seems clear that those were the cases when the recipient of the email was the address of the main identity. Mails to the secondary identity did not work, because the archive folder had not been properly set for it.

One could, however, argue that TB should auto correct those settings for all identities of an account after changing the IMAP server directory. It seems to me that this already happens for the other special folders (sent, templates, etc.), correct?

But I agree that "auto-flattening" is not absolutely necessary and might be seen by some as hiding a technical circumstance which has no implications other than looking different.

Bottom line: I create a change request for auto correction of archive folder settings for all identities of an account in case of IMAP server directory change. Agree?

One could, however, argue that TB should auto correct those settings for all identities of an account after changing the IMAP server directory. It seems to me that this already happens for the other special folders (sent, templates, etc.), correct?

While working on this I also tried to confuse the "save to sent", templates, and drafts with the INBOX directory thing and they were not confused, unlike archiving. I determined that is because they don't do an imap copy or move when saving to sent, drafts or templates. What they do is an imap append command to the server which basically saves the message into a temp file and then sends the file content to the server. So it uses a different transfer method. Saving an archive tells the server to move or copy the messages to a different folder so the whole message is not sent to the server. Archiving also seem to support the "undo" mechanism which is another complication that I don't think the others have.

Bottom line: I create a change request for auto correction of archive folder settings for all identities of an account in case of IMAP server directory change. Agree?

Sounds good.

Please check https://bugzilla.mozilla.org/show_bug.cgi?id=1888070 and see if I summarized everything correctly! Thanks!

See Also: → 1888070

I'm closing this as WONTFIX but we can continue any discussion of any future changes/fixes or work-arounds over at bug 1888070.

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.