697706 - archive_keep_folder_structure not effective when archiving messages from Inbox

Status: RESOLVED FIXED

(Thunderbird :: Account Manager, defect)

Description

[Arrigo Marchiori]

When the variable "archive_keep_folder_structure" messages is set to "true", messages coming from Inbox should be archived into into Archives\Year\Inbox.
The problem is that messages are archived into Archives\Year instead.
Messages from Inbox\Subfolder are archived into Archives\Year\Subfolder. This shows that the "Inbox" directory is just skipped.

Software version: Thunderbird 7.0.1, under both FreeBSD and Windows XP, with Lightning extension installed (both enabled and disabled).

Steps to reproduce:
1- create an empty profile
2- create a dummy e-mail account and set it so that:

   mail.identity.id1.archive_folder = mailbox://user@localhost/Archives
   mail.identity.id1.archive_granularity = 1
   mail.identity.id1.archive_keep_folder_structure = true

3- start writing a message, save it in drafts
4- move the draft into the Inbox folder
5- right-click the message and select "Archive".
   The message is moved into Archives\2011
6- if you create other folders (e.g. sub-folders of Inbox) you can reproduce the effects described above: the folder structure is kept except for the level "Inbox".

The Error Console only shows one entry (but I am not sure it related):
> Warning: Use of getAttributeNodeNS() is deprecated. Use getAttributeNS() instead.
> Source File: chrome://messenger/content/messenger.xul
> Line: 0

Thank you in advance!

Comment 1

[Hashem Masoud]

(In reply to Arrigo Marchiori from comment #0)
More info in bug 630784

Comment 2

[Arrigo Marchiori]

(In reply to Hashem Masoud from comment #1)

Thank you for the pointer, Hashem.

However, I should add that this bug is not related to IMAP. The test I proposed above does not configure any POP3 or IMAP servers, and my mail servers are all POP3.

Comment 3

[WADA:World Anti-bad-Duping Agency]

Confirmed with Tb 7.0.1.

Archive setting of a POP3 account, which are set via UI of Account Settings/Copies&Folders, Message Archives, Archive Options.
(only one identity is defined for this account)
> mail.identity.idX.archive_enabled = true (usually not shown in about config if true)
> mail.identity.idX.archives_folder_picker_mode = 0 (Archives of an account)
> mail.identity.idX.archive_folder = mailbox://y-01@y.y.y/Archives (Archives of my own)
> mail.identity.idX.archive_granularity = 0 (A single folder)
> mail.identity.idX.archive_keep_folder_structure = true

  Folder where a mail is held    Folder the mail is archived to
   root-level Inbox               Archives             <= this bug
   root-level XXX                 Archives/XXX         <= as expected
   Inbox/AAA                      Archives/AAA         <= this bug
   Inbox/Inbox/BBB                Archives/Inbox/BBB   <= this bug
   XXX/YYY                        Archives/XXX/YYY     <= as expected

Problem occurs with any valid archive_granularity value.

Comment 4

[Luke Filewalker]

Hello

Any news about this bug? It's still present in current TB 14.0.

Comment 5

[Arrigo Marchiori]

This bug is still present on version 15.0.

Comment 6

[Arrigo Marchiori]

Bug still present on version 24.1.0

Comment 7

[krzysztof.krason]

Still occuring in EarlyBird 35.0a2

Comment 8

[Marcel Gelinas]

Still occurring.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Comment in source.
> http://mxr.mozilla.org/comm-central/source/mail/base/content/mailWindowOverlay.js#1782
> 1786       if (archiveFolder.canCreateSubfolders && batch.keepFolderStructure) {
> 1787         // Collect in-order list of folders of source folder structure,
> 1788         // excluding top-level INBOX folder
It looks intentional.
Affected by support of namespace="Inbox." imap server?
If so, I yhink mail.server.server#.namespace.personal etc. is better counted when keeping FolderStructure, or "exclding top level Inbox" is better optional, in order to avoid confusion of POP3 users and no-namespace imap server users. .

Comment 10

[u563314]

This is a 4 year old issue, could a DEV comment? Would anyone (@WADA) know the reason why this is intentional?

Comment 11

[u563314]

Could anyone point me in the right direction on how could I can modify the code myself? If I understand correclty Thunderbird is a JS application. So at first I attempted to locate the `mainWindowOverlay.js` in the install folder but it was not found, then I did a grep search for `excluding top-level INBOX folde` (http://mxr.mozilla.org/comm-central/source/mail/base/content/mailWindowOverlay.js#1813) but still no luck.

Comment 12

[Kent James (:rkent)]

You are looking in the right place for the archive code in mailWindowOverlay.js  Although not all of TB is JS, the archive methods are all in JS.

The correct way to adapt the behavior would be an extension. Unfortunately, in one of my extensions where I adapted the archive behavior, I have this note:

  function processNextBatch_exquilla()
  {
    log.debug("Calling overridden processNextBatch");

    // I really hate to use a frozen version of this. I need to add a hook in the core
    //  code instead. See bug 832034.

That is, there are not adequate hooks to override this in the existing code.

Comment 13

[luti]

Have just installed Thunderbird 78.3.1 in new CentOS 8 installation these days - I can't believe this annoying "feature" is still present...

In particular, because I have some installations (Thunderbird installed between 2015 and 2019 in CentOS 7 and Fedora 29 / 30 / 31 / 32) where this is respected (so I get messages to Archives-2020-Inbox, as expected...). The last one at Fedora 32, installed last year (in Fedora 29 at that time), and regularly updated by now. No problem there.

Is there any (hidden?) setting to achieve this - by my opinion just normal - behavior? Or, any other workaround?

Comment 14

[glukout39]

For version Thunderbird 91.3.2 (64-bit),
archive: omni.ja
file: modules\MessageArchiver.jsm
possible problem code location: line 252

while (folder != rootFolder && folder != inboxFolder) {
folderNames.unshift(folder.name);
folder = folder.parent;
}

INBOX folder removed archive from structure.
What the reason to process inboxFolder other than any folders?
This turns "keep original folder structure" to "keep original folder structure, EXCLUDE INBOX" :(
User don't get expected structure.

Comment 15

[glukout39]

My proof of concept test i made on Thunderbird 91.3.2 (64-bit):

1. I created folder "zabbix" as "INBOX" subfolder on IMAP account.

2. Copied any message to "INBOX\zabbix" folder.

3. I created local folder "archive"

4. Configured archiving to "archive" folder, "one folder" option (no "by years", no "by monthes"), check "keep original folder structure".

5. Right mouse button on message in "INBOX\zabbix" - Archive action.

6. Thunderbird created "zabbix" folder as "archive" folder and place message to "archive\zabbix".

7. Extract file "modules\MessageArchiver.jsm" from "C:\Program Files\Mozilla Thunderbird\omni.ja".

8. Change line 252 from
   while (folder != rootFolder && folder != inboxFolder) {
   to
   while (folder != rootFolder /*&& folder != inboxFolder*/) {
   then repack file back to onmi.ja

9. Repeate archive message action as (5) above.

10. Thunderbird created "INBOX\zabbix" folder as "archive" subfolder folder and place message to "archive\INBOX\zabbix".

Comment 16

[R]

Someone here said the year folder is the actual INBOX. Another one mentioned that the dialog box in the archive settings leads astray.

See this link from 2016: https://support.mozilla.org/en-US/questions/1103531

I can't think of any sensible reason for this behaviour. Or is there an explanation?

Comment 17

[gene smith]

While working on bug 1880014 I noticed that when messages in Inbox were archived (with include directory option) that Inbox folder were never created. All other folders had their folder name created. I thought it was maybe related to bug 1880014 but found this.
Also, the examples in the archiving option dialog show, for maintaining folder structure, "Inbox" as the only example.
I tried the change suggest in comment 15 and comment 16 and it fixes the issue so a message archived from Inbox now appears in an Inbox folder created under archives.

diff --git a/mail/modules/MessageArchiver.sys.mjs b/mail/modules/MessageArchiver.sys.mjs
--- a/mail/modules/MessageArchiver.sys.mjs
+++ b/mail/modules/MessageArchiver.sys.mjs
@@ -285,25 +285,24 @@ MessageArchiver.prototype = {
         }
       }
     }
 
     // Create the folder structure in Archives.
     // For imap folders, we need to create the sub-folders asynchronously,
     // so we chain the actions using the listener called back from
     // createSubfolder. For local, createSubfolder is synchronous.
     if (archiveFolder.canCreateSubfolders && batch.keepFolderStructure) {
-      // Collect in-order list of folders of source folder structure,
-      // excluding top-level INBOX folder
+      // Collect in-order list of folders of source folder structure.
       const folderNames = [];
       const rootFolder = srcFolder.server.rootFolder;
       const inboxFolder = lazy.MailUtils.getInboxFolder(srcFolder.server);
       let folder = srcFolder;
-      while (folder != rootFolder && folder != inboxFolder) {
+      while (folder != rootFolder) {
         folderNames.unshift(folder.name);
         folder = folder.parent;
       }
       // Determine Archive folder structure.
       for (let i = 0; i < folderNames.length; ++i) {
         const folderName = folderNames[i];
         if (!dstFolder.containsChildNamed(folderName)) {
           // Create Archive sub-folder (IMAP: async).
           if (isAsync) {

I tried this with and without "INBOX." as the personal namespace and with and without INBOX. as the imap server directory. Also, tried it on POP3 account and seems OK there too. See comment 9 for theory as to why excluding Inbox is intentional.
If there really is a valid reason for keeping this the way it currently is, at least the archiving examples in the UI need to show a different folder than Inbox since, the way it works now, an Inbox folder is NEVER created under the archive folder.

Comment 18

[Wayne Mery (:wsmwk)]

I researched bug 1831494 and found this bug, and I agree with comment 9. Quite intentional.

The implementation of "Keep existing folder structure of archived messages" is via Bug 607295 - Provide UI for new archive granularity and folder structure options - on the heels of Bug 451995 - implement archive method for auto-saving mail.

In bugzilla I'm not finding a direct statement of how "Inbox" should be handled with Bug 607295 or later. Perhaps a reason to not create Archives/Inbox was to not stray from user's expectations from Bug 451995, that messages archived from Inbox go to the "Archives" folder.

Comment 20

[Wayne Mery (:wsmwk)]

If anyone finds examples of this at https://connect.mozilla.org/ please post them here - I didn't find any.

Comment 21

[Alexander Kriegisch]

Wayne Mery encouraged me to point to bug 1831494. It was just marked as a duplicate of this one, but it adds value by describing in detail and with an example how to easily reproduce this issue.

BTW, how can I mention a user? Something like @wsmwk does not seem to be working.

Comment 22

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 697706 - Add preference to recreate inbox folder when archiving. r=#thunderbird-reviewers

• Implement the mail.identity.default.archive_recreate_inbox preference and the corresponding nsMsgIdentity attribute archiveRecreateInbox. This defines how the inbox folder is handled when archiving while preserving the folder structure. If the pref is false, the inbox is treated like the root folder, if true, it is recreated like any other folder.
• Make this preference available in the account archive options. Update and fix the example shown there.

Comment 23

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 697706 - Migrate Archive Options strings to Fluent. r=#thunderbird-reviewers

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/5f803b631b49
Migrate Archive Options strings to Fluent. r=john.bieling
https://hg.mozilla.org/comm-central/rev/445334850d02
Add preference to recreate inbox folder when archiving. r=babolivier,darktrojan,vineet