365651 - [IMAP] Folder rename causes all headers to be downloaded again

Status: UNCONFIRMED

(MailNews Core :: Networking: IMAP, defect)

Description

[Phillip Susi]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1
Build Identifier: version 1.5.0.9 (20061207)

I am running a dovecot imap server and I just renamed last year's linux-kernel mailing list folder to linux-kernel-old.  It had well over 100,000 messages in the folder, and once renamed, thunderbird began downloading all 100,000+ headers again.  This took quite some time and is unnecessary as it already had all the headers prior to the rename.  


Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[David :Bienvenu]

I don't know that there's any guarantee on the part of the imap server that the renamed folder will use the same uid's, or have the same uid validity.

Comment 2

[Anthony DiSante]

I can confirm this bug.

Most folders won't have 100,000 messages in them, but it's common to have several thousand messages in a folder, and if you have the "download for offline use" checkbox checked, then it's EXTREMELY frustrating and time-consuming to have Thunderbird re-download the thousands of messages, just because you renamed the folder.

And I don't see why end-users should have to know or care about folder UIDs on the server.  Renaming a folder simply has nothing to do with changing the contents of that folder.

And even if the server does change the folder's UID, isn't there some way for Thunderbird to intelligently handle the situation, rather than treating it as a new folder when it isn't?

Comment 3

[Wayne Mery (:wsmwk)]

David in comment #1:
> I don't know that there's any guarantee on the part of the imap server that the
> renamed folder will use the same uid's, or have the same uid validity.

-> ENH?

Comment 4

[timeless]

ftp://ftp.rfc-editor.org/in-notes/rfc3501.txt
6.3.5.  RENAME Command
      The value of the highest-used unique identifier of the old mailbox
      name MUST be preserved so that a new mailbox created with the same
      name will not reuse the identifiers of the former incarnation,
      UNLESS the new incarnation has a different unique identifier
      validity value.  See the description of the UID command for more
      detail.

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
I think RFC 3501 merely says;
  If IMAP server returns same "unique identifier validity value" after rename, 
  IMAP server MUST preserve "highest-used unique identifier".

To timeless:
Is the RFC description guarantees that IMAP server preserves each UID for existing mails when same "unique identifier validity value" is returned?

Comment 6

[Wayne Mery (:wsmwk)]

bienvenu, are there implications, indirectly, for gloda?

Comment 7

[David :Bienvenu]

yes, gloda should just know that the name changed - but it should be getting a notification that the folder was renamed/moved, from the nsIMsgFolderNotificationService, so it should be able to do the right thing, as long as it realizes the newly downloaded headers are the same as the ones it knew about.

Comment 8

[Andrew Sutherland [:asuth] (he/him)]

If the headers are totally new and they do not have the "gloda-id" property from the 'original' header, gloda will re-index the message.  The re-indexing will realize it already knows about the message, but it will almost fully reprocess the message.  (The full-text search table will not need to be updated.)

If the "gloda-id" property comes across but events are still generated about the header (for example, a message added notification), gloda will re-index the message until the event-driven indexing queue is overwhelmed and gloda falls back to sweep indexing.  The sweep indexer will see the "gloda-id" property and decide it has no work to do.

It should not be too hard to modify gloda to avoid the redundant re-indexing, but I would like to better understand the actual series of notifications that gloda will see...

It sounds like gloda will see:
1) nsIMsgFolderListener.folderRenamed (or folderMoveCopyCompleted with aMove=false)
2) a slew of nsIMsgFolderListener.msgAdded notifications, containing headers that have been freshly generated from IMAP.

Comment 9

[David :Bienvenu]

Yes, those are the notifications I would expect.

As long as compact and rebuild index for local messages handle maintaining the gloda-id property, then those won't cause re-indexing.

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to comment #5)
> To timeless:
> Is the RFC description guarantees that IMAP server preserves each UID for
> existing mails when same "unique identifier validity value" is returned?

so is this bug INVALID?

Comment 11

[Wayne Mery (:wsmwk)]

(In reply to comment #10)
> (In reply to comment #5)
> > To timeless:
> > Is the RFC description guarantees that IMAP server preserves each UID for
> > existing mails when same "unique identifier validity value" is returned?
> 
> so is this bug INVALID?

auth, bienvenu, can you update as to whether this should be confirmed, dup, or closed?

Comment 12

[darkweasel]

I've just been affected by this - or something extremely related. Basically, when renaming an IMAP folder (I'm on Thunderbird 17.0.2 on OS X 10.8.2, it's a Google Apps account if that's relevant), Thunderbird treats that folder and its subfolders like a new folder. Not only does it download all headers/messages again, it also destroys existing sorting preferences in those folders and all filter rules that move messages into the renamed folders.

It should IMHO at least be possible for Thunderbird to recognize the equivalence between the old and the new folder when the renaming was done from within Thunderbird itself.

Comment 13

[Wayne Mery (:wsmwk)]

(In reply to darkweasel from comment #12)
> it also destroys existing sorting preferences in
> those folders and all filter rules that move messages into the renamed folders.

this part is bug 82262