Dragging "deep" trees from one IMAP account to another does not copy everything

RESOLVED WORKSFORME

Status

Thunderbird
Mail Window Front End
RESOLVED WORKSFORME
9 years ago
8 years ago

People

(Reporter: Alain Knaff, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

9 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9) Gecko/2008052912 Firefox/3.0
Build Identifier: 2.0.0.18 (20081125)

Recently, I helped a friend move his mail to a new provider. I thought to myself "this is going to be easy, just set the two accounts at once in his Thunderbird, then drag everything from old to new".

In reality, this turned out to be rather laborious.
1. I was unable to select more than one top-level folder (with Control-Click or Shift-Click), but had to move each folder one-by-one
2. There was a bug: when moving a folder containing subfolders, the subfolder's messages would not be moved.
3. At random times, I became completely impossible to move anything at all, and I had to restart Thunderbird to continue moving stuff
4. There is a consistency problem: dragging folders (left-hand pane) copies them, whereas dragging messages (from right-hand pane) moves them.

Reproducible: Always

Steps to Reproduce:
The following steps show how to reproduce problem 2 above:

1. On an Imap server that supports folders containing both subfolders and messages (such as dovecot with maildir), set up two accounts mailtest1 and mailtest2
2. Add account mailtest1 to Thunderbird
3. Add account mailtest2 to Thunderbird
4. In mailtest1 set up a folder named parent
5. In mailtest1's parent folder set up a subfolder named child1
6. In mailtest1's parent folder set up a subfolder named child2
7. Send 3 messages to mailtest1
8. As mailtest1 drag them into each folder: one in parent, one in child1, onde in child2.
9. Now drag the entire "parent" tree to mailtest2.

Step 9 should copy everything. However, in mailtest2, child1 and child2 will be empty. The message directly under parent does make it however.
Actual Results:  
Mailtest2's parent/child1 and parent/child2 folders will be empty (no message). Parent will have its message

Expected Results:  
Mailtest2's parent/child1 and parent/child2 should have its message
> Actual Results:  
> Mailtest2's parent/child1 and parent/child2 folders will be empty (no message).
> Parent will have its message.

Get IMAP log, and check "what is done as expected by Tb" and "what is not done correctly by Tb". ( http://kb.mozillazine.org/Session_logging_for_mail/news )
Attach log file, if "Tb's fault" is found in log and log analysis by developers will be required.
WORKSFORME with Tb latest-trunk on MS Win-XP SP3.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre)
> Gecko/20081214 Shredder/3.0b2pre

Tested with Gmail IMAP(imap.google.com).
(0) Preparation
   (Folder structure of Account-1)
      Test-001 (no mail is held)
        + X1 (mail-000 is held)
            + X1-01 (mail-001 is held) 
            + X1-02 (mail-002 is held)
   (Folder structure of Account-1)
      Test-001 (no subfolder)
(1) Drag "X1 under Test-001 of Account-1" to "Test-001 of Account-2".
    => Folder X1, X1/X1-01, X1/X1-02 were created. Mail exists in any folder.

If your test mails in child1 and child2 already exist in mailtest2 (in [Gmail]/All Mails, [Gmail]/Sent Mails etc. of mailtest2), mail in child1/mail in  child2 will disappear after "OK to APPEND command", because duplicate mail check by Gmail works.
(a) Mail in child1 and mail in child2 are mails sent from mailtest2.
    Mail data is already held in "[Gmail]/All Mails" upon mail send,
    and is labeled as "Sent Mails" (a mail in "[Gmail]/Sent Mail" older for Tb).
(b) Mail in child1 and mail in child2 is copy of mail in parent.
    Mail data is held in "[Gmail]/All Mails" by first APPEND to parent.
    So APPENDed mail to child1/APPENDed mail to child2 disappear.
(c) Test was executed multiple times.
    Mail data is already held in "[Gmail]/All Mails" by first successful
    "APPEND" in first test.
Please exclude such cases.
Correction of previous post.
"Appended or copied mail disappears(by DUP check)" occurs only when a same mail is appended or copied to a mail folder multiple times.
So "Appended or copied mail disappears" won't occur in your case.
Sorry for my confusion on Gmail's behaviour.
(Reporter)

Comment 4

9 years ago
While trying to gather the logfile, I noticed that the problem only happens when the mails in the source folder are "new", i.e. have not yet seen by thunderbird.

Such a situation can (for instance) be created by using a different mail client for setting up and populating the source account.

The define both accounts in the Thunderbird under test. The problem then occurs.

This is also what happened with my friend. Another friend had helped him switch to the new provider earlier, but simply changed the host name, but forgot to copy the mails. When we noticed that oversight, I had to define the old account anew, i.e. for Thunderbird this was now an account which it had "never" seen before.

When trying to reproduce the problem at home, I moved the mails from INBOX to the child1 and child2 folders, but without visiting them afterwards. Probably, that's why Thunderbird didn't have them in its cache.
(Reporter)

Comment 5

9 years ago
Created attachment 353071 [details]
nspr.log describing the "deep copy" problem
(Reporter)

Comment 6

9 years ago
Any news on this item?
(Reporter)

Updated

9 years ago
Version: unspecified → 2.0

Comment 7

9 years ago
Did you try http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-central/ (WFM per comment 2)

I know it used to be an issue, but didn't see it last time I tried to reproduce.
Alain can you update us please ?
Whiteboard: closeme 2009-09-24
(In reply to comment #5)
> nspr.log describing the "deep copy" problem

maltest1 has next folders.
> * LSUB () "/" "parent"
> * LSUB () "/" "parent/child1"
> * LSUB () "/" "parent/child2"
Tb created/subscribed "parent", "parent/child1", "parent/child2" at maltest2, issued fetch to "parent" at mailtest1(1 mail exists), and appended the mail to "parent" at mailtest2.
fetch to "parent/child1", "parent/child2" at mailtest1 is not seen in log.
How many mails was held in "parent/child1", "parent/child2" at mailtest1 when you tested?

MULTIAPPEND is relevant? (I tested Gmail IMAP which has no MULTIAPPEND)
Drga&Drop of root-level folder is relevant? (I tested Drag&Drop of second level) 

FYI. Bug 458570 can produce phenomenon like comment #0.
Additional question.
If number of mails in "parent/child1" and/or "parent/child2" is greater than ZERO, did Tb already know it when you executed Drag&Drop?
AFAIR, I read description like next in a bug.
  If Tb doesn't know existence of mail in subfolders, Tb doesn't do fetch from
  source folder and append to target folder, because mail count Tb knows==ZERO.
RESO INCO per lack of response to last question. If you feel this change was made in error, please respond to this bug with your reasons why.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → INCOMPLETE
(Reporter)

Comment 12

8 years ago
Just tested this in firefox 3.0b4, and the problem still occurs.

And what's worse: if the target account was not yet logged in, the drag-and-drop won't trigger a password prompt, and instead nothing happens at all, leaving the user to wonder. Even logging in afterwards doesn't fix this: drag-and-drop of folders still does nothing at all.

To get it back to a level as functional as before, you need to make sure target account is logged in _before_ attempting the drag-and-drop. In that case, message directly in moved folder is moved, whereas any messages in subfolders are ignored (if these folders have not been used yet).

Again: this seems to be tied to whether the subfolders had been "visited" before or not.

It seems to me that the moving copying operation doesn't make sure its pre-conditons are met before proceeding, but rather relies on whatever happens to be cached due to previous operations. Preconditions such as "do I have a list of messages in source?", or "is target logged in?"
Status: RESOLVED → UNCONFIRMED
Resolution: INCOMPLETE → ---

Comment 13

8 years ago
(In reply to comment #12)
> Just tested this in firefox 3.0b4, and the problem still occurs.
> 
> And what's worse: if the target account was not yet logged in, the
> drag-and-drop won't trigger a password prompt, and instead nothing happens at
> all, leaving the user to wonder. Even logging in afterwards doesn't fix this:
> drag-and-drop of folders still does nothing at all.

Alain, if you still see the above issue when NOT logged in, please file a separate bug and report the bug# here.

> To get it back to a level as functional as before, you need to make sure target
> account is logged in _before_ attempting the drag-and-drop. In that case,
> message directly in moved folder is moved, whereas any messages in subfolders
> are ignored (if these folders have not been used yet).
> 
> Again: this seems to be tied to whether the subfolders had been "visited"
> before or not.
> 
> It seems to me that the moving copying operation doesn't make sure its
> pre-conditons are met before proceeding, but rather relies on whatever happens
> to be cached due to previous operations. Preconditions such as "do I have a
> list of messages in source?", or "is target logged in?"

possible related bugs https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=nowordssubstr;type0-1-0=nowordssubstr;field0-1-0=component;short_desc=drag;field0-0-0=short_desc;bug_severity=major;bug_severity=normal;resolution=---;query_format=advanced;value0-1-0=imag%20%20address%20compos%20attach%20eml%20rss%20feed%20icon;value1-0-0=closeme;short_desc_type=allwordssubstr;type0-0-0=nowordssubstr;value0-0-0=imag%20address%20compos%20attach%20eml%20rss%20feed%20icon%20header%20;field1-0-0=status_whiteboard;product=MailNews%20Core;product=Thunderbird

Updated

8 years ago
Whiteboard: closeme 2009-09-24
(Reporter)

Comment 14

8 years ago
Indeed, on Thunderbird 3 (Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.5) Gecko/20091209 Fedora/3.0-3.fc11 Lightning/1.0b1 Thunderbird/3.0), the original problem (messages in subfolders not copied) seems to be gone now (unless it is merely masked by Thunderbird's new overly agressive caching, see bug #529857)

However, the new problem that I raised in my last message still persists, I filed it as a new bug #538191, as requested
Thanks for the feedback Alain.

WFM per c #14
Status: UNCONFIRMED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.