123721 - Offline move/copy across imap servers fails

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Backend, defect)

Description

[David :Bienvenu]

If you do an offline move/copy of messages across imap servers, the offline
playback fails when you go online. Fix upcoming.

Comment 1

[David :Bienvenu]

Attachment: proposed fix

fix is to check if we're move/copying across imap servers, and use the generic
folder move/copy in that case.

Comment 2

[Navin Gupta]

Comment on attachment 68063 [details] [diff] [review]
proposed fix

r=naving

Comment 3

[(not reading, please use [e-mail] instead)]

Comment on attachment 68063 [details] [diff] [review]
proposed fix

sr=sspitzer

is this a spurious comment:

+// If there

Comment 4

[David :Bienvenu]

fix checked in.

Comment 5

[grylchan]

David,
I am reopening this bug.

using commercial builds:
2002-02-15-10-trunk nt 4.0
2002-02-15-13-trunk linux 2.2
(can't try mac due to that known mac bug)

copy works fine.

Move doesn't work. I tried drag/drop, context menu, and file menu
move and neither of them worked. Tried both imap and webmail accounts.

What happens is this:  
    - after downloading some mesgs go offline
    - try moving megs between acnts by any means
    - appears to work as I can click/read the newly moved mesg while offline
    - go online
    result: error mesg appears:"The current command did not succeed.
            The mail server responded: Message has no header/body
            seperator".

            Part of the error mesg (according to lxr) is in imapMsgs.properties 
            but I don't know where the part "Message has no header/body
            seperator" comes from. My search on lxr came up with these
            files: nsStreamConverter.cpp, mimemoz2.cpp,nsHttpTransaction.cpp

            Then I get the new msg arrow on the account and the
            mesgs that I moved,are now are back in their original folder

            I also saw this error mesg: "The current command did not succeed.
            The mail server responded: Invalid message in APPEND."
            I don't see this mesg as consistently as I see the other one.

Comment 6

[David :Bienvenu]

Gary, that message is coming from the imap server, complaining that the message
we're trying to append was incorrectly formed. Could you attach an imap protocol
log of the failed offline playback of a move? That msg means we're trying to
playback the offline move, but it's failing for some reason.

Comment 7

[grylchan]

Attachment: protocol log between my judge and webmail act when moving mesgs

I moved a mesg entitled 'tree status' and another one
called 'webmail 3'.

Comment 8

[grylchan]

Attachment: smaller atachment: protocol log

used carbuncle and parp imap mail accounts to move mesg.
This log is much smaller than judge/webmail one.

Comment 9

[grylchan]

This has fallen under the radar.
hmm nominating this for nsbeta1. 
Since I think people might be doing this between servers.

Using 2002032803 commercial trunk on nt 4.0

If I try to move or drag downloaded mesgs only from 
one imap mail folder to another imap mail folder while offline:
  -it appears to work. I can click on it, read downloaded mesg
  -but when I go back online this error mesg now appears
     'The Current Command did not succeed. The mail server
      responded: Message had no header/body seperator'
      Same as comment 5.

It works fine in offline if:
  -move/drag non-downloaded mesgs between imap accnts
  -copy downloaded/non-downloaded mesgs betweeen imap acnts

it's having problem with downloaded mesgs.

Comment 10

[Michael Buckland]

Discussed at Mail News bug meeting.  Decided to make ADT2 and plus this bug.

Comment 11

[David :Bienvenu]

I think what caused this was the changes to make multiple copying of local
messages to imap folder work. Now, it looks like releasing the url state and
closing the mock channel is causing the saving to the temp file to get
stopped/cancelled before we can write out data to the temp file. That results in
an empty temp file, which we try to append to the destination imap folder.
That's just a guess, but it looks likely.

Comment 12

[David :Bienvenu]

upon more investigation, I'm not sure that change did cause the problem.

What's happening is we read the message from the offline cache in order to save
it to the temp file before appending it to the other imap server. When we finish
with that read, the file transport sends us an on stop request. Unfortunately,
we've already got a copy listener attached to this operation, and it's really
waiting for the on stop request for the whole download to the temp file to be
done, i.e., on the write. It wouldn't shock me if the file transport has changed
in this respect - I'll have to investigate. My guess is that move/copying an
offline message to another server or even a local folder won't work, even when
online...I'll investigate more. 

nsImapService::AppendMessageFromFile(nsImapService * const 0x041e1a90,
nsIEventQueue * 0x01461198, nsIFileSpec * 0x05898288, nsIMsgFolder * 0x0449966c,
const char * 0x039853aa, int 0x00000001, int 0x00000000, nsIUrlListener *
0x044996d8, nsIURI * * 0x00000000, nsISupports * 0x05129b78, nsIMsgWindow *
0x0436cc60) line 2241
nsImapMailFolder::EndCopy(nsImapMailFolder * const 0x04499728, int 0x00000001)
line 2747 + 158 bytes
nsCopyMessageStreamListener::EndCopy(nsCopyMessageStreamListener * const
0x05129c14, nsISupports * 0x05829d4c, unsigned int 0x00000000) line 162 + 30 bytes
nsCopyMessageStreamListener::OnStopRequest(nsCopyMessageStreamListener * const
0x05129c10, nsIRequest * 0x05822008, nsISupports * 0x05829d4c, unsigned int
0x00000000) line 199
nsImapCacheStreamListener::OnStopRequest(nsImapCacheStreamListener * const
0x0561fc48, nsIRequest * 0x05996b2c, nsISupports * 0x05829d4c, unsigned int
0x00000000) line 6999 + 43 bytes

Comment 13

[David :Bienvenu]

hmm, I take that all back. What it looks like is that we're not able to read
from file transport to download the message to the temp file, and that's why we
get the stop request - because the read failed. I'll attach a segment of the
file transport log that shows a successful read from the offline store, followed
immediately by an unsuccessful open/read. It's as if the transport isn't
initialized with the file size and read size for the last one...

Comment 14

[David :Bienvenu]

Attachment: file transport log

this is the file transport log.

Comment 15

[David :Bienvenu]

sigh - so the problem is in nsMsgDBFolder::GetOfflineFileTransport, when we take
the msg hdr and figure out the offline offset and offline size for the msg in
question, we get 0. I'm not sure if someone's crunched the offline offset and
size, or if we're looking at the wrong msg hdr. My guess is the former, perhaps
because we're moving the message.

Comment 16

[David :Bienvenu]

Attachment: proposed fix

boy, was I off track. The problem, which I suspect has been around for a long
time,  is that we delete the source msg hdr, but if you don't shut down the app
and restart, we still have the msg hdr in memory, and we still think we have an
offline copy of the message because m_flags still has the offline bit set.  (we
cache the m_flags member variable). When we go to find the offline size and
offline offset, however, we get them from the mork columns, which have all been
deleted, so we get 0 for each value. The fix is to clear the m_initedValues
member when we delete a header so that we'll try to get all values from the db,
and they'll come back as 0, which will mean that we don't think the message is
offline anymore. I'll need to test this change to make sure it doesn't cause
any regressions, but it fixes this problem for me.

Comment 17

[David :Bienvenu]

Comment on attachment 68063 [details] [diff] [review]
proposed fix

obsoleting checked in patch.

Comment 18

[David :Bienvenu]

Navin, can I get a review for the last attachment? thx.

Comment 19

[Navin Gupta]

Comment on attachment 77600 [details] [diff] [review]
proposed fix

r=naving

Comment 20

[Jaime Rodriguez, Jr.]

adt1.0.0-, nsbeta1-. Not many people will encounter this one.

Comment 21

[David :Bienvenu]

fix checked into trunk.

Comment 22

[grylchan]

Using commercial trunk builds
2002041603 Win NT 4.0
2002041603 Mac 10.1.3, Mac 9.2.2
2002041607 linux 2.2

I can now move downloaded mesgs, while offline, between different
imap mail accounts with no problem. When I go online, the mesgs
I moved become brand new mesgs and trigger Biff but that's to be expected
I assume.


Tested moving more than 1 mesg, moving to subfolders, and copying
and no problems while doing this offline.

marking verified on trunk only.