87531 - offline doesn't download messages when filters are used

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Backend, defect)

Description

[Christopher Blizzard (:blizzard)]

Build is a 0.9.2 build on May 24, 2001.

If I use offline mail and try to download messages from a server where filters
are used the messages aren't downloaded from the folders where messages where
filed to.  It's rumored ( via Gary Chan ) that if you download them more than
once that the messages will be downloaded but I couldn't get that to work.

Comment 1

[David :Bienvenu]

I thought this was fixed before I went on vacation - are you really running a
may 24th build, or a june 24th build?

Comment 2

[grylchan]

There are problems David. Still trying to figure out
necessary steps to replicate them..(I was using yesterday's
build to test). Will have some more info later today..

Comment 3

[grylchan]

Still investigating but here's what I found out so far.
Using comm. branch
2001-06-26-06-0.9.2/ win nt 4.0
2001-06-26-04-0.9.2/ linux 2.2 red hat 7
2001-06-26-03-0.9.2/ mac os 9.0.4

test 1
1.create a brand new profile
2.set up your imap acct
3.have couple of folders besides inbox
4.set up filters to distribute mail to diff folders
5.Go to Download and Sync window
6.Click select button
7.Select just the folders that have filters on them
  Do not select the inbox
8.Click mail mesgs checkbox & Check work offline once download/sync is complete
9.click ok
10.Send mail from another mail account to yourself
   that will be filtered when you download/sync.
   Only send mail that will be filterd. Do not send general mail
   that will go to the inbox.
11.Go online
12.Don't click on any folder or don't do a 'get mesg'
13.Go to Download and Sync window
14.click ok
Result: no messages downloaded
Expected: news mesgs to be downloaded in my filtered folders

Test 2
1. Repeat steps 1-6 from test 1.
2. This time select INBOX and folders w/filters for download.
3. Make sure mail messages checkbox and work offline once
   download/sync is complete is checked
4. click ok
5. Send mail from another account to yourself that
   will be filtered. Do not send general mail that
   will goto the inbox.
6. Go online in the first mail account
7. Don't click on any folder or don't do a 'get mesg'
8. Go to Download and Sync window
9. Click ok
result: 
     1.Filtered folders show in bold w/new mesg counts.
       but when you click on the folder, no mesgs appear
       (windows I saw this). Once you click on each folder
       bold/new mesg count dissapears. 
     2 On Linux, the bold folder/new mesg count is persistent.
       I can actually see the new megs in my filtered folders.
       but when I click on them, they are NOT downloaded.

If I go back online (don't do anything else) and then repeat
the download/sync process, it will successfully download
the new megs and you are able to read them.

Test 3
1.Repeat test 2 steps exactly EXCEPT in step 5 when you send
  mail from another account. Send messages that will be
  filtered and messages that will goto your inbox.
Result:
  Inbox mesg is downloaded, but my filtered messages
  are not downloaded.

On windows:
I have also seen the folder count for new mesgs get confused.
Though I have downloaded and read my new mesgs, the new mesg
count for the folder still shows new mesgs and if I
repeat the download and sync process again, the same messages
get downloaded again. So I end up duping it.

I don't think it has to do with 2 seperate mail accounts
but not positive.


Chris's test scenario is below (from an email conversation):

> laptop.  It seems that it will download messages for the first account 
> ( the messages are italic instead of standard font ) however on the 
> second account the message bodies are never downloaded.  I can watch 
> going offline update all of the folder counts but it never goes out 
> and gets the bodies of all of the messages.
>
> There are only two differences that I know about with the accounts:
>
> o The one that works only has one folder selected for download ( INBOX )
>
> o The one that doesn't work has many folders selected and uses client 
> side filters.
   

Comment 4

[Christopher Blizzard (:blizzard)]

Another thing that I've noticed is that if you do something along the lines of this:

o Download messages and go offline.
o Go to a folder that has had messages loaded into it and start reading mail.
o Start reading and deleting messages.
o Go online and check for new messages.
o Mail will load the messages that you have deleted in that folder while offline
as new so you have to delete them again.

I haven't been able to reproduce this 100% otherwise I would be able to give you
better tests but I've seen it enough and with other related problems I've
stopped using offline for now. :(

Comment 5

[David :Bienvenu]

OK, the basic problem here is that we don't update the inbox first before
updating the other folders. So if you don't have the inbox selected for offline
use, we won't get new mail first before downloading other folders. And if the
inbox is selected for offline use, we will update it and filter headers into
other folders, but it's basically a race as to whether those other folders will
get downloaded before or after the headers are filtered into it. So the fix is
to update the inbox for each account, and wait until that is done before
downloading message bodies for the other folders in that account. This is not
trivial, but I'll have a crack at it.

Comment 6

[David :Bienvenu]

Attachment: proposed fix

Comment 7

[David :Bienvenu]

Navin and Seth, can I get reviews? The fix is to add a url listener to get new
messages so that when get new messages finishes, I can go onto the next
server,and when I'm done with servers, I can download the messages in each folder.

Comment 8

[Navin Gupta]

r=naving

Comment 9

[(not reading, please use [e-mail] instead)]

+      nsCOMPtr <nsINntpIncomingServer> newsServer =
do_QueryInterface(m_currentServer);
+      if (!newsServer) // news servers aren't involved in offline imap
+      {

why is that necessary?  It looks like AdvanceToNextServer() is already skipping
the news servers.

can AdvanceToNextServer() be changed to not depend on nsINntpIncomingServer and
only on nsIImapIncomingServer?

or does the imap nsImapOfflineDownloader ever operate on
non-nsIImapIncomingServers? (like local or pop?)


sr=sspitzer

Comment 10

[David :Bienvenu]

I'll remove the skipping of news servers but I do want pop servers to do get new
mail.

Comment 11

[(not reading, please use [e-mail] instead)]

sounds good.  

I didn't realize the imap downloader was used for things other than imap.

sr=sspitzer on the final fix you come up with.

I noticed you have an mBundle change...I hope that wasn't my mBundle /
nsMsgStatusFeedback bug biting you.

Comment 12

[David :Bienvenu]

fix checked in.

Comment 13

[grylchan]

David-
Using Commercial builds:
2001-08-15-09-trunk - nt 4.0
2001-08-15-14-trunk - linux 2.2, mac 9.0.4

I think I am ready to mark this verified. For my 3 test
cases, test 2 always passes. Test 1 and Test 3 there
is slight irregularaties. I am seeing things like mail
filtered correctly but the messages are still grey when I
go offline (and i have to go online and click on them to download 
them). Or say for Test 3: all messages get filtered and downloaded
but the inbox mesgs aren't downloaded. But if I click on them while 
online, they'll get downloaded. Or another instance where for 3 
filtered folders, 2 out of 3 folders download correctly.

I don't think this is a bug for couple of reasons:
  -My test cases are REALLY extreme. The average user, who has
   been offline, is not going to online (and not click on any folder,
   message, etc..) and just immediately go to download and sync and
   then immediately go offline. More like once they go back online,
   they'll click on some other mesgs, write some mail, get mesgs, etc..
 -you comment on your code that it's like a 'race', so I think I am
  either downloading/syncing to fast or timing with message mail server
  is as such that I will get sometimes inconsistent results.

So if you (or anybody else) doesn't object, I'll mark this as verified?

I'm sure Chris will let me know if filters ain't working. :-)

Comment 14

[grylchan]

marking as verified.
Reverified

with commercial branch builds
2001091003 nt 4.0
2001091005 linux 2.2, mac 9.1

extreme test cases w/filters is a new bug: bug 99212