248742 - Loss of "fetch headers only" message (junk, offline, dragged)

Status: RESOLVED DUPLICATE of bug 287728

(MailNews Core :: Networking: POP, defect)

Description

[Mike Cowperthwaite]

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8a2) Gecko/20040623

Trying out the new headers-only mode (bug 185184).  Several spams came from the 
account, and had to be manually flagged as junk.  Later, a ham message arrived 
and was automarked as junk and moved to the Junk folder.  I went offline, then 
discovered this message was in the Junk folder; I un-junked it and dragged it 
back to the original Inbox.

Later, I went back online, and revisited the Inbox in order to download the 
content.  The message was not visible in the thread pane (no mailview or 
quicksearch active).

Looking in the Inbox file, I found two copies of the headers for the message.
The first, which presumably was the leftover fragment from before being moved to 
Junk, had these headers:
  X-Mozilla-Status: 0418
  X-Mozilla-Status2: 00010000
The second, presumably the version that was dragged back from the Junk folder, 
had these:
  X-Mozilla-Status: 0419
  X-Mozilla-Status2: 00000000

I'm at a loss to figure out how to get to the message now.

I made a simple attempt to reproduce by sending mail from my primary account to 
this headers-only account.  The H-O account has a filter to move messages from 
people in the address book back to my primary account's inbox; after the test 
message was moved, I dragged it to the original Inbox, and checked: the header 
was visible, and clicking the link got me the message.  Looking at the Inbox 
file again, the original copy of the message (before filtering) has the headers:
  X-Mozilla-Status: 0409
  X-Mozilla-Status2: 00000000
The downloaded copy has:
  X-Mozilla-Status: 0001
  X-Mozilla-Status2: 00000000


More information to come if I see the symptom again.

Comment 1

[Howard Chu]

For completeness' sake, what were your Leave On Server preferences?

Can you get to your POP mailbox using some other mechanism (e.g. telnet) and see
if the missing message is still there?

Comment 2

[Mike Cowperthwaite]

(In reply to comment #1)
> For completeness' sake, what were your Leave On Server preferences?

"Leave on server" is unchecked.

> Can you get to your POP mailbox using some other mechanism (e.g. telnet) and
> see if the missing message is still there?

No, this ISP does not allow that access.

Comment 3

[Mike Cowperthwaite]

(In reply to comment #1)
> Can you get to your POP mailbox using some other mechanism (e.g. telnet) and
> see if the missing message is still there?

OK, I simply opened a new profile and replicated the account, without 
header-only load -- the message had been removed from the server.

Comment 4

[Howard Chu]

(In reply to comment #2)

> (In reply to comment #1)
> > For completeness' sake, what were your Leave On Server preferences?
> 
> "Leave on server" is unchecked.

I think this may take some more thought to unravel. When a message is Moved
manually or by the Spam detector, it uses the Copy service which Copies it to
the destination and Deletes from the source. In the Delete code, if the message
being deleted is Partial and Leave on Server is unchecked, then it is also
marked for Delete from the POP server.

>I made a simple attempt to reproduce by sending mail from my primary account to 
>this headers-only account.  The H-O account has a filter to move messages from 
>people in the address book back to my primary account's inbox; after the test 
>message was moved, I dragged it to the original Inbox, and checked: the header 
>was visible, and clicking the link got me the message. 

When a message is Moved by a filter, it does not use the Copy service, so it
doesn't get deleted the way it did above.

> > Can you get to your POP mailbox using some other mechanism (e.g. telnet) and
> > see if the missing message is still there?
> 
> No, this ISP does not allow that access.

You can always telnet to the POP port and talk POP directly...
  telnet myserver pop3
  user myusername
  pass mypass
  list
  top <msgnumber> 0
  retr <msgnumber>
  quit

Comment 5

[Mike Cowperthwaite]

(In reply to comment #4)
> When a message is Moved manually or by the Spam detector, it uses the Copy
> service which Copies it to the destination and Deletes from the source.
> In the Delete code, if the message being deleted is Partial and Leave on
> Server is unchecked, then it is also marked for Delete from the POP server.

I've been playing around a bit with this to try and narrow down the symptoms for 
this and bug 249361, but I'm getting erratic results and I don't have time right 
now to embark on an exhaustive test.

xref 89034

Comment 6

[WADA:World Anti-bad-Duping Agency]

Loss of partial downloaded message occurs if "Leave messages on server" is
uncheked but "Until I delete or move them from Inbox" is checked.

(Case-1) Setup a POP3 account for header download with "Leave messages on server"
         uncheked but "Until I delete or move them from Inbox" checked.
   1-1. Check "Fetch header only"
   1-2. Check "Leave messages on server"
   1-3.   Check "Until I delete or move them from Inbox" ( <== *Checked* )
   1-4.   Uncheck "For atmost NN days" (To make test case simple)
   1-5. Uncheck "Leave messages on server" again
   1-6. Uncheck "Message larger than NN kb"
        of "To save disk, do not download:" of "Disk Space" is unchecked.
        (To make test case simple)
   1-7. Get messages -> popstate.dat indicates "b"  
   1-8. Move header only mail to other than Inbox
        -> popstate.dat indicates "d" (<= This is the problem)
   1-9. Server connection by Get Messages -> Mail is removed from server
   1-10. Additional download -> loss of mail

(Case-2) Uncheck "Until I delete or move them from Inbox" 
   2-1. Check "Leave messages on server" in order to uncheck options under it.
   2-2.   Uncheck "Until I delete or move them from Inbox" option.
   2-3. Uncheck "Leave messages on server" option again.

   - "Fetch header only" is checked
   - "Leave messages on server" is unchecked
     - "Until I delete or move them from Inbox" is *unchecked* now

   2-4. Get messages -> popstate.dat indicates "b"  
   2-5. Move message to other folder -> popstate.dat indicates "b" (unchanged)
   2-6. Move the mail back to Inbox -> popstate.dat indicates "b" (unchanged)
   2-7. Copy this header-ony mail to other folder
   2-8. Server connection -> No action
   2-9. Additional download(Inbox) -> downloaded
        -> popstate.dat indicates "k".
   2-10. Server connection -> No action
   2-11. Additional download for second copy -> downloaded
         -> popstate.dat indicates "k" (unchaged).

(Problem-A)
When "Leave messages on server" is unchecked, Mail BackEnd should ingnore
options under it, if current Account Manager's behaiviour is correct.
If current Mail BackEnd's behaviour is correct, Account Manager should reset
options under "Leave messages on server" when this option is unchecked.

(Problem-B)
Mail status in popstate.dat should be "d" instead of "k" after successuful
additional download at step (2-9) when "Leave messages on server" is unchecked. 

Comment 7

[Mike Cowperthwaite]

(In reply to comment #6)
> When "Leave messages on server" is unchecked, Mail BackEnd should ingnore
> options under it, if current Account Manager's behaiviour is correct.
> If current Mail BackEnd's behaviour is correct, Account Manager should reset
> options under "Leave messages on server" when this option is unchecked.

Apparently, the two checkboxes actually should be independent: bug 207571.


(In reply to comment #4)
> (In reply to comment #2)
> 
> > (In reply to comment #1)
> > > For completeness' sake, what were your Leave On Server preferences?
> > 
> > "Leave on server" is unchecked.
> 
> I think this may take some more thought to unravel. When a message is Moved
> manually or by the Spam detector, it uses the Copy service which Copies it to
> the destination and Deletes from the source. In the Delete code, if the 
> message being deleted is Partial and Leave on Server is unchecked, then it is
> also marked for Delete from the POP server.

Howard, on rereading, that explanation sounds as if you read my comment 1 as 
"Leave on server is checked" -- but I said UNchecked: I'm set up to NOT leave 
messages on server, to delete them on download.  Also, to follow up on the 
earlier part of this post, I also have "Until I delete or move..." UNchecked.

Comment 8

[Tony King (aqk)]

*** Bug 324520 has been marked as a duplicate of this bug. ***

Comment 9

[Mike Cowperthwaite]

I've been running into similar bugs today: bug 287728 and bug 340010.

Comment 10

[Joshua Wiman]

Just got hit by this exact bug and lost two important messages. Complete loss of email should REALLY be on a higher priority fix!

Comment 11

[David :Bienvenu]

In 2.0, we only delete messages from the server if they've been deleted locally, or moved to the trash, not if they're moved elsewhere. So this should not be an issue in 2.0

Comment 12

[Kathleen]

well, hurry up with 2.0, eh?

Comment 13

[Mike Cowperthwaite]

2.0 is out.  Anyone still encountering this problem while using it?
I'm very unlikely to encounter the symptom any longer; the only account where I use Download Headers receives a spam message maybe twice a year (aggressive filtering on the ISP's part, I think).

Comment 14

[(not reading, please use [e-mail] instead)]

sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.

Comment 15

[Olaf Schreiner]

(In reply to comment #13)
> 2.0 is out.  Anyone still encountering this problem while using it?

Yes. See my comment in duplicate bug 287728.

Comment 16

[Wayne Mery (:wsmwk)]

Mike, Joshua, dupe to bug 287728?  Or one depend on the other?
OS=ALL?

Comment 17

[:aceman]

OK, so after the "move out of inbox is no longer considered deletion" part was reportedly solved in 2.0, is the the remaining problem here the one reported in bug 287728? can we dupe it?

Comment 18

[Mike Cowperthwaite]

I'm no longer using POP.