wrong messages deleted with "leave on server for at most X days"

RESOLVED WORKSFORME

Status

MailNews Core
Networking: POP
--
critical
RESOLVED WORKSFORME
15 years ago
9 years ago

People

(Reporter: Leston Buell, Assigned: Bienvenu)

Tracking

({dataloss})

Trunk
x86
Linux
dataloss

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

15 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6a) Gecko/20031029
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6a) Gecko/20031029

I wanted to try the new 1.6a "leave on server for at most X days" feature. I set
the feature for "at most 10 days". I checked my mail with Mozilla. When i
rechecked my account with Pine, it had deleted all of the messages except the
ones i had received within the last few *minutes*. This makes me think that it
deleted those "at most 10 *minutes*" old.

This feature is covered in bug 107883.

Marking as "critical" because of data loss.

Reproducible: Didn't try

Steps to Reproduce:

Updated

15 years ago
Keywords: dataloss
(Assignee)

Comment 1

15 years ago
can you attach or e-mail me (bienvenu@nventure.com) your prefs.js?  I've been
using this for months and haven't had a problem. The preferences UI was reworked
a bit, but I doubt that's involved. The setting is in days, and is implemented
in days in the backend.

I might also need a pop3 protocol log which can be generated by following these
instructions for POP3.

thx.

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap
Assignee: sspitzer → bienvenu
Status: UNCONFIRMED → NEW
Ever confirmed: true
(Reporter)

Comment 2

15 years ago
Created attachment 134780 [details]
pop3 log

Here's my pop3 log.
(Reporter)

Comment 3

15 years ago
Created attachment 134782 [details]
prefs.js

And here's my prefs.js.
(Reporter)

Comment 4

15 years ago
The bug seemed to happen only the very first time i used the feature. No new
messages are being deleted now.

Comment 5

15 years ago
I checked "leave on server for at most 7 days" in 1.6b, and Mozilla doesn't seem
to delete any messages at all. I checked my mail from another machine and saw a
load of mails, some of which were three weeks old.
(Assignee)

Comment 6

15 years ago
Leston, are old messages getting deleted now?

Stefan, this feature uses the time stamp of when you first downloaded the
message with the version of mozilla that implements this feature. So it has to
be 3 weeks from when you downloaded the message with 1.6b, not 3 weeks from when
the message was sent, or 3 weeks from from when you downloaded it with 1.5. The
first time you run 1.6b starts the clock running on previously downloaded
messages as well.
(Reporter)

Comment 7

15 years ago
I only got this problem the very first time that i used the feature. It seems
that the only way to test this, then, would be to set up a new e-mail account on
the same server and try it. Unfortunately, i don't have the ability to do that.
Although what i experienced is distressing, no one else has added "me too"
comments to this bug or voted for it, so it's extremely hard to evaluate this bug.
(Assignee)

Comment 8

15 years ago
I haven't had any dups of this bug or reports of similar problems.
(Assignee)

Comment 9

14 years ago
marking wfm, since this can't be reproduced. This seems to be isolated.
Status: NEW → RESOLVED
Last Resolved: 14 years ago
Resolution: --- → WORKSFORME

Comment 10

14 years ago
As I already said in bug 107883, the timestamps of messages downloaded before a
version of Moz with this feature are negative. This may cause problems as
described here. Such messages may appear years old to Moz and it will happily
delete them. But I don't think it is this straightforward. Somebody would notice
it sooner. I can only report what happened to me:

I installed 1.6 and enabled this feature. I downloaded some mails. I
immediatelly noticed, that the old messages have negative timestamps. But all
worked ok.
I think I have downloaded messages several days since then and all was ok. Then
I deleted about 700 old messages. As I work offline all these changes have a lag
and are processed in the next online time. I have seen the spot where Moz
deleted those messages from the server (took quite a while). Unfortunatelly I
don't have a log of this because I mistakenly disabled it :( Then, next day I
wanted to download new mails and it didn't work. Moz tried to download all 1700
mails I had on that server but failed due to bug 230294 (fortunatelly:)). I
looked around what's wrong and found out that my popstate.dat is empty! So, my
messages are fortunatelly not deleted from the server by I have no reference to
them (in popstate.dat). I have restored the file from a recent backup and will
try to reproduce this problem.

Of course, I am not sure this is caused by this bug. But all other accounts work
ok, I enabled this feature in them too, but they didn't have any old messages.
I have also tried to use the same profile from 2 PCs over my new home network,
so that my be a cause too. But no filesystem corruption was reported and no
other files are corrupted. As I said, I am working on reproducing it. My OSes
are Win98 and Win95. The timestamps and mail download are done on the win98.

Comment 11

14 years ago
Of course, the 2 PCs DIDN'T access the profile at the same time.

Comment 12

14 years ago
Ok, the clearing of popstate.dat file is bug 237131, seems like and independent
issue.

Comment 13

14 years ago
After my server recovered from bug 237131, I found out there are less messages
than there should be. I looked at their dates and the olders one is from
26.11.2003. I filed bug 237131 on 12.3.2004, some days after I installed 1.6. If
I count it correctly the difference is about 3 months. When I installed 1.6, I
set keep messages on server for 99 days. This would be exactly that period.
While that would actually be correct behaviour that Moz deleted messages older
than that. The problem is how could it get the correct dates of the messages?

1. David says, at install, 1.6 sets the timestamp of all messages already found
in popstate.dat to the current time. Then it starts to count the specified
interval for aging the messages (here 99 days). It would delete no messages for
the next 99 months.

2. I agree with this implementation, it is easy and does make sense. But as I
reported, there is some bug which prevents the timestamps from beeing correct.
All old messages get a negative time. Thus, on next connect Moz should delete
ALL those messages preserving NONE on the server - because ALL are older than
the specified time (positive(now()) number - negative number=very large positive
number). But even this is not the case! For me, not all messages were deleted.
Only the original reporter says they were.

So how could Moz guess which messages to delete so that it got it right???

David, I really suspect there is some integer overflow involved. At least on
Win98, if you can't personally reproduce this. The negative initial timestamps
are a fact. I can reproduce them any time. And maybe that incidentally
positive-negative=small positive, so that it makes sense to Moz and it doesn't 
crash completelly or breaks something more serious.
(Assignee)

Comment 14

14 years ago
No, I can't reproduce any negative numbers in popstate.dat - I doubt it's
integer overflow since it's 32 bit math, but it might be some problem in the
win98 time/date functions. That seems unlikely, though.

Can you paste in a few actual lines from your popstate.dat that show this
problem? thx!

Comment 15

14 years ago
Sure:

# POP3 State File
# This is a generated file!  Do not edit.

*pop3.atlas.sk username
k 00000780 -1637279712
k 0000069f -1637279712
k 000006b2 -1637279712
k 00000821 -1637279712
k 00000776 -1637279712

File migrated from 1.5 without timestamps. Timestamps created with 1.6 on
23.3.2004 18:19 CET.

I understand your doubts. You use Now() for getting the time. And all subsequent
timestamps when downloading NEW messages are correct. I think you use the same
Now() to get their time and stuff it into the same variables. But something must
be different here and in the initial run (when Moz makes up time for OLD messages).

Are those 32bit variables unsigned? The output to the file seems to use signed
formats...

Comment 16

14 years ago
I now installed 1.7.0 on top of a 1.5 profile. There were no timestamps in the
old popstate.dat file. I checked 'leave on server for at most x days' and
restarted Mozilla a few times (to install langpack). No timestamps appeared.
After connecting to the server, the logging in phase was quite long (bug
256978), so I got suspicious and pressed stop (I am not sure if stop actually
sends a QUIT to the server). Then I tried it again (I can't remember if I
restarted Moz) and got hit by bug 226669, so I couldn't download anything. The
result of all this was a clear popstate.dat file. Yes, both this and 226669
could have wiped it, so this is not a clean confirmation. But Moz shouldn't have
started deleting the messages in the beginning. Again, it could have those
negative timestamps in memory and act upon them. But I didn't witness them on
disk this time, because I lost the file. Sorry.
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.