231735 - Messages marked as unread on search after upgrade to 1.6

Status: RESOLVED WORKSFORME

(MailNews Core :: Database, defect)

Description

[Peter Kirk]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113

I just upgraded from 1.5 to 1.6. I have many thousands of old messages stored in
many folders under Local Folders. When I tried to search for messages in Local
Folders for the first time after the upgrade, I got a message that Mozilla was
building some kind of summary for each folder. When the search was complete I
found that thousands of my old messages had been marked as unread. Most of the
messages so marked were very old ones which had been imported first from cc:Mail
to Outlook and then moved from Outlook to Mozilla. But a small proportion of
more recent messages were also marked as unread.

Afterwards I marked the messages as read and tried another search. No summaries
built, no messages marked unread.

Reproducible: Couldn't Reproduce

Steps to Reproduce:
1. Start with lots of old messages in Local Folders in Mozilla 1.5.
2. Upgrade to 1.6.
3. Search for something in Local Folders (my search was actually for all
messages dated today).

Actual Results:  
Some existing messages were marked unread. The search worked OK.

Expected Results:  
No existing messages should be marked unread.

Comment 1

[David :Bienvenu]

yeah, messages imported from Outlook before 1.6 were not correctly imported such
that they could survive having the summary file regenerated. The Outlook import
didn't add an x-mozilla-status line. If you delete a message in the folder, and
then use the folder property menu to "Compact this folder", it will add back the
x-mozilla-status line and this problem won't happen again. The bug in Outlook
import has been fixed, but too late for you :-( We hoped that adding back the
x-mozilla-status line on folder compact would catch this for most people since
they should be compacting their folders occasionally, or have it automatically
done by setting the "Automatically compact folders when it will save XX KB"
pref. I don't know why your summary files got regenerated but we can't prevent
summary files from getting regenerated when they get out of sync with the mail
folder.

Comment 2

[Peter Kirk]

Comment 1 doesn't fully describe the problem. I had compacted folders many times
between importing from Outlook and (six months later) upgrading to 1.6. Perhaps
these folders didn't actually get compacted, because they are archives from
which I don't delete messages. There needs to be a way to force these
x-mozilla-status lines to be added to all files even when they don't need to be
compacted.

Comment 3

[David :Bienvenu]

right, if you don't delete at least one message, compact won't do anything. I
suppose we could try to notice that a folder contains messages w/o
x-mozilla-status lines and set a flag that says we should compact them. But we
can  only do that if you read messages in the folders, and it sounds like you
don't do that very often.

Comment 4

[Peter Kirk]

Well, building the summary file seems to read the messages enough to note the
absence of x-mozilla-status, so that could flag the need to compact. Better
still the folder could be automatically compacted if a message without
x-mozilla-status is encountered.

But the real bug seems to be something different. From what you say, when a
folder is compacted the status lines added to messages without them default to
assuming that the message has been read. And this is the sensible default if
messages are imported; at least presumably most people will read incoming
messages in one mail client before importing them to another one. So why doesn't
the process of building a summary file use the same sensible default and assume
that messages without status lines have been read? This seems to be what
happened in 1.5. Why the change in 1.6? This is what this bug is really about.
And that is something which has not been resolved.

Comment 5

[David :Bienvenu]

I don't think 1.5 assumed the message is read (in fact, I'm pretty sure it
didn't). I think you just didn't get your summary files regenerated with 1.5. 

Yes, when we regenerate the summary file, we will know that x-mozilla-status
lines are missing and we can note that the folder should be compacted. That's
fine, but summary file regeneration should be very rare (though you can just
delete the summary files and that would be a workaround for this problem, if we
did note that the folder should be compacted). 

When we add x-mozilla-status lines at compact time, we don't assume the message
is read or unread - we use the .msf file to see if the message is marked read or
not.  But when regenerating the .msf file, we can't do that, since we've already
deleted the msf file in preparation for regenerating the msf file.

We could assume that messages with missing x-mozilla-status lines are read. That
would probably annoy a different set of users who would complain that all their
unread messages were marked read when the summary file was regenerated. But,
that's probably a smaller set of people/messages.

Comment 6

[Peter Kirk]

OK, I understand now. I still think it would be better to assume read than to
assume unread, if one thing or the other has to be assumed. I suppose that would
make this an enhancement request (but an open one) rather than a (resolved) bug.

Perhaps the real bug is that all my MSF files were regenerated. Is that normal
on upgrade to 1.6? If it is, there should be a mechanism to preserve information
like the read status before deleting the old MSF files, which were presumably
valid in 1.5. Perhaps all folders should be compacted (whether or not they need
it) as part of the upgrade process.