Closed Bug 43297 Opened 24 years ago Closed 21 years ago

Mail imported from Outlook Express repeatedly gets marked as unread

Categories

(MailNews Core :: Backend, defect, P3)

x86
Windows 98
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: peter.oshea, Assigned: Bienvenu)

References

Details

Attachments

(2 files, 1 obsolete file)

When I first ran Mozilla, at about M15, I imported email folders from Outlook
Express using Mozilla's import.  All of the messages were initially marked
as unread.  Using "Message...mark...all as unread" on each folder, they became
marked read.

Each time I install a new version of Mozilla (post-M15 nightlies, M16, and now
most recent nightlies) and move a message into one of these folders, all of the
imported OE messages are re-marked as unread.  None of subsequent native-Mozilla
messages become marked unread, only those originally imported from OE.

Deleting the .msf files for each folder didn't change this behavior.

Moving these messages to different folders didn't change it either.

I can supply an email folder with these messages for testing purposes.
The fact that deleting the .msf files didn't fix this problem means it's not a 
mail database problem, but a problem with the format of the imported mail. 
Scott, could you give this to Tony to look at?
Assignee: bienvenu → putterman
Component: Mail Database → Mail Back End
Attached mail file has some messages (older than 4/28/00) imported from OE, and 
the rest native to Mozilla.
reassigning to tonyr@fbdesigns.com
Assignee: putterman → tonyr
Status: UNCONFIRMED → NEW
Ever confirmed: true
QA Contact: lchiang → esther
What does mozilla use to keep track read/unread status?  OE imported messages are 
nothing special, just RFC822 as they were received by Outlook Express.
cc: putterman to answer Tony's questions.
cc'ing bienvenu
Jeff has been handling these bugs, and I thought he had one just like this.
Still haven't heard from anyone about how mozilla remembers what messages were 
read?  Does it depend on having an X-??-Status header in the message?  If so that 
would likely be the problem.
yes, we need to have an x-mozilla-status and an x-mozilla-status2 header put in
the imported messages.
Well, that's bad news (for me at least!).  Wish I had known this about 4 months 
ago :(  I'll have to investigate but I imagine this will be a significant amount 
of work.
Status: NEW → ASSIGNED
Target Milestone: --- → M19
If you're doing any parsing of the outlook express mail folders, this shouldn't
be hard because you just need to add the x-mozilla-status lines anywhere in the
header part of the message (i.e., after the "From " envelope separator). If
they're already in Berkeley format and you're just copying them over to the
mozilla profile directory, then yes, there's some work involved. But luckily,
mozilla already has lots of code to parse berkeley mailboxes and I can point you
at the appropriate code.
I don't currently parse the messages so the work is in adding that.  I only get 
chunks of messages at a time from OE although I do think I know when messages 
start so it shouldn't be too difficult.  Where do I look to find out what exactly 
I should write out for the 2 headers?
the status flags are defined in mailnews/base/public/nsMsgMessageFlags.h - the
flags are 16 bits long, e.g. X-Mozilla-Status 0001 and X-Mozilla-Status2 0000 -
Status2 should be 0000, I believe. X-Mozilla-Status should be 0000 if the
message is unread, and 0001 if it's read. I don't think you need to set any
other flags
QA Contact: esther → nbaca
Peter, is the problem still occuring for you? I know it's been awhile.
i probalby have the same problem here on win XP professional.

my OE mail was imported with Mozilla 1.0 (final)
ist it possible to fix my mail archive in some way?
i find it very annoying to have more than 3000 mails in a few folders marked 
as unread each time i start mozilla.

thanx!
Same thing has been happening to me. 
I imported my mail from Eudora (W32/v5.0.2) under Mozilla1.0/Win32 and have been
reading it in Mozilla 1.0 (linux)

I would have thought that selecting all the items and marking them as read would
have added/changed the missing Mozilla-Status lines. Should it?

All show as read after adding the two lines after the "From - " line,
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Hello!
I never worked on Mozilla's development, just reported a several bugs in the past.
I need to know what criteria is used to separate two messages on the message
file because I want do write a script to add "X-Mozilla-Status:" and
"X-Mozilla-Status2:" to the header of each imported message. I'm always having
this bug and it's not necessary to import new messages few times after closing
and starting Mozilla Mail this happens. Each time this happens my previous
labelled messages (that I already marked to "none" once again) get labelled once
again. So I wonder if it isn't a major client bug.

Regards and HAPPY CHRISTMAS!
A new message in the berkeley mailbox format is simply indicated by a line that
starts with "From ". Real lines that start with "From " will be escaped as
">From " - you might also need to distinguish between real headers and headers
in forwarded messages when checking if an existing message has an
x-mozilla-status line. The basic structure is as follows:

From xxx
Hdr1:
Hdr2:...

message body
... (could contain x-mozilla-status lines in forwarded messages, though we do
make every effort to make sure these don't escape into the wild :-) )

From xxx

So, a blank line separates the headers from the message body, and a line
starting with "From " separates the messages. Usually, there will be a blank
line before the "From " line but that's not required.

I believe this import problem has been fixed, so if you use newer versions of
Mozilla, the problem should be gone, at least for importing.
Thanks for the format.
Newer versions of Mozilla still have the same problem. Probably not when
importing but after that I started to use Mozilla (because I can share my
bookmarks and e-mails between Windows and Linux, it allowed me to migrate from
Windows to Linux without losing anything) so I never imported things once again.
What should be implemented on e-mail client is the capability to add
"X-Mozilla-Status:" and
"X-Mozilla-Status2:" to all messages that don't have it.
Just one more question: X-Mozilla-Status is always immediately after "From "
that indicates the beginning of a new message? And what can happen if one
message has those lines twice?

Best regards,

Manuel Silva
I am having the exact same problem with many of our guys, and have been ever
since I converted us over to Mozilla when 1.0.1 came out.  We are all on Windows
2000, so this is *not* a Win98 specific issue - and we were using Outbreak
(Outlook 2000), so it is also not just an OE issue.

I am dismayed that this bug is so old and still remains unfixed.  And before
anyone says anything, if I *was* a programmer capable of fixing this, I would be
most happy to - alas, I am not, so all I can do is beg someone else to *please*
fix this before my boss gets so frustrated that he forces me to switch us all
back to Ouutbreak

Charles
As the reporter, I probably should've kept pestering people over the years on
this bug.  But I long ago got tired of waiting for movemail support in Mozilla,
and switched to Evolution.  
the problem is that the original import bug was fixed long ago, I believe, but
we're stuck with the mis-imported messages.  Other than going in by hand and
putting in X-Mozilla-Status: 0001 lines in each message, there's no fix for
this. If we had time to rewrite folder compaction so that it added
X-Mozilla-Status lines to messages without them, we could fix this, but I don't
think anyone has time to do that.
I have to disagree with the previous post - I don't think it's fixed at all - or
at least there's a similar bug still around...

I imported my Outlook mail under Win XP Home using Mozilla 1.3b and still some
of the mail is marked as unread.

After first importing it I clicked each folder in turn and hit 'Control-A' and
'M' to 'select all' and 'mark as read'.  Everything was fine until I restarted,
and then I found everything was marked as unread again.  Maybe it wasn't
everything, but most was marked as unread.  I have since discovered that if I
attempt the job in smaller sections then Mozilla 'remembers' that the mail is
read.  If I try marking 5 large folders as read one after another (where large
means that they contain 200 or so messages) then when I restart they are unread
again.  If, however, I mark one large folder as read, exit, and restart Mozilla,
then the mark as read 'sticks'.  I am running out of folders which I haven't
'fixed' in this way, but have two left which I will leave in case you want me to
carry out experiments on them.  They currently don't have the string 'X-Moz'
anywhere in their data files.  But then, neither do the ones that I've 'fixed'
by marking as read and exiting regularly, apart from one message which has:

X-Mozilla-Status: 000b
X-Mozilla-Status2: 10000000

but that's from somebody who I think was trying Mozilla at the time.
I'm using Mozilla 1.2.1 under LINUX, the last stable release (I suppose) and
each year I copy my messages from the previous year to a sub-folder and I
restart completely the Inbox folder (deleting Inbox and Inbox.msf files from
Mail/my.mail.server directory). I thought I wouldn't have problems anymore but I
was mistaken. Quite oftenly Mozilla hangs during the automatic mail-check (not a
Mozilla crash but just can't get the mail for that account until restart
Mozilla) and after that some folders are rebuilt ("Rebuilding summary for
Inbox..."). When this happens, all definitions from the folder are lost, such
mail sorting criteria. When this happens to a folder which has messages imported
from any other program (in my case from Outlook Expre$$) the imported ones
become unread once again.
Probably the .msf files are getting corrupted in some way...

Regards,

Manuel
-> Cavin. Cavin has worked on import bugs in the past, and he'd know if we're
putting in the X-Mozilla-Status lines or not, and if not, where to fix it.
Assignee: tonyr → cavin
Status: ASSIGNED → NEW
*** Bug 197967 has been marked as a duplicate of this bug. ***
*** Bug 187498 has been marked as a duplicate of this bug. ***
This patch scans message headers during folder compaction and adds the
X-Mozilla-Status and X-Mozilla-Status2 lines if they're missing. Unfortunately
it has one problem that needs to be addressed: I scan the entire set of header
lines while the headers are being copied, and if the lines aren't there then I
append the lines just before the message body gets written (i.e. as the last
two lines of the header). Normally the X-Mozilla-Status lines are near the
start of the header, and there are various asserts that will be triggered if
they don't appear in the first 10K of the message (I don't see why the asserts
are there; they seem to just be sanity checks). If a message has a header >10K
in size then the approach as implemented fails.

Since normally the status lines are right after the "From " line it would
probably be acceptable to scan only the first part of a very long header for
them. Alternately we could buffer up the entire header in memory and then
perform the scan and insert them after the "From " line. Suggestions?
cool. The assertion could be removed, if we're really only hitting it because
some messages have more than 10K of headers.

re X_MOZILLA_STATUS, it's defined in nsMsgLocalFolderHdrs.h, without the ':'.,
but you should be able to use it.

I don't see any code that sets the status offset in the mail hdr, something like
msgHdr->SetStatusOffset(filePos). That's how we know where the x-mozilla-status
lines are in the mail message. Are you sure that's getting set?
I can verify this problem with Windows XP in Mozilla 1.3. It is a VERY annoying bug, and it's the one thing that's preventing me from migrating to Mozilla mail as opposed to outlook.

Does anyone know if the Linux version of Mozilla has this bug as well?


Your word wrap is borken. (Bugzilla bug?)

IIRC you cannot import OE mails with the GNU/Linux version of Mozilla, so the
bug cannot occur at all. CMIIW, I have been a few weeks away from GNU/Linux.


\V/ Live long and prosper

PointedEars
I'm going to look at this patch - I suspect there might be a simpler patch using
the same approach. We should, however, fix the outlook import code as well if
it's still broken, which I've heard it is.
Attached patch proposed fixSplinter Review
this is similar to the first patch, except that instead of looking for the
status line headers, it uses the fact that the src db hdr knows if there is a
status header or not. This makes the patch a lot simpler and makes the normal
code path much more like it was before. It also sets the status offset for the
new hdr, which the previous patch did not.
Attachment #120417 - Attachment is obsolete: true
taking
Assignee: cavin → bienvenu
Comment on attachment 124917 [details] [diff] [review]
proposed fix

r/sr=sspitzer

we should rel note (for 1.5 and beyond) that the fix is to compact folders.

david, can you log the spin off bug about fixing the import code?
Attachment #124917 - Flags: superreview+
Attachment #124917 - Flags: review+
fix checked in.
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
This checkin have added an "may be used uninitialized" warning to brad TBox:

+mailnews/base/src/nsMsgFolderCompactor.cpp:557
+ `PRUint32 writeCount' might be used uninitialized in this function

Indeed, AFAICT, in the NS_ASSERTION(PR_FALSE, "not an envelope") branch the
writeCount is never initialized, but is still used later on...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Please don't re-open bugs for warnings. writeCount won't be set in a very
particular situation that you should never encounter. However, it's true it
should be handled.
Status: REOPENED → RESOLVED
Closed: 21 years ago21 years ago
Resolution: --- → FIXED
http://bugzilla.mozilla.org/show_bug.cgi?id=209335 filed for new warning.
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: