Closed Bug 43297 Opened 25 years ago Closed 22 years ago

Mail imported from Outlook Express repeatedly gets marked as unread

Categories

(MailNews Core :: Backend, defect, P3)

Product:
MailNews Core
Component:
Backend
Platform:
x86
Windows 98
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: peter.oshea, Assigned: Bienvenu)

References

Details

Attachments

(2 files, 1 obsolete file)

mail file - folder name "misc"
19.13 KB, text/plain
Details
proposed fix
6.56 KB, patch
sspitzer
: review+
sspitzer
: superreview+
Details | Diff | Splinter Review
When I first ran Mozilla, at about M15, I imported email folders from Outlook Express using Mozilla's import. All of the messages were initially marked as unread. Using "Message...mark...all as unread" on each folder, they became marked read. Each time I install a new version of Mozilla (post-M15 nightlies, M16, and now most recent nightlies) and move a message into one of these folders, all of the imported OE messages are re-marked as unread. None of subsequent native-Mozilla messages become marked unread, only those originally imported from OE. Deleting the .msf files for each folder didn't change this behavior. Moving these messages to different folders didn't change it either. I can supply an email folder with these messages for testing purposes.
The fact that deleting the .msf files didn't fix this problem means it's not a mail database problem, but a problem with the format of the imported mail. Scott, could you give this to Tony to look at?
Assignee: bienvenu → putterman
Component: Mail Database → Mail Back End
Attached mail file has some messages (older than 4/28/00) imported from OE, and the rest native to Mozilla.
reassigning to tonyr@fbdesigns.com
Assignee: putterman → tonyr
Status: UNCONFIRMED → NEW
Ever confirmed: true
QA Contact: lchiang → esther
What does mozilla use to keep track read/unread status? OE imported messages are nothing special, just RFC822 as they were received by Outlook Express.
cc: putterman to answer Tony's questions.
cc'ing bienvenu
Jeff has been handling these bugs, and I thought he had one just like this.
Still haven't heard from anyone about how mozilla remembers what messages were read? Does it depend on having an X-??-Status header in the message? If so that would likely be the problem.
yes, we need to have an x-mozilla-status and an x-mozilla-status2 header put in the imported messages.
Well, that's bad news (for me at least!). Wish I had known this about 4 months ago :( I'll have to investigate but I imagine this will be a significant amount of work.
Status: NEW → ASSIGNED
Target Milestone: --- → M19
If you're doing any parsing of the outlook express mail folders, this shouldn't be hard because you just need to add the x-mozilla-status lines anywhere in the header part of the message (i.e., after the "From " envelope separator). If they're already in Berkeley format and you're just copying them over to the mozilla profile directory, then yes, there's some work involved. But luckily, mozilla already has lots of code to parse berkeley mailboxes and I can point you at the appropriate code.
I don't currently parse the messages so the work is in adding that. I only get chunks of messages at a time from OE although I do think I know when messages start so it shouldn't be too difficult. Where do I look to find out what exactly I should write out for the 2 headers?
the status flags are defined in mailnews/base/public/nsMsgMessageFlags.h - the flags are 16 bits long, e.g. X-Mozilla-Status 0001 and X-Mozilla-Status2 0000 - Status2 should be 0000, I believe. X-Mozilla-Status should be 0000 if the message is unread, and 0001 if it's read. I don't think you need to set any other flags
QA Contact: esther → nbaca
Peter, is the problem still occuring for you? I know it's been awhile.
i probalby have the same problem here on win XP professional. my OE mail was imported with Mozilla 1.0 (final) ist it possible to fix my mail archive in some way? i find it very annoying to have more than 3000 mails in a few folders marked as unread each time i start mozilla. thanx!
Same thing has been happening to me. I imported my mail from Eudora (W32/v5.0.2) under Mozilla1.0/Win32 and have been reading it in Mozilla 1.0 (linux) I would have thought that selecting all the items and marking them as read would have added/changed the missing Mozilla-Status lines. Should it? All show as read after adding the two lines after the "From - " line, X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000
Hello! I never worked on Mozilla's development, just reported a several bugs in the past. I need to know what criteria is used to separate two messages on the message file because I want do write a script to add "X-Mozilla-Status:" and "X-Mozilla-Status2:" to the header of each imported message. I'm always having this bug and it's not necessary to import new messages few times after closing and starting Mozilla Mail this happens. Each time this happens my previous labelled messages (that I already marked to "none" once again) get labelled once again. So I wonder if it isn't a major client bug. Regards and HAPPY CHRISTMAS!
A new message in the berkeley mailbox format is simply indicated by a line that starts with "From ". Real lines that start with "From " will be escaped as ">From " - you might also need to distinguish between real headers and headers in forwarded messages when checking if an existing message has an x-mozilla-status line. The basic structure is as follows: From xxx Hdr1: Hdr2:... message body ... (could contain x-mozilla-status lines in forwarded messages, though we do make every effort to make sure these don't escape into the wild :-) ) From xxx So, a blank line separates the headers from the message body, and a line starting with "From " separates the messages. Usually, there will be a blank line before the "From " line but that's not required. I believe this import problem has been fixed, so if you use newer versions of Mozilla, the problem should be gone, at least for importing.
Thanks for the format. Newer versions of Mozilla still have the same problem. Probably not when importing but after that I started to use Mozilla (because I can share my bookmarks and e-mails between Windows and Linux, it allowed me to migrate from Windows to Linux without losing anything) so I never imported things once again. What should be implemented on e-mail client is the capability to add "X-Mozilla-Status:" and "X-Mozilla-Status2:" to all messages that don't have it. Just one more question: X-Mozilla-Status is always immediately after "From " that indicates the beginning of a new message? And what can happen if one message has those lines twice? Best regards, Manuel Silva
I am having the exact same problem with many of our guys, and have been ever since I converted us over to Mozilla when 1.0.1 came out. We are all on Windows 2000, so this is *not* a Win98 specific issue - and we were using Outbreak (Outlook 2000), so it is also not just an OE issue. I am dismayed that this bug is so old and still remains unfixed. And before anyone says anything, if I *was* a programmer capable of fixing this, I would be most happy to - alas, I am not, so all I can do is beg someone else to *please* fix this before my boss gets so frustrated that he forces me to switch us all back to Ouutbreak Charles
As the reporter, I probably should've kept pestering people over the years on this bug. But I long ago got tired of waiting for movemail support in Mozilla, and switched to Evolution.
the problem is that the original import bug was fixed long ago, I believe, but we're stuck with the mis-imported messages. Other than going in by hand and putting in X-Mozilla-Status: 0001 lines in each message, there's no fix for this. If we had time to rewrite folder compaction so that it added X-Mozilla-Status lines to messages without them, we could fix this, but I don't think anyone has time to do that.
I have to disagree with the previous post - I don't think it's fixed at all - or at least there's a similar bug still around... I imported my Outlook mail under Win XP Home using Mozilla 1.3b and still some of the mail is marked as unread. After first importing it I clicked each folder in turn and hit 'Control-A' and 'M' to 'select all' and 'mark as read'. Everything was fine until I restarted, and then I found everything was marked as unread again. Maybe it wasn't everything, but most was marked as unread. I have since discovered that if I attempt the job in smaller sections then Mozilla 'remembers' that the mail is read. If I try marking 5 large folders as read one after another (where large means that they contain 200 or so messages) then when I restart they are unread again. If, however, I mark one large folder as read, exit, and restart Mozilla, then the mark as read 'sticks'. I am running out of folders which I haven't 'fixed' in this way, but have two left which I will leave in case you want me to carry out experiments on them. They currently don't have the string 'X-Moz' anywhere in their data files. But then, neither do the ones that I've 'fixed' by marking as read and exiting regularly, apart from one message which has: X-Mozilla-Status: 000b X-Mozilla-Status2: 10000000 but that's from somebody who I think was trying Mozilla at the time.
I'm using Mozilla 1.2.1 under LINUX, the last stable release (I suppose) and each year I copy my messages from the previous year to a sub-folder and I restart completely the Inbox folder (deleting Inbox and Inbox.msf files from Mail/my.mail.server directory). I thought I wouldn't have problems anymore but I was mistaken. Quite oftenly Mozilla hangs during the automatic mail-check (not a Mozilla crash but just can't get the mail for that account until restart Mozilla) and after that some folders are rebuilt ("Rebuilding summary for Inbox..."). When this happens, all definitions from the folder are lost, such mail sorting criteria. When this happens to a folder which has messages imported from any other program (in my case from Outlook Expre$$) the imported ones become unread once again. Probably the .msf files are getting corrupted in some way... Regards, Manuel
-> Cavin. Cavin has worked on import bugs in the past, and he'd know if we're putting in the X-Mozilla-Status lines or not, and if not, where to fix it.
Assignee: tonyr → cavin
Status: ASSIGNED → NEW
*** Bug 197967 has been marked as a duplicate of this bug. ***
*** Bug 187498 has been marked as a duplicate of this bug. ***
This patch scans message headers during folder compaction and adds the X-Mozilla-Status and X-Mozilla-Status2 lines if they're missing. Unfortunately it has one problem that needs to be addressed: I scan the entire set of header lines while the headers are being copied, and if the lines aren't there then I append the lines just before the message body gets written (i.e. as the last two lines of the header). Normally the X-Mozilla-Status lines are near the start of the header, and there are various asserts that will be triggered if they don't appear in the first 10K of the message (I don't see why the asserts are there; they seem to just be sanity checks). If a message has a header >10K in size then the approach as implemented fails. Since normally the status lines are right after the "From " line it would probably be acceptable to scan only the first part of a very long header for them. Alternately we could buffer up the entire header in memory and then perform the scan and insert them after the "From " line. Suggestions?
cool. The assertion could be removed, if we're really only hitting it because some messages have more than 10K of headers. re X_MOZILLA_STATUS, it's defined in nsMsgLocalFolderHdrs.h, without the ':'., but you should be able to use it. I don't see any code that sets the status offset in the mail hdr, something like msgHdr->SetStatusOffset(filePos). That's how we know where the x-mozilla-status lines are in the mail message. Are you sure that's getting set?
I can verify this problem with Windows XP in Mozilla 1.3. It is a VERY annoying bug, and it's the one thing that's preventing me from migrating to Mozilla mail as opposed to outlook. Does anyone know if the Linux version of Mozilla has this bug as well?
Your word wrap is borken. (Bugzilla bug?) IIRC you cannot import OE mails with the GNU/Linux version of Mozilla, so the bug cannot occur at all. CMIIW, I have been a few weeks away from GNU/Linux. \V/ Live long and prosper PointedEars
I'm going to look at this patch - I suspect there might be a simpler patch using the same approach. We should, however, fix the outlook import code as well if it's still broken, which I've heard it is.
Attached patch proposed fix — — Splinter Review
this is similar to the first patch, except that instead of looking for the status line headers, it uses the fact that the src db hdr knows if there is a status header or not. This makes the patch a lot simpler and makes the normal code path much more like it was before. It also sets the status offset for the new hdr, which the previous patch did not.
Attachment #120417 - Attachment is obsolete: true
taking
Assignee: cavin → bienvenu
Comment on attachment 124917 [details] [diff] [review] proposed fix r/sr=sspitzer we should rel note (for 1.5 and beyond) that the fix is to compact folders. david, can you log the spin off bug about fixing the import code?
Attachment #124917 - Flags: superreview+
Attachment #124917 - Flags: review+
fix checked in.
Status: NEW → RESOLVED
Closed: 22 years ago
Resolution: --- → FIXED
This checkin have added an "may be used uninitialized" warning to brad TBox: +mailnews/base/src/nsMsgFolderCompactor.cpp:557 + `PRUint32 writeCount' might be used uninitialized in this function Indeed, AFAICT, in the NS_ASSERTION(PR_FALSE, "not an envelope") branch the writeCount is never initialized, but is still used later on...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Please don't re-open bugs for warnings. writeCount won't be set in a very particular situation that you should never encounter. However, it's true it should be handled.
Status: REOPENED → RESOLVED
Closed: 22 years ago → 22 years ago
Resolution: --- → FIXED
http://bugzilla.mozilla.org/show_bug.cgi?id=209335 filed for new warning.
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: