561562 - Import messages from Windows Mail and Live Mail

Status: NEW

(MailNews Core :: Import, defect)

Description

[Phil Lacy]

Windows Mail and Live Mail are the incompetent heirs to OExpress.

The messages are in .eml plain text format so they can be imported via drag and drop or probably other means however any flags such as 'read' or 'star' will not transfer into TB.

This bug first needs documentation on how Windows Live Mail stores these flags.

Comment 1

[Ben Bucksch (:BenB)]

Phil, great!
You could already get started on importing the .eml. You can add flag support later once you know how.

Comment 2

[Kohei Yoshino [:kohei]]

Hi, is this something we'd like to have in Thunderbird 3.3?

Comment 3

[Mark Banner (:standard8)]

(In reply to comment #2)
> Hi, is this something we'd like to have in Thunderbird 3.3?

This would be generally wanted for Thunderbird, we don't need to be version specific about it (obviously the sooner it arrives the better...).

Comment 4

[Kohei Yoshino [:kohei]]

:)

Comment 7

[Steve Thompson]

(In reply to comment #6)
> *** Bug 432902 has been marked as a duplicate of this bug. ***

As I read this bug, it has to do with importing from windows or hot mail. My problem (432902) had nothing to do with either. I would select import and the next panel/window that came up had a button that did nothing. I had yet to tell it from where to import. I do not believe this bug is the same issue.

Comment 8

[Hiroyuki Ikezoe (:hiro)]

I will take this sooner or later.

Comment 9

[Hiroyuki Ikezoe (:hiro)]

Attachment: Just import mail data

Unfortunately I could not find any way to obtain message flags (e.g. Read, Replied, Forwarded...) in Windows Live Mail. So the patch just imports eml files.

Comment 10

[David :Bienvenu]

(In reply to Hiroyuki Ikezoe (:hiro) from comment #9)
> Created attachment 599452 [details] [diff] [review]
> Just import mail data
> 
> Unfortunately I could not find any way to obtain message flags (e.g. Read,
> Replied, Forwarded...) in Windows Live Mail. So the patch just imports eml
> files.

This is great, thx for working on it.

For Windows Live Mail message flags, you'd need to parse the data in the Mail.MSMessageStore file - there's information here about that - http://msdn.microsoft.com/en-us/library/ms709546%28VS.85%29.aspx, in particular the database info. It looks like a fairly complicated schema, and I don't know big the database library we'd need to link with is in order to read that file format.

Comment 11

[David :Bienvenu]

Comment on attachment 599452 [details] [diff] [review]
Just import mail data

this code assumes we're storing mail in berkeley mailboxes: ImportWMMailImpl::ImportMailbox

But that's not necessarily true anymore, now that we have pluggable stores. With pluggable stores, you need to get the store from the server or folder, and then get the output stream from the store. If the store says the stream is reusable, then you can reuse it; otherwise, you need to get an output stream for every message you import. Here's an example of getting an output stream:

http://mxr.mozilla.org/comm-central/source/mailnews/local/src/nsPop3Sink.cpp#548

Note that the existing import code is probably broken as far as pluggable stores are concerned. I should fix that. I'll file a bug to do so...

Comment 12

[David :Bienvenu]

bug 729676 filed to fix the other import code. I'll try to come up with a quick patch so you can get an idea of how to fit the pluggable store stuff into an import module.

Comment 13

[David :Bienvenu]

I can see right away that doing the pluggable store work for mail import is going to bit-rot your patch, but it's needed for your patch to do the right thing with pluggable stores. You might want to wait a day to work on this until I've got mail import working with pluggable stores.

Comment 14

[Hiroyuki Ikezoe (:hiro)]

(In reply to David :Bienvenu from comment #13)
> I can see right away that doing the pluggable store work for mail import is
> going to bit-rot your patch, but it's needed for your patch to do the right
> thing with pluggable stores. You might want to wait a day to work on this
> until I've got mail import working with pluggable stores.

Thank you for the notice. I will try with the new interface.

Comment 15

[Phil Lacy]

(In reply to David :Bien
> 
> For Windows Live Mail message flags, you'd need to parse the data in the
> Mail.MSMessageStore file - there's information here about that -
> http://msdn.microsoft.com/en-us/library/ms709546%28VS.85%29.aspx, in
> particular the database info. It looks like a fairly complicated schema, and
> I don't know big the database library we'd need to link with is in order to
> read that file format.
If i haven't missed something. This info has been out a while and still not updated to include the newer windows live mail.
I believe this is info on the old .dbx files used by oe6 and is for legacy access to that.

I never could find the windows live mail version.

I've stopped looking thru their api issues in 2010 so maybe later releases finally give interfaces for accessing the live mail version.
Hopefully i'm wrong

Comment 17

[Hiroyuki Ikezoe (:hiro)]

Attachment: Import Windows Live Mail messsages with pluggable stores

Comment 18

[Hiroyuki Ikezoe (:hiro)]

(In reply to David :Bienvenu from comment #10)
> (In reply to Hiroyuki Ikezoe (:hiro) from comment #9)
> > Created attachment 599452 [details] [diff] [review]
> > Just import mail data
> > 
> > Unfortunately I could not find any way to obtain message flags (e.g. Read,
> > Replied, Forwarded...) in Windows Live Mail. So the patch just imports eml
> > files.
> 
> This is great, thx for working on it.
> 
> For Windows Live Mail message flags, you'd need to parse the data in the
> Mail.MSMessageStore file - there's information here about that -
> http://msdn.microsoft.com/en-us/library/ms709546%28VS.85%29.aspx, in
> particular the database info. It looks like a fairly complicated schema, and
> I don't know big the database library we'd need to link with is in order to
> read that file format.

Though I am not sure those interfaces can be used for Windows Live Mail or not, the interfaces have no exported functions. So we can not use them I think.

Comment 19

[David :Bienvenu]

I'll try running with this a bit later, but in general, this looks good - a couple nits:

instead of:
+  NS_IF_ADDREF(*aKey = key);
+  return rv;

you can do:

key.forget(aKey);

similarly,

+  NS_IF_ADDREF(*aRootFolder = rootFolder);

can be rootFolder.forget(aRootFolder);

Comment 20

[David :Bienvenu]

When I imported from Windows Live Mail, I only got my first account, which happened to be imap, not my pop3 account, and it ended up with a slightly tweaked name - "Gmail (davi 66c"  whereas the account is Gmail (david.bienvenu). Also, it imported my Gmail imap account as if it was a local folders account, i.e., it imported the contents of my imap folders as local mail. The first two issues should be fixed before we land. For the third issue, it would be great if we could import imap accounts as real accounts, if we can get access to that information.

Comment 21

[David :Bienvenu]

Comment on attachment 639211 [details] [diff] [review]
Import Windows Live Mail messsages with pluggable stores

minusing based on the account name problem, and the failure to import multiple accounts.

Comment 22

[Hiroyuki Ikezoe (:hiro)]

(In reply to David :Bienvenu from comment #20)
> When I imported from Windows Live Mail, I only got my first account, which
> happened to be imap, not my pop3 account, and it ended up with a slightly
> tweaked name - "Gmail (davi 66c"  whereas the account is Gmail
> (david.bienvenu). Also, it imported my Gmail imap account as if it was a
> local folders account, i.e., it imported the contents of my imap folders as
> local mail. The first two issues should be fixed before we land. For the
> third issue, it would be great if we could import imap accounts as real
> accounts, if we can get access to that information.

I can not see the first two issues on my local. Can you post the problematic .oeaccount files please? Those files can be found in c:\Documents and Settings\(your_user_name)\Local Settings\Application Data\Microsoft\Windows Live Mail\Gmail (dadid.bienvenu)\.

Comment 23

[Hiroyuki Ikezoe (:hiro)]

I can not reproduce the two issues with the account files sent from dbienvenu on local Windows XP.

Comment 24

[Hiroyuki Ikezoe (:hiro)]

Oops! I am sorry I tested without the patch.

I can reproduce the first issue on my local now. The second one (tweaked name) has never been seen here.

Comment 25

[Hiroyuki Ikezoe (:hiro)]

David, can you please test the patch for bug 773145 first? The patch is the almost the same without mail data import part.