Last Comment Bug 10837 - [FEATURE] Import tool for 4.x address books
: [FEATURE] Import tool for 4.x address books
Status: VERIFIED FIXED
:
Product: SeaMonkey
Classification: Client Software
Component: MailNews: Address Book & Contacts (show other bugs)
: Trunk
: All All
: P3 normal with 1 vote (vote)
: M15
Assigned To: (not reading, please use seth@sspitzer.org instead)
: ppandit
Mentors:
: 30190 (view as bug list)
Depends on:
Blocks: 10791
  Show dependency treegraph
 
Reported: 1999-07-29 18:54 PDT by scottputterman
Modified: 2004-11-22 17:25 PST (History)
7 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description daver 1999-08-16 14:51:59 PDT
David,  where do we stand on this?
Comment 1 davidmc 1999-08-16 15:52:59 PDT
I was supposed to start in one milestone, then finish in a later milestone. I

have not started this yet at all, though.  I am still working on sorting; I have

been syncing my source tree since last Wednesday, for a total of about four

calendar days slip, just for syncing.  I have had a long sequence of build system

failures on my Mac, due mainly to corrupt project data which seems somehow

related to xpidl changes.



After I do sorting, I can start on the ipmort tool.  Starting is easy, because it

only involves stripping things out of teh 4.5 code so it can build in a

standalone fashion.  So that is just abunch of violent hacking and stubbing for

two of three days.  Finishing the import tool is both hard and undefined; I tried

to avoid being given the task, since it is undefined.  But Phil made it clear in

no uncertain terms that he wanted me to do it.



So far, the best theory I have heard suggested is that Tony Robinson has some

kind of plugin system for import, and this might be used as the frontend.  All I

will have is a backend for import, and providing a frontend is undefined.  I will

not be able to provide an frontend by myself, and that's why I should not own the

finishing phase for the import tool.
Comment 2 Phil Peterson 1999-09-01 10:17:59 PDT
Bulk move mail/news M10 bugs to M11
Comment 3 Phil Peterson 1999-09-17 10:49:59 PDT
Not on the B1 list. Triage to M15 since it's probably > 1 milestone's worth of
work.
Comment 4 davidmc 1999-11-09 14:06:59 PST
I've started working on this task; yesterday I set up a Mac Codewarrior project
and started compiling, and triaging dependency problems on the fly.

There is very bad news, which invalidates all time estimates I have given for
this task.  I no longer know whether the task can be done, or if it can be done,
I have no idea how long it could take to remove all the problems.

The problem is the underlying third party db used by address books, because it
seems the headers (and presumably the sources) have been hacked to depend on
header files (especially basic integer types, such as those in NSPR), and I am
very strongly forbidden to look at the sources for that db anymore, on pain of
execution by electric chair, or something with a similar dramatic flavor.

Technically, it's unclear whether I am allowed to see the header files at all
either, since they contain code and I'm not supposed to see the code, even when
it might appear in a source level debugger.  My comfort zone is only seeing the
APIs used in our own code, but not seeing the definitions of APIs in the header
files, because this seems to best support the spirit of the constraint.

Someone who understands this third party db will have to go into the source which
I am not allowed to see, and remove all dependencies on header files which will
not be present in the standalone import tool project configuration.  If I did
this, I'd remove dependencies on other header files entirely, but someone else
might take another approach.

Such changes to the underlying db might have subtle effects in either linkage or
runtime behavior that I will have great trouble resolving without being able to
see the causes of the effects myself.  So I would expect some long and drawn out
handshaking in development with another engineer trying to fiddle with the db
while I am trying to use it, but without being able to see the db.

I will follow up more on this topic in another separate entry to this bug report.
Comment 5 davidmc 1999-11-09 14:12:59 PST
I constructed the Codewarrior project by taking a snapshot of libaddr from David
Bienvenu's 4.7 tree on my Linux box, and then also taking a snapshot of header
files only from the third party db, but without looking in these files because
I'm not allowed to see the sources.  Then I started compiling.  The libaddr files
were coming along nicely, but the files which apply the third party db had
hangups from missing header files and undefined types.  I tried whacking the
lines that include external header files from the db headers, without otherwise
looking at the db code, but this merely caused undefined types such as int32,
etc.  Now I can go back to working on the libaddr files which don't use the db,
but I don't see how I'll make much progress until the db dependencies are fixed.
Comment 6 davidmc 1999-11-09 14:15:59 PST
My original theory was that the db would build standalone, and that it was not
entwined with code from 4.5, but that assumption turns out to be wrong.  The
degree of dependency is probably very, very small, such as the addition of new
methods overloaded to deal with native types specific to NSPR data formats; but
any degree of dependency is a showstopper when I have an absolute injunction not
to see the sources.
Comment 7 David :Bienvenu 1999-11-09 14:18:59 PST
It turns out that the address book code doesn't use those methods (read and
writeint32). I can remove them from my copy of those files and you can re-ftp, I
guess.
Comment 8 davidmc 1999-11-09 14:21:59 PST
My first impulse was to declare this task impossible, and write off the
possibility of importing 4.x family address books to mozilla.  However, that's
too pessimistic an outlook.  It is still possible to do, it's just much harder.
But I don't know how hard; I get very uncomfortable when there is no way to say
whether a final success state is even reachable.  There is some potential for the
task to be a time swamp that eats time and never reaches a conclusion.

My reponse to such risk is to write off the task as undoable to control risk.
But perhaps someone else would like to buy the risk and play craps themselves, as
long as it does not require that I must also take responsibility for the outcome.
Comment 9 David :Bienvenu 1999-11-09 14:22:59 PST
OK, I've removed the int32 stuff from my tree
~client45/ns/lib/libneo/nstream.h,cpp
Comment 10 davidmc 1999-11-09 14:24:59 PST
On the Mac there were more missing header files than just the NSPR integer types.
It also seemed that some header file related to AppleScript constants was being
included, and whacking this include caused some contants to become undefined when
some other constants resemling AppleScript selectors could not be found.

I worry that there might be more such things, and finding them involves looking
at the header files, which feels questionable to me.
Comment 11 davidmc 1999-11-09 14:26:59 PST
I will post summaries of error messages generated by the Mac compiler.
Comment 12 davidmc 1999-12-16 17:20:59 PST
I've decided my approach was going down a blind alley, and now
I'm going to cut my losses and change the general strategy.

Now the goal is to generate an ldif file, and this ldif file
can be imported by Mozilla 5.0 code or associated tools; the
code to write this ldif in the 4.5 style is already present
in the old 4.t5 codebase (which is where I started from)
and requires much less massaging to make work for the current
problem.  We can still have auto-import be very auto-magic
for users, by hiding that this path uses a two step process.

So I'm going to scrap my recent work, which was going in the
direction of generalized import to any destination MDB database.
A lot of earlier work I did on this task is still valid which
does not involve the MDB stuff in particular; but making this
change involves eating the cost of MDB import done so far.

Eating this cost and switching to an ldif text-based data flow
path looks cheaper though, since I still had a bunch of work
to do with the direct MDB import, especially with debugging
and integration.  Basically the ldif path is much close to the
floor in terms of complexity, and has low integration risks,
although it is generally no efficient for large import content.

Another major risk in the MDB import approach I was using, was
that handing over this code to someone else had greater chance
for confusion over what was meant by the import architecture.
In contrast, making a 4.5 style ldif file has obvious results.

I talked with Candace and she says mozilla has at least first
draft versions of code for importing ldif to address books, and
that this might be used by Tony's tools for importing.   The
new approach I plan to use for creating ldif should fit right
into other data migration paths he must already handle.
Comment 13 davidmc 2000-02-21 14:39:03 PST
assigning to bienvenu, so he can pass it on to whoever owns the stuff I did.
Comment 14 David :Bienvenu 2000-02-29 08:22:35 PST
accepting
Comment 15 (not reading, please use seth@sspitzer.org instead) 2000-03-02 20:54:44 PST
adding me to the cc list.
Comment 16 (not reading, please use seth@sspitzer.org instead) 2000-03-02 20:56:10 PST
*** Bug 30190 has been marked as a duplicate of this bug. ***
Comment 17 (not reading, please use seth@sspitzer.org instead) 2000-03-02 20:58:36 PST
I just marked a migration bug a duplicate of this one.

I think this bug covers the automatic migration of addressbooks issue.

david b, since I'm the migration lackey, would you like to pawn this bug off on
me?

I wouldn't hold it against you.  At least, not for long.
Comment 18 David :Bienvenu 2000-03-03 07:25:54 PST
I still have to get DavidMc's stuff to build and run on mozilla for windows, 
linux, and Mac, checked into CVS, with project and make files - this will be a 
week's work, probably, so it'll be a while before it's in any state to get 
pawned off :-(
Comment 19 David :Bienvenu 2000-03-20 19:30:59 PST
giving to Seth to administer the coup de grace to this bug. CC'ing JF who kindly 
volunteered to help with the mac project.
Comment 20 (not reading, please use seth@sspitzer.org instead) 2000-03-20 19:35:20 PST
accepting.  I'll work on this tomorrow.
Comment 21 (not reading, please use seth@sspitzer.org instead) 2000-03-21 23:39:58 PST
I've got this working.

there are some polish issues that need to be finished.

I'll show david b what I've got tomorrow or thursday, and start checking it in.
Comment 22 (not reading, please use seth@sspitzer.org instead) 2000-03-23 20:34:40 PST
automatic migration of 4.x addressbooks works.
import of 4.x addressbooks works.

it only works in the commercial build.

ducarroz is going to turn it on for the mac tonight.

marking fixed.
Comment 23 Jean-Francois Ducarroz 2000-03-23 22:05:28 PST
Fixed on Mac as well.
Comment 24 ppandit 2000-06-28 14:25:33 PDT
Using 6/27 release build on nbaca's NT system

Able to import address book entries from a 4.7 profile. 

Par

Note You need to log in before you can comment on or make changes to this bug.