11049 - UNIX Movemail

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect, P3)

Description

[(not reading, please use [e-mail] instead)]

see this thread:
news://news.mozilla.org/375D61D1.B3CD128F%40informatik.med.uni-giessen.de

read this:
http://lxr.mozilla.org/seamonkey/source/mailnews/movemail/README

old builtin movemail code from 4.x in
http://lxr.mozilla.org/seamonkey/source/mailnews/movemail/src/movemail.c

Comment 1

[(not reading, please use [e-mail] instead)]

marking m15

Comment 2

[Phil Peterson]

Bulk-resolving requests for enhancement as "later" to get them off the Seamonkey
bug tracking radar. Even though these bugs are not "open" in bugzilla, we
welcome fixes and improvements in these areas at any time. Mail/news RFEs
continue to be tracked on http://www.mozilla.org/mailnews/jobs.html

Comment 3

[Phil Peterson]

Reopen mail/news HELP WANTED bugs and reassign to [e-mail]

Comment 4

[(not reading, please use [e-mail] instead)]

#12801 was the bug where I divorced local mail and pop3.
now that bug #12801 is fixed, doing movemail should be much easier.

Comment 5

[Adam D. Moss]

I'm trying to implement this, since no-one else
has picked it up to date.

I've assigned it to myself although I won't be
extremely upset if someone else with more free time
wishes to work in parallel and beats me to a working
implementation.  =)

Comment 6

[Adam D. Moss]

Assigning to myself.

Comment 7

[lchiang]

Once implemented, I'll need to find someone to test this.

Comment 8

[Adam D. Moss]

I have a basic moronic implementation sort of
working in my tree now.  At least, it's being
tickled in the right places.

I'm having some trouble though, chiefly that I
can't identify the mechanism by which I should
feed the spool (or at a pinch a message at a time)
to the right mailbox and have Mozilla become aware
of the new messages/headers etc.

So currently I have something very crude hanging on
GetNewMail which sidesteps a lot of stuff and simply
appends the spool file wholesale to a specific folder
in the correct mozilla mail directory.  Even this isn't
quite doing the trick, since (understandably) mozilla
does not build an index for this folder until it is
restarted, and even then usually crashes upon opening
one of these messages.

I need some more hand-holding on #mozilla (nick:Aspirin)
or in email, I suspect.

--Adam

Comment 9

[(not reading, please use [e-mail] instead)]

adamn, does 

http://lxr.mozilla.org/seamonkey/source/mailnews/movemail/src/movemail.c

answer of you questions?

if not, you can find me in #mozilla as sspitzer or sspitzer_msg_me

Comment 10

[Adam D. Moss]

I've read this code but I was under the impression that
the piece that I'm actually stuck on, which is having my
messages consumed by Mozilla and excreted into a folder
through the 'right' interface, would not resemble the
solution in this classic-codebase code.

Please correct me if I'm wrong.

To clarify...  my main blocking point right now is
figuring out the mechanism by which I should feed this
mbox, either wholesale or a message at a time, into
the right folders.  There must be a magic interface for
this which also handles message filtering etc., but even
after reading through the POP/POP-protocol code it is
not clear to me where the messages are being produced
and consumed.

Thanks,
--Adam

Comment 11

[David :Bienvenu]

You probably want to be looking at mailnews/local/nsParseMailbox.cpp - that's 
how mail messages get streamed into the inbox, and how the filtering code gets a 
crack at the incoming messages. nsPop3Sink::BeginMailDelivery(PRBool* aBool) 
creates a mailbox parser and streams data into it. Let me know if you have more 
questions after looking at this code.

Comment 12

[scottputterman]

Are you talking about that fact that messages aren't showing up in the thread
pane of an open folder?

If so, the way this is hooked up for all of the other protocols is that each
folder has a db. When new messages come in, the db calls various
nsIDBChangeListener methods. The folder is a listener and converts the results
to something that it can send onto the ui.  The parse mailbox code is where the
updating of the db happens.

Comment 13

[Adam D. Moss]

Thanks, that was a big help.  I think I have the mailbox
population pretty much licked now.

My next problem is that messages deposited in the Inbox
by the movemail module are unopenable -- clicking on
the message summary in the summary pane does not open
the message.

Something is *trying* to do it, since a
mailbox_message://blah/Inbox#0 URL is being displayed
as a debugging message, but it's not actually
working.

It's not due to the actual contents of the mailbox
trying to be opened, as moving a working pop-originated
Inbox and its summary-file over the movemail-account
ones still renders the messages unopenable.

I'll chew on that one some more, though if anyone
instinctively has a strong idea what's causing it
then I'd welcome further input.

Cheers,
--Adam

Comment 14

[Adam D. Moss]

Right, I have a working Movemail implementation now.
I'll attach it here after some clean-up.
--Adam

Comment 15

[(not reading, please use [e-mail] instead)]

excellent!  when it comes time for a reviewer / tester, I'll do it.

Comment 16

[Adam D. Moss]

Initial patches submitted to netscape.public.mozilla.mail-news and
netscape.public.mozilla.patches, Subject: "[patch] Movemail support."
--Adam

Comment 17

[(not reading, please use [e-mail] instead)]

I hope to try out and check in adam's fixes soon, before it suffers bit rot.

I will have to make some changes (see 
news://news.mozilla.org/38C2ED30.E22D0E6E%40netscape.com)

marking m15.

Comment 18

[Roland Mainz]

Question: Does this "movemail" implement "movemail IMAP-style" (= movemail
leaves mail in /var/mail/$USER, adds references to it's database and marks mail
in /var/mail/$USER as "read" (is that possible ?)) ? This would allow users to
use the mailer of their choice (like pine, /usr/ucb/from, dtmail etc.) to
read/send messages...

Comment 19

[selmer (gone)]

Mass moving to M16 to get these off the M15 radar.  Please let me know if this
is really an M15 stopper.

Comment 20

[Adam D. Moss]

In short answer to Roland's question, 'No.'

As an almost-equally-brief progress update, I have movemail
support working again in my local tree after letting it
bitrot w.r.t. the main tree for too long.  Will submit a
new patch when I have time.

--Adam

Comment 21

[Roland Mainz]

Is there someone interested in the "movemail IMAP-style" idea ?

Comment 22

[Adam D. Moss]

Attachment: changes relative to 2000-05-11 CVS

Comment 23

[Adam D. Moss]

Attachment: new files relative to 2000-05-11 CVS

Comment 24

[Adam D. Moss]

Here are the changes for embreonic Movemail
support relative to current CVS.  They are
merely an update of the previous patches which
formerly broke over time due to interface and
general code drift.

The update still doesn't 'do' any more than the
previous patches (eg. still doesn't write/truncate
the spool file, so concurrent 'get new mail's will
get the same mail multiple times).

Comment 25

[Ben Bucksch (:BenB)]

*** Bug 7836 has been marked as a duplicate of this bug. ***

Comment 26

[Mike]

M16 has been out for a while now, these bugs target milestones need to be 
updated.

Comment 27

[Roland Mainz]

What's the status of this bug/RFE ? Is "movemail" in Mozilla working - if YES
the only "remaining" issue would be to update the "Create new mailnews-account
wizard"...

Comment 28

[(not reading, please use [e-mail] instead)]

sad to say, movemail is not working in mozilla.

I haven't had the cycles to help adam finish it up.

this will probably slip to 6.1

Comment 29

[Roland Mainz]

Is there a change to "bump"-up the priority of this RFE (well, we have 12 votes
for this bug...) ?

Os is there a way we can help ?

Comment 30

[Adam D. Moss]

The status is that the backend mechanics need
only a little more twiddling to be in fine
working order, but the integration with the
core needs to be done in a more intelligent and
flexible way than we've approached the problem to
date, to address various issues -- and this is
beyond my available time and knowledge at this
point with respect to the rapidly-diminishing window
onto 6.0.

I might find time to crib some catagorization code
from elsewhere in the codebase, but I don't know how
big a job that'd be (Seth, any idea?).  I'll report
any change of status right here.

Comment 31

[(not reading, please use [e-mail] instead)]

help is always appreciated.

the big things to do are:

1) restore adam's work.  it's been a while, there has probably been some bit rot
as the mailnews code has probably shifted.

2) write/truncate the spool file (see adam's comments below)

3) making the ui modular so movemail only shows up on unix
see news://news.mozilla.org/38C2ED30.E22D0E6E%40netscape.com

most of the work is in task #3.

Comment 32

[Alec Flett]

I was thinking about the dynamic issue - an easy thing to do might be to use
dynamic overlays to insert the movemail option into the account wizard page - a
tiny bit less work, and a whole lot less risk.

Comment 33

[(not reading, please use [e-mail] instead)]

alecf: how about the account manager?

Comment 34

[Alec Flett]

account manager wouldn't need anything more than the extra entry in the
.properties file to verbosely display "Movemail" in the server pane.

Comment 35

[Adam D. Moss]

I've updated the movemail code again to counter
bitrot/interface-rot.  When I have the time to
perform at least perfunctory testing I'll
attach the new patches.

Treading water.

--Adam

Comment 36

[lchiang]

*** Bug 45979 has been marked as a duplicate of this bug. ***

Comment 37

[Adam D. Moss]

My local tree is patched up to build with
respect to trunk driftage again.  Untested for
a while -- will post a fresh patch when I have
time to test.

Comment 38

[Adam D. Moss]

Attachment: 09-26 Diffs between trunk and movemail tree

Comment 39

[Adam D. Moss]

Attachment: 09-26 New files within movemail tree

Comment 40

[Adam D. Moss]

Well, my local tree still appears to be working
w.r.t. movemail support so I've attached a fresh
set of patches and a tarball of new files.

Comment 41

[Adam D. Moss]

Does anyone have suggestions concerning the:
a) Easiest
b) Right
ways of integrating with the front-end (so that
only UNIXoids get the movemail options -- at the
moment these options are given to all users using
these patches)?  AFAICS that's the main higher-level
thing holding this up -- right?

I'd also like opinions on whether this has majorly
missed the mozilla1.0 boat, and whether either option
a) or b) above would assist in swimming the channel.
It's definitely missed NS6.0 (correct?).

Comment 42

[Adam D. Moss]

(n.b. when I say 'holding this up' I mean 'keeping
this out of CVS', I think.)

Comment 43

[David Hallowell]

Adding 4xp keyword. This really needs to be fixed ASAP it was a well used
feature in Netscape Communicator and previous versions of Netscape.

If people get their mail delivered to their UNIX mailbox then movemail provides
the only option for them to read their mail using Netscape/Mozilla. Therefore
we're limiting the acceptance of Mozilla as a UNIX mailreader to people who get
their mail through POP3 or IMAP.

Most UNIX users want to rid their systems of Netscape 4.x don't let something as
simple as this stop them.

Comment 44

[(not reading, please use [e-mail] instead)]

adam, first thanks for being more than patient with me for not getting this in 
yet.

movemail has missed the NS 6.0 boat.  It has not missed the Mozilla 1.0 boat.

I would like to redeem myself by helping you finish this work and getting it 
checked into the trunk.  We will have to do some work to make it so the UI only 
shows up for the UNIX builds.  But that should not be too bad.

I'll go talk to my manager and see if I can get some time to work on this on the 
trunk.

I'd hate to see your movemail work bit rot (again.)

Comment 45

[Alec Flett]

I think we can check in all the backend changes safely (onto the trunk) and wory
about the frontend in a little while. 

As for getting the front end correct, we need to start using categories to
represent the server types. Then, we can populate the frontend (the wizard) with
a list of server types. 

My suggestion is to reflect it into RDF and then construct the combo box with a
<template> - This will also allow us (at some point) to add the same combo box
to change the server type in the account manager.

Comment 46

[Adam D. Moss]

Thanks again seth and alecf!  Let me know what is
within my humble powers to help on this.
--Adam

Comment 47

[(not reading, please use [e-mail] instead)]

marking as mozilla 1.0

Comment 48

[(not reading, please use [e-mail] instead)]

I spent my saturday and landed adam's patch on to the trunk.  (thanks for
providing a fresh and easy to land patch).

movemail works for me on linux.  I set up an internal movemail account in 4.x,
and migrated to 6.0 and was able to read mail!

as we know, there is a bunch of UI missing, but that is another bug.

I'll go log some bugs on that.  I'll also ask for a new "movemail" module and
bugzilla category (under mailnews).  

adam and I will be the on the hook for it.

marking fixed.

Comment 49

[(not reading, please use [e-mail] instead)]

verified.  people are using movemail on the trunk.

Comment 50

[Roland Mainz]

Bug 27853 has been marked as a "duplicate" of bug 56669. Is it OK with replacing
"Bug 11049 blocks bug 27853" with 56669 ?

Comment 51

[timeless]

*** Bug 94937 has been marked as a duplicate of this bug. ***

Comment 52

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b737251c3bb5
[fenix] Add tests for ReaderModeController (https://github.com/mozilla-mobile/fenix/pull/11049)