1775 - stream converters need to implement the new StreamConverter interface.

Status: RESOLVED INVALID

(Core :: DOM: Serializers, defect, P2)

Description

[rhp (gone)]

I've made a couple of temporary changes for netlib. I have a
version of libmime running in a DLL on my machine that works
with Raptor and will render Berkeley mail folder messages.

To do this (and keep things pretty modular, I need to
register my stream handling function for RFC822 messages
from outside of netlib. Can we add NET_RegisterContentTypeConverter
to the \mozilla\network\module\net.defmodule definition
file? When we get the XP-COM interface for this function,
we can remove this export.

Also, I've added the lines:

"exts=\"eml\"                 type=message/rfc822     \
desc=\"RFC822 Messages\"      icon=resource:/res/network/gopher-text.gif",

to the \mozilla\network\mimetypes\ngtypes.h. This is the
system association for mail files on most Win32 machines.

We need an XP-COM way of registering both the stream converters and
mime associations.

- rhp

Comment 1

[leger]

Setting all current Open/Normal to M4.

Comment 2

[Paul MacQuiddy]

setting paulmac as QA contact for all gagan's bugs (sorry for the spam)

Comment 3

[Gagan]

Rich:
Do you want to help us do this?
Marking till Necko lands...

Comment 4

[rhp (gone)]

I do want to be involved since I will be the first customer of this
functionality, but I'm not sure I can take it on my plate to own. Are we out of
luck if I can't do it because this is a must have for us with the new netlib.

- rhp

Comment 5

[rhp (gone)]

This really needs to assigned to someone in the netlib group...phil, any
thoughts?

- rhp

Comment 6

[Phil Peterson]

Reassigning to [e-mail]. Rich, are you blocked on this, or is it just
something we have to do before B1?

Comment 7

[rhp (gone)]

No, nothing blocked here, in fact I have this all working today with an XP-COM
interface for netlib stream converters. This is just a heads up in the event
that netlib is rewritten since I could see this slipping by the wayside for a
while. Just want to stay on the radar screen.

- rhp

Comment 8

[chris hofmann]

is this m6 or is it now moved into the necko sched.

Comment 9

[rhp (gone)]

This is a Necko thing and should tie in with that schedule. Actually,
it will probably change from a stream converter to a new DTD to handle
message/rfc822 data.

- rhp

Comment 10

[chris hofmann]

moving to m7.  warren can you set the right target milestone if thats not
 right?

Comment 11

[rhp (gone)]

Warren,
In light of our meeting last week, will there still be an interface like this,
or will the DTD approach be the way this stuff is handled. Also, was there a
follow up on this discussion?

- rhp

Comment 12

[Warren Harris]

I think this probably falls on rickg's plate now. (Rick, let me know if not.) I
wouldn't expect to see this before M9 at the earliest.

Comment 13

[rickg]

Warren -- I'm giving this back to you as a means of closing the loop in the
issues. (I may own some of this). Rich is asking for 2 things:
   1. Registering stream converters
   2. mime associations

I won't be the owner of #1 (I think you are), and I *may* own #2 as soon as we
can all agree on the plan we're following.

Comment 14

[Scott MacGregor]

Adding myself to the cc list.

Comment 15

[chris hofmann]

m9

Comment 16

[Scott MacGregor]

Actually I'm going to bump this back to M8. It's part of our mailnews necko
integration stuff. We need it in order to integrate mailnews which is an M8
item. We're talking to gagan and rickg about this stuff.

Comment 17

[Warren Harris]

Off to gagan...

Comment 18

[Gagan]

Necko landing...

Comment 19

[Gagan]

Pushed past necko landing...

Comment 20

[Scott MacGregor]

*red flag* *red flag*

we know we can't land necko without a pluggable stream converter service right?
I'm going to jump up and down and do acrobatic stunts unless we're sure necko
isn't going online until after M8 (which is probably reasonable). In which case
I buy that this could be an M9 bug. But to me this is a requirement for necko
landing if we think we're going to be turning necko on.

Comment 21

[leger]

Changing all Networking Library/Browser bugs to Networking-Core component for
Browser.

Occasionally, Bugzilla will burp and cause Verified bugs to reopen when I do
this in a bulk change.  If this happens, I will fix. ;-)

Comment 22

[Gagan]

Over to Jud for the latest on this...

Comment 23

[Judson Valeski]

latest interfaces for this are in mozilla/netwerk/streamconv/public. I am
actively working on this.

who's doing the 822<->html converter, mscott or rhp?

Comment 24

[rhp (gone)]

Scott is more in tune with what needs to go on to plug in libmime, but I can
surely help if pointed in the right direction...scott?

- rhp

Comment 25

[Scott MacGregor]

That's probably me Jud. Since we have that hack working in the file channel this
isn't such a pressing need for us right now. We can probably move this off the
M9 radar unless you are planning on being done with M9.

Mime is currently using the nsIStreamConverter that I have in
netwerk\base\public. We should compare it to the one you are working on and see
how much more (or less) stuff needs to be done.

Comment 26

[Judson Valeski]

*** Bug 11310 has been marked as a duplicate of this bug. ***

Comment 27

[Judson Valeski]

This interface has been specified. I'm changing the summary to the real issue.
I'm not sure this is really M9.

Comment 28

[chris hofmann]

m10

Comment 29

[Scott MacGregor]

hey Jud, is the stream converter service hooked into layout yet? i.e. if we do
use the new stream converter, will the doc loader know to use the converter
service to convert to something it can handle?

Comment 30

[Judson Valeski]

mscott, yes. The stream converter sits in the nsChannelListener (which sits
inbetween necko and the doc loader). All you have to do is use the new stream
conv api, and add you mime-type to the nsChannelListener::OnStartRequest().

Comment 31

[Scott MacGregor]

Excellent! Thanks Jud. I'll start converting mime over this week. It should be
pretty straightforward because mailnews\mime was already converted to use
nsIStreamConverter2(??) which is pretty similar to the one you eventually ended
up using I think.

Comment 32

[Judson Valeski]

mscott has mail/news using the new api. miltipart-mixed-replace, and FTP
directory listings are using it as well. Gessner/pnunn/ and I are looking at it
for stream scanning -> mime-type guessing. I'm assigning this bug to akkana as
she has some stream converter logic to convert to the new api. This is not a
priority as she's already got her stuff going using her api. If she decides do
kill this bug that woudl be fine w/ me, but it would be nice to be unified on a
stream converter api.

Comment 33

[Akkana Peck]

I'd like to do this, and plugging in the code that actually does the conversion
is no problem; the problem is that I don't have time to figure out how the
stream converter APIs work.  I'd love help with this.

Comment 34

[Akkana Peck]

Changing Component to Output.

Comment 35

[Warren Harris]

Hi Akkana, Any progress on this one? Is this really M20 -- it shows up on the
beta criterion list for necko (bug#12833).

(And what's the Output component?)

Comment 36

[Ben Bucksch (:BenB)]

Warren,
akk asked me to take a look on this. I did, but it is fairly complex for me. I
would need a month or so without success garanty.
If this needs to be in beta, don't count on me. If not, *maybe* I'll fix it.

Comment 37

[Akkana Peck]

Clarification on what I'm looking for here:
I made a start on this, in htmlparser/src/nsTextConverter.{h,cpp} based on the
sample stream converter APIs, but I kept hitting questions that I didn't
understand about the interface -- for instance, what the several stream
arguments mean and how they all relate to each other, and whether asynchronous
mode really needs to be supported.

It would be a great help if someone who understands the stream converter API and
has a need for this (and therefore knows how it's going to be used) would get
these files to where they compile, then I can plug in the code that does the
actual conversion.

Comment 38

[rhp (gone)]

Hi Akkana,
This is a situation where I need to go from RFC822 to Plain Text, but without 
the HTML - text stream converter interface in place, I'm kind of stuck. Is 
there any change of this getting moved up from M20?

- rhp

Comment 39

[leger]

Bulk move of all "Output" component bugs to new "DOM to Test Conversion" 
component.  Output will be deleted as a component.

Comment 40

[Paul MacQuiddy]

spam: qa contact to [e-mail]

Comment 41

[John Morrison]

qa -> sujay

Comment 42

[Akkana Peck]

Per beppe, marking future.

Comment 43

[Akkana Peck]

Anthonyd is taking over Output.  Not sure whether this is still an issue, though.

Comment 44

[Akkana Peck]

Really assigning to anthony this time.  Not clear whether anyone still wants
this, though.

Comment 45

[Ben Bucksch (:BenB)]

> Not clear whether anyone still wants this, though.

I still think, it's a good idea, generally, theoretically. But I won't do the
work ;-P.

Comment 46

[rubydoo123]

--> kin

Comment 47

[Daniel Bratell]

I think this bug is obsolete since the conversion moved to content. If nobody
comments I will INVALID it.

Comment 48

[(not reading, please use [e-mail] instead)]

actually, this is still needed.

for why, see nsMessenger::SaveAs()

but we've worked around it for this long, I don't expect it to happen any time 
soon.

Comment 49

[Boris Zbarsky [:bzbarsky]]

So what is this bug about at this point, Seth?  Is it about making our document
encoders and decoders implement nsIStreamConverter so that you can, say, pump in
HTML and get out plaintext or something along those lines?

nsMessenger::SaveAs seems to just need a RFC822 to text/plain converter; right
now you use this saveListener object to handle that?

Akkana, I'd be willing to help out with stream converter stuff; I think I have a
pretty good understanding of how they work....

Comment 50

[Akkana Peck]

I'd be happy to help on how to call the serializers, if you need it.  You might
want to start with the automated test in htmlparser/tests/outsinks, since that's
a standalone converter from html to whatever (plaintext in the case of the
existing tests).

Comment 51

[Akkana Peck]

Oh, Boris, there's also some serializer doc at
http://www.mozilla.org/editor/serializers.html
If there are omissions there that you think should be mentioned, let me know and
I'll update the document (or feel free to make your own additions, if you feel
so inclined).

Comment 52

[Boris Zbarsky [:bzbarsky]]

Well, I'm still trying to figure out what the goal of this bug is... once this
bug is fixed, what do we want to be able to do?

Comment 53

[(not reading, please use [e-mail] instead)]

bz, sorry for the delay.

here's what this bug is about.

in mozilla mail, we allow the user to do File | Save As | File...

They can save as:

.eml (rfc/822), .txt (plain text), or .html (html)

when saving to .txt, we need these two converters:  "rfc/822 -> html" | "html ->
plain text"

There's no stream converter (at least, there wasn't) that can go from html ->
plain text

the way the current code works is if you save a message as plain text, we still
convert
rfc/822 to html, and we buffer it up.

then, in nsSaveMsgListener::OnStopRequest(), we call a static function
ConvertBufToPlainText()
(in nsMessenger.cpp, which uses the parser and a nsIHTMLToTextSink to convert.

so to clean up the code in nsMessenger.cpp, we'd a stream converter that could
take html and convert to plain text.

if you have more questions, let me know.

I need to re-read more of the bug, because the summary doesn't match what
mailnews needs anymore.  (has this bug morphed? or is the original issue still
open?)

Comment 54

[Boris Zbarsky [:bzbarsky]]

Seth, thanks for the clarification.  Sounds like it would be good if content
registered a converter component that basically wraps one of the serializers....
I'll have to look at how the serializers work, exactly.  ;)

Comment 55

[Anne (:annevk)]

This seems no longer relevant. If it still is, a new focused bug would be better.