18722 - Implement dynamic XSLT

Status: VERIFIED WONTFIX

(Core :: XSLT, enhancement, P5)

Description

[Dan Rosen]

We need XSLT in Mozilla, in a bad way.

XSLT could be effectively used in Ender to provide XML capabilities. It could be
used in a XUL editor even (see bug 15144). It could be used in web-based apps to
provide constant styling as a DOM tree is modified by a client-side script. It
could have a *lot* of power, and make the browser absurdly useful as an internet
application platform.

It doesn't seem the current XSLT implementation can be used that way, but I'm
working on an implentation, for my thesis, which can. It will have the following
capabilities:
- functionality-by-namespace: XML in certain namespaces may be registered to
have additional functionality in code. XSLT is covered by this (anything with
the xsl namespace can know how to transform things), FO could conceivably be, as
well as XUL.
- dynamic transformations: both the source document and stylesheet may change
(by scripting, in the context of an editor, etc.) and the result document will
change to reflect those changes without having to be regenerated from scratch
for each change.
- chaining: the result of one transformation may be the source or stylesheet of
another.
- modularity: this will be usable with Mozilla's DOM as well as anybody else's,
due to an abstraction based on the W3C's "infoset" recommendation.

Please consider this "bug" as an offer of enhancement. I expect my thesis work
to be mostly usable by May of 2000, and I'm certain Mozilla will still have the
need for such functionality at that time. The URL provided is my thesis page.

Comment 1

[Nisheeth Ranjan]

Welcome to the Mozilla community, Dan.  Best of luck with your thesis.  We look
forward to your contribution.

Marking this as milestone M15...

Comment 2

[Dan Rosen]

I've updated my thesis page with a fairly thorough semi-technical explanation of
the XSLT software, which may be very useful information to anybody associated
with this Mozilla bug. The technical info section is password-protected, but
please feel free to email me for the password.

Comment 3

[Dan Rosen]

Okay, since some postings on netscape.public.mozilla.layout.xslt and xsl-list,
this bug has gotten very popular in terms of votes. I guess I ought to say
"thank you," but I wanted to make sure folks were aware of a couple things:

1. This enhancement WILL NOT be ready until May 2000. For some people,
especially those who are particularly eager about this, May is not nearly soon
enough.
2. This is work I'm doing for my honors thesis in CS at Brown. I will not be
receiving outside help coding the engine (since my work for Brown has to be my
own); the only help I will be getting is on Mozilla integration which is not
part of my thesis, and advice on design and algorithms (which would, by the
way, be much appreciated).

Basically, the point I wanted to make is this: in May, I'll hopefully have
finished this and life will be grand; but until then, if there is extreme
pressing need for an XSLT processor in Mozilla, TransforMiiX is the clear
present solution.

Therefore, I've changed the summary of this bug to "Incremental XSLT," to make
the separation between this and TransforMiiX (and I probably ought to get in
touch with Keith Visco to make sure I'm not stepping on toes), and assigned this
bug to myself.

Okay?
Cool.

Comment 4

[Mona M. Everett]

I am part of an XML group for a large consulting company.  We are going to have
a very difficult time supporting Mozilla if you do not have XSLT.  CSS just does
not cut it!

Comment 5

[Eric Galluzzo]

Is there an equivalent to XT's xt:node-set function in Mozilla's proposed XSLT
processor?  I've found this capability (converting a result tree fragment to a
node-set) essential in my own XSLT stylesheets.  For example, I wrote a
topological-sort algorithm in XSLT, which returned a result tree fragment, which
I had to convert to a node-set in order to use.

Comment 6

[Dan Rosen]

[e-mail]: i suppose so, if people want it :)

by the way, discussion of xslt in mozilla in general is better suited for the
newsgroup, netscape.public.mozilla.layout.xslt. it is entirely possible that
this particular bug will never get resolved and that work on transformiix will
be much more viable for immediate use in the browser...

Comment 7

[Dan Rosen]

updating whiteboard and milestone (q: is there an designated milestone for
"sometime in the hopefully not-too-distant future?)

Comment 8

[btruong]

I vote for adding a XSLT engine to Mozilla. Microsoft has already done it in IE 
5.

Comment 9

[Dan Rosen]

Status update, sort of:

The real-time processor will not be anywhere near primetime for quite a while
due to my thesis killing me :( ... however, there is hope. They are attempting
to integrate TransforMiiX into Mozilla for M16, a goal which is just contingent
on the cleaning of several month-old bit rot. Not a minor task, to be sure, but
the TransforMiiX processor already works.

As for the real-time processor (note: incremental means something else), it will
be very useful but needs time to be done *right*, especially in terms of
integration with the rest of Mozilla's processing model. Future plans include,
as mentioned, XML editor, XUL constructor, script-based transformations, remote
XUL transformations, and other crazy fun stuff to wrap your brain around.

Stay tuned. Until then, support TransforMiiX as much as you can :)

Comment 10

[Axel Hecht]

Hi folks,
please change the component of this BUG to XSLT, as there is now a component
on bugzilla, just for XSLT and transformiix.

Thanx

Axel

Comment 11

[gwolfjdc]

We are building a distributed OAM system and a Browser front end is the KEY. 
The way in which MS IE5.x handles XSLT is EXACTLY what we need, the only 
problem is the limited platform support. Mozilla / Netscape would blow away IE 
if it had this capability. It would allow us and other companies to start 
tranforming the browser from a 'TV set' into an application delivery platform.

Comment 12

[Dan Rosen]

Setting to future, reassigning to the other me.

Comment 13

[Dan Rosen]

Changing summary to something more appropriate... Also, see bug 51420, since it
is the underlying conceptual task behind this feature.

This feature, by the way, will also let XUL templates migrate from using RDF to
being dynamic XSLT, if we choose to go that way (cc'ing hyatt, whose idea it
was, and trudelle, since he might think this is useful).

Comment 14

[Dan Rosen]

Ignore previous comments about XUL templates, since some RDF datasource fu would
be involved.

Setting dependency on 59317 (yes, I'm finally getting to work on this some):
content construction will need insertion point pseudo-nodes.

Comment 15

[Dan Rosen]

UPDATE
------

Hi everybody. I thought, since there is a huge number of people CC'ed on this
bug, that I should post an update here. First of all, thanks for your interest.
I'll hopefully be making some significant progress on this bug in the coming
couple months, since Netscape may let me spend some portion of my time working
on this.

That said, this bug will probably be generating some serious bugzilla spam, so
if you're not interested in watching gritty details play out, feel free to
remove yourself from the CC or "voted for this bug" lists -- nobody will be
offended.

Also, a reminder, if you have any XSLT-in-Mozilla related questions or comments,
the newsgroup netscape.public.mozilla.layout.xslt is where you want to be asking
them.

The short story is that I have some preliminary architecture, a processing model
for dynamic XSLT that's efficient -- on paper, at least -- and a lot of energy
to work on this. I'm trying to coordinate my efforts with Axel Hecht, Peter
Vanderbeken, and Keith Visco of the TransforMiiX project (the XSLT
implementation currently used in Mozilla). With any amount of luck, we'll get
things together, and you'll see code showing up in the tree (from me, at least)
within the next few weeks.

Regards,
dr

Comment 16

[Dan Rosen]

finished v0.1 draft of my "how the hell is this supposed to work" document,
posted at http://people.netscape.com/dr/xslt (should be mirrored from internal
staging server midnight, supposedly).

cc'ing axel hecht, peter vanderbeken, and keith visco

Comment 17

[Axel Hecht]

Added arch keyword, I am quite sure this is pretty involved with alot of other
post 6.0 plans.
Is there an ETA for those from the architecture and XML/DOM groups?
vidur? nisheeth?
How much of layout is moving in our vicinity? Is XML/DOM moving more towards
events? What kind of events?
I personally feel quite unsafe to think about this without knowing what will
change.
This feature heavily relies on the capabilities of our XPath implementation,
who will be using this? XPath like this, XBL, XSLT, XLink for sure, others?

On Dan writings:
I am currently thinking about the XPath implementation, and in my mind is some
measure of how many levels of depth influence the Expression. If you have some
number of children C, and a depth D, XPath would have a complexity O(C^D)?

About the modification listeners, I see them more as the Steps, instead of the
XPath expression.

Concerning the Insertion Points, can we unify those? I don't see this as a 
trivial goal, given the amount of content creation elements in XSLT, it's
all of sections 7 and 8 in the XSLT specs.

We may wanna open a seperate bug for the XPath implementation, but I'd like to
get a scetch of the stuff ahead first.

Axel

Comment 18

[Dan Rosen]

axel: good choice of keyword :)

i'm definitely planning on implementing xpath first, with interfaces (nsIXPath)
so that any client can use them for a one-pass evaluation. i'll have another set
of interfaces that will let them be used dynamically (nsIDynamicXPath). your
evaluation of xpath calculation is correct afaict, but re-calculation (given the
prior result and what was changed in the evaluated document) should be quicker.
i believe that both xbl and xslt will use nsIDynamicXPath, so this obviously
needs to be really good :)

i'll open a new bug to implement dynamic xpath.

with respect to events, dom level 2 events are what we use all over the place,
and hyatt has a patch sitting in his tree to implement mutation events (which
are the ones relevant here).

with respect to unification of insertion points, that's what 59317 is (if i
understand you correctly). all i need is placeholders -- the functionality
you're referring to (secs. 7 and 8 in the spec) will be implemented using those
insertion points in part, but it's much more complex than just that. i'll get
into more detail in my next revision of the "how this will work" document on my web.

with respect to how re-architecture will affect this... well, i'm sure it will,
especially if lots of work on coalescing the object models happens. i'm going to
try to get involved in that effort so i know what's going on. regardless,
though, the implementation won't be doing anything special or "nonstandard" to
dom nodes, so as long as we implement the dom (which i think is a sure bet) this
should be able to work just fine.

btw, i'll be writing up a short "vision" document, hopefully today, which
includes a couple things i think this will be useful for...

Comment 19

[Dan Rosen]

vision document added to website, implementation document updated (pending
staging server mirroring content). seeking feedback, as usual. work progresses.

Comment 20

[Dan Rosen]

Axel posted some kickass research into what needs optimizing in bug 59783.
Anybody who's interested in query optimization, and the guts of this dynamic
XSLT implementation, should definitely check out what he found.

Comment 21

[Dan Rosen]

doesn't look like i'll be doing any xml work anytime soon :( ... reassigning to
peterv for safekeeping.

Comment 22

[Axel Hecht]

dr's documents have gone, these bugs are cluttered and from what we know about
the costs during XSLT transformations, this is not going to happen.
For those that want a similiar feature, we have a js interface, which will be
redone shortly, so once you're done with modifying you can just redo the transform.
We try to make this fast day by day, and put the little resource we have there.

so WONTFIX

Comment 23

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Just to let all voters for this bug know: Mozilla does already support XSLT.
This bug was about having *dynamic* XSLT, which is something rather different
and *extreamly* hard to implement (noone has ever done it).

I am very much to fault for tricking people into beleving that this was the
XSLT-in-mozilla bug. Sorry about that :-)

Comment 24

[Dan Rosen]

Well, it's a sad day that I knew would eventually come. I'd still like to
implement this some day, and I do still have my docs if you're interested, but
who knows if I'll actually ever do it. :P

Verifying WONTFIX.

Comment 25

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Agreed, it'd be nice to do this someday, but it's very far down the
prioritylist. If you want you can attach your docs to this bug though.

Comment 26

[Uriel]

If you attach your docs would be great, and also, why not reopen, assign to
[e-mail], target it as FUTURE, and add "helpwanted" keyword, that way
there are more chances that somebody interested in working on this will find
it... no reason to close it if everybody agrees that it would be great to have
this.

Comment 27

[Axel Hecht]

Dan, is this your paper? If so, take the following:

<xsl:template match="a">
 <xsl:value-of select="@b"/>
 <xsl:value-of select="position()"/>
</xsl:template>

Note that the result tree fragment does not only depend on the context node and 
template, but also on the position of the context node in the current node set.
That is, the source tree and the select XPath expression of the prior 
apply-templates.

Apart from this, here are a few arguments why we should not have this:
- The cost a single change in the source document takes on re-transformation is
hard to evaluate.
- The cost a single change in the style document takes on re-transformation is
plain evil. (Note that I don't think that dr's paper covers this.)
- Changes to a document rarely consist of just one modification, and there is no
established infrastructure to group those.
- There is no standard to expose either the source nor the style document to 
scripts running on the result document. And the existing use cases on the net 
need the result document.

I'm all for leaving this WONTFIXed.

Comment 28

[Jens Müller (:tessarakt)]

> - Changes to a document rarely consist of
> just one modification, and there is no
> established infrastructure to group those.

Caching them and then re-transform based on multiple cached changes?

Comment 29

[Dan Rosen]

Uriel:

> no reason to close it if everybody agrees that it would be great to have
> this.

I don't believe there is anywhere near that level of agreement. In fact, I've
never once heard of anybody requesting dynamic XSLT functionality. It's very
difficult to do, and at this point, enough energy has already been devoted to
TransforMiiX, which seems to satisfy everybody's needs.

Axel:

> - The cost a single change in the source document takes on re-transformation
> is hard to evaluate.

I think this isn't impossible to evaluate. I'd still, someday, like to put
together a prototype implementation of dynamic XSLT. For fun, if you can imagine
that ... That would help to understand the computational complexity involved here.

> - The cost a single change in the style document takes on re-transformation
> is plain evil. (Note that I don't think that dr's paper covers this.)

I had thought about this. I believe it's no more evil in terms of cost than
changes in the source document... but it's certainly more difficult to program,
and it's more difficult, I think, to prove the correctness of such a program.

> - Changes to a document rarely consist of just one modification, and there is
> no established infrastructure to group those.

This is a bigger problem for us. There's no API to "freeze" and "thaw" a
document (in the same way you freeze and thaw a UI in gtk+ ... perhaps there are
better words to describe this), in such a way that DOM events can be aggregated
or queued.

A non-XSLT example of why such an API would be good is the XUL tree widget.
Before hyatt wrote outliner, the tree widget was used for the message list in
mailnews. And it was slow as molasses in Siberia: for operations on elements in
the tree, the tree had to be redrawn for each changed element.

Hopefully somebody out there is considering this problem.

> - There is no standard to expose either the source nor the style document to
> scripts running on the result document. And the existing use cases on the net
> need the result document.

That's interesting. I didn't consider that. Of course, it's not a problem that
there's no standard -- that never stopped anybody from developing new APIs --
but it's an interesting technical point.

Comment 30

[Dan Rosen]

For you nonbelievers, I just found out that somebody has actually managed to 
implement this:

  http://opera.inrialpes.fr/people/Nabil.Layaida/publi/iXSLT.pdf

This paper is published in the proceedings of WWW2002 (Honolulu, Hawaii). For 
further interesting reading, you should also have a look at "Lilac: A Two-View 
Document Editor with User-Definable Document Structure" (K. Brooks, 1991) 
and "Incremental Updates in Structured Documents" (G. Lindén, 1994) among 
others.

That's not to say anybody should actually implement this in Mozilla ("wontfix" 
seems entirely appropriate to me). This is only to say I'm not crazy :)