Closed Bug 184746 (dbaron) Opened 17 years ago Closed 16 years ago

dbaron's layout architecture tasklist

Categories

(Core :: Layout, defect)

defect
Not set

Tracking

()

VERIFIED INVALID

People

(Reporter: dbaron, Assigned: dbaron)

References

(Depends on 1 open bug)

Details

(Keywords: arch, meta)

This is intended to be a meta-bug for major architectural changes that I think
are needed in layout.  I intend to file dependent bugs later, with more detailed
descriptions.  (Some of these bullet points are clearer than others, and I could
expand almost all of them to at least a few paragraphs.)  However, for now, I'll
just paste the list of thoughts that I have (which I've been adding to for a few
weeks now since I originally wrote it) into the bug.

Some of these things are cleanups that need to happen for our sanity, and some
of them are more along the lines of prerequisites for implementing new concepts
from CSS3 or other sources.

Prerequisites:
 * Rewrite block-within-inline handling ("{ib}") to happen during
   reflow. [ for FC and reflow cleanups ] (bug 142585)
 * Switch to XBL form controls [ for reflow cleanups ]
 * Finish moving form control data out of layout (text controls!).

Clean up central objects:
 * Design the loading process to look like the way it should work.
   (e.g., waiting for stylesheets should delay frame construction, image
   loads should start from content / stylesheets, old page should go
   away when new one has something to display) (bug 84582 / etc. / etc.)
   + Global object proxying to allow loading documents to be behind the
     scenes.
   + Eliminate timers all over the place and replace with a
     well-thought-out system.
 * Eliminate redundancy (e.g., pres shell / pres context).
 * Per-medium singletons, allowing sharing/prototypes/fastload of
   some style/layout objects? (bug 117568)

Clean up style system:
 * Less bloaty CSS stylesheet backend (code size and object size)
   (bug 125246 / etc.)
 * Solve the dynamic reresolution problems (ReResolveStyleContext)
   (style tree must be isomorphic to content tree, not rendering tree)
 * Clean up user/UA stylesheets (bug 179006)

Clean up frame construction / frame tree:
 * Make everything use 'display' -- no construction by tag.  Use things
   like -moz-listbox instead.
 * Implement in terms of 'display-role' / 'display-model'.
   + to allow code consolidation and refactoring (splitting of monsters
     like nsBlockFrame) in reflow code
   + Make marginally extensible (arrays of factory functions, registered
     and placed in hashtable?, perhaps with more of the initialization
     work in the frame classes rather than in the constructon code).

Clean up reflow:
 * Separate intrinsic width (maximum width) and minimum width (max
   element size) calculations from reflow (See bugs marked
   [reflow-refactor] in SW, although with numerous caveats relating to
   the way setting 'width' on a block still allows its descendants to
   influence table sizing)
 * Implement in terms of 'display-role' / 'display-model'. (see FC above)
 * Make the recursion of reflow move along the content tree rather than the
   frame tree (http://bugzilla.mozilla.org/show_bug.cgi?id=172031#c16).
   (Should we still use real recursion to do reflow? Do we want
   interruptable reflow?)
   [ This may depend on CSS WG decisions about 'overflow'. ]

Clean up rendering:
 * Fix units/scaling mess. (bug 177805 / bug 153080 / bug 63336)
 * Allow anything to have a view, and make it not break correct layering
   (e.g., for floats).

Other:
 * Fix the event state manager not to use frames (bug 130620, etc.)
I have big reservations about the currently proposed split for the display-role/ 
display-model properties (as I mentioned in Mandelieu). I think we have a great
opportunity here to study the problem and give feedback to the working group
about what a better, more logical, and less redundant split might be.
> Make the recursion of reflow move along the content tree rather than the
> frame tree (http://bugzilla.mozilla.org/show_bug.cgi?id=172031#c16).

I doubt this is a good idea although I could be convinced.

> (Should we still use real recursion to do reflow? Do we want
> interruptable reflow?)

Yes and yes. We can and should leverge our incremental reflow architecture to
get interruptible reflow. Building some sort of explicit reflow stack will not
make interruptible reflow easier IMHO.

> Allow anything to have a view, and make it not break correct layering
> (e.g., for floats).

We're pretty close to this already. I don't know exactly what the float layering
issues are but I do know that every float should have a view and I'm confident
any layering issues can be resolved in the view manager without a vast amount of
work.
Some additional ideas from bzbarsky:

[19:31:45] <dbaron> bz_gone: I want to know what your list of things to fix is.
[19:32:21] <bz_gone> dbaron: hmm... at the top of my list right now are sane
apis for notification of changes to CSS (sheets and rules)
[19:32:35] <bz_gone> dbaron: batching of sheet changes (eg for alternate switches)
[19:32:46] <bz_gone> dbaron: async loading of sheets
[19:33:06] <bz_gone> dbaron: the fact that frame construction is unreadable and
fragile
[19:33:43] <bz_gone> dbaron: Image loading from content and style system instead
of layout
[19:33:45] <dbaron> bz_gone: In the grand scheme of things, I don't find frame
construction code very fragile, actually...
[19:34:01] <bz_gone> dbaron: it's mostly fragile due to code duplication....
[19:34:12] <bz_gone> dbaron: so you have to make the same change in multiple
places if you change things
[19:34:27] <dbaron> bz_gone: ok, sure.
[19:34:35] <bz_gone> dbaron: then there's the whole mess with the MIME service,
stream converters, etc
[19:34:53] <bz_gone> dbaron: I have a bunch of problems with that
[19:35:16] <bz_gone> dbaron: ranging from our lack of support for args to helper
apps
[19:35:38] <bz_gone> dbaron: to arch issues with detecting .ps.gz files from
file:// and ftp:// as postscript
[19:36:24] <bz_gone> dbaron: and I'm _still_ hoping to end up with one codepath
for saving files instead of the current 3....
[19:36:48] <bz_gone> dbaron: that's that major arch stuff, I think....
[19:37:03] <bz_gone> dbaron: oh, and nsCSSDeclaration.... ;)
[19:37:20] <bz_gone> dbaron: though that's more on your hit list
[19:37:27] <dbaron> bz_gone: mind if I paste this into my bug?
[19:37:42] <bz_gone> dbaron: not at all.
[19:38:26] <bz_gone> dbaron: oh, and async loading of sheets means
ReResolveStyleContext needs to be as good as possible for those cases when we
_do_ start constructing frames before the sheet loads...
[19:39:24] <dbaron> bz_gone: in most cases we should delay frame construction
[19:40:01] <bz_gone> right
[19:40:15] <bz_gone> but in some cases we'll still end up doing it... (if the
sheet is very slow)
[19:40:32] <bz_gone> oh, and on the topic of stylesheet notifications,
nsIDocumentObserver in general is weird
[19:40:47] <bz_gone> eg the content sink has to know about frame construction in
the current scheme...
[19:41:20] <bz_gone> so making nsIDocumentObserver be a little more useful is on
the list too
Getting gecko to properly support pagination (page-break-inside, etc) is a
prerequisite to properly supporting the 'projection' media. It would be nice to
also support multiple page widths, which I understand we have zero support for
right now. jkeiser said some work was already progressing in this direction?
Alias: dbaron
See bug 15608 comment 26 (bzbarsky) on dynamic changes and subtrees.
PresShell / PresContext combining is bug 154199.

As to moving data out of text controls, that can only be done by sacrificing
performance or making huge changes in the plaintext editor.  Specifically, it
would have to simply operate on the text chunk in the textarea, and the textarea
frame would have to directly display that text chunk with -moz-pre-wrap.  This
would actually improve performance probably, since there's much less
construction / tear-down cost.

I have one more item:
* take the splitting stuff out of layout, or modularize it at least
This is the source of huge and increasing complexity in layout.  A new printing
architecture currently under discussion (use clipping during paint instead of
splitting during layout to decide what parts of an object show up on a page)
could get rid of this.  This will cause some harm the CSS3 multiple-page thing
Hixie pointed out, though that would be a huge deal to implement anyhow
(multiple sizes for blocks??).  I'd appreciate if we could wrangle that out in
the appropriate bug rather than here, though.

One of my favorites on this list is:
> * Make everything use 'display' -- no construction by tag.  Use things like
>   -moz-listbox instead.
My opinion is fairly similar to your next points.  If we are going the route of
display construction only, I'd say:
* move all special frame construction logic out of nsCSSFrameConstructor and
into singleton display-type-constructor objects
* allow these display-type-constructor objects to be registered with
nsCSSFrameConstructor as a map from display type to constructor object, for
extensibility reasons (mathml and svg anyone?).

I have one more idea to throw out here, a bit more radical.  It stems from the
brainstorm, "what would happen if all our layout objects were the same object--a
'tile' containing x/y/width/height and a few other items of layout state?"  We
move the reflow and painting algorithms out into singleton objects--one per
display type.  Reflow becomes a matter of destroying and recreating, or possibly
just modifying, the tile associated with the content.  This also makes it easier
to plug new display types in, and makes things fairly fast (no virtuality
necessary).  In order to do this, we would need to first do two things which are
both good on their own:
  * move style contexts to content (or at least keep them around when their
frame goes away and make them accessible when there *is* no frame)
  * move *all* state that needs to persist longer than the frames, out of frames
and into content (or somewhere else besides frames, at least).  dbaron has a few
of these specifically in the top of this bug (form controls specifically)
Depends on: 188803
> "what would happen if all our layout objects were the same object--a
> 'tile' containing x/y/width/height and a few other items of layout state?"

I don't really understand how this is different from what we have today, except
that it would require an extra pointer to the frame-type-specific object.
> what would happen if all our layout objects were the same object--a
> 'tile' containing x/y/width/height and a few other items of layout state?

I found out yesterday that this is exactly what XUL does.  Each box holds a
pointer to a BoxLayout object (there are a few different layout objects, eg
SprocketLayout, GridLayout, etc) and asks it to handle the layout when needed. 
The win is that a single type of box object can be used for many different
purposes, since all the reflow logic is encapsulated inthe layout object (which
is a stateless singleton that just handles the reflow).
Note bug 139912, which the display frame construction stuff would certainly fix.
Depends on: 194100
An additional thought: nsObjectFrame could be redesigned so that it delegates
*everything* to another frame -- either (for alternate contents) a frame
construted for the object by display type or a frame (nsImageFrame,
nsFrameFrame) for the object.
We need to simplify the way we handle incremental reflows -- reflow commands
could probably be eliminated in favor of dirty bits and a parameter that says
"blow everything away since an ancestor was dirty".  (This is certainly the way
I'd plan to handle min-width and max-width calculation.)
Alias: dbaron
No longer depends on: 39965, 188803, 194100
Keywords: arch, meta
Summary: dbaron's layout architecture tasklist
That's odd.  kairo@kairo.at: Your last change wiped out BugsThisDependsOn,     
        Alias,Keywords,Summary(!)
restoring bug fields after kairo@kairo.at's strange commit
Alias: dbaron
Depends on: 39965, 188803, 194100
Keywords: arch, meta
Summary: dbaron's layout architecture tasklist
A few more thoughts on incremental reflows (I think I wrote something like this
in another bug, but maybe not):

There are only a few cases for which we need to optimize:
 1. incremental loading of pages
   1a. specified style changes to very small parts of pages
 2. resizing of the window
 3. changes to 'top', 'left', 'right', and 'bottom'

Pretty much everything else doesn't matter.  In particular, any changes to the
specified style for an element should mean *everything* inside that element gets
recomputed.

The optimization to make in (2) is not to recalculate intrinsic widths (min/max
widths), which would be simpler to do in a world where min/max width computation
were separate from reflow.  (See bugs marked with [reflow-refactor] in the
status whiteboard for other things that such a refactoring would fix.)

It would be nice if the optimization we'd need to make for (3) meant we could
just move things and recompute overflow areas.  I think this was the original
intent of those properties.  I also thought this when I commented in bug 157681.
 However, I'm afraid that's not actually the case, and the rules in the current
CSS2.1 draft are correct as far as WinIE compatibility goes.

The optimizations necessary for (1) are a bit more complicated.  I listed (1a)
as a subcategory of (1) because in our current implementation all the
(relatively minimal) optimizations needed for (1a) follow from those necessary
for (1).  In other words, the optimizations necessary for making incremental
page loading work reasonably mean that changes to arbitrary style information
for small parts of the page don't take time that scales with the size of the
page (in any significant way, at least -- perhaps some tiny component).  I
suspect that this would be true in any good design to handle the things
necessary for (1).

The optimizations necessary for (1) should maintain the invariants that:
 A. The page always ends up displayed the same way, no matter what the units of
incremental layout were during its load
 B. The result after any incremental load of part of a page should be the same
as if that part were the whole page (excluding unloaded images, etc.)
Otherwise we confuse authors and encourage hacks to prevent incremental reflow.

Incremental loading of content requires recomputation of preferred widths and of
sizes for almost any content that has descendants modified.  More on this later,
perhaps...
How about

4. Inserting and deleting content during editing operations

?
Right, except it's likely to be 1b.
Another thing that would be nice, that isn't really an architecture issue, but
more an implementation issue, is to not let the reflow-code hold strong
references to nsIContent objects. Once bug 215981 is fixed it is possible to
traverse an entire tree without ever calling addref or release on a node.

Of course this is only possible as long as you know that the tree won't change
during the traversal, but that shouldn't be possible to happen during reflow, right?
This is already happening in bug 190735. For example, almost all calls to
nsIFrame::Getcontent() no longer add a reference.
I ask the contributors here to add some further thoughts for the users of
your works when considering future changes to the layout engine. I speak as
a technical communicator who must translate your decisions into conceptual
material that is at minimum comprehensible, but that ideally is also
conceptually clean. I'm speaking from the experience of constructing a book
on Mozilla; gratuitous adverts for that book may be found elsewhere.

In particular I'm speaking for non-embedded(XPFE) learner programmers, who
occassionally are exposed to: frame and frame-like artifacts of the layout
engine, artifacts related to the Model-View-Controller pattern,
and the extensions to that pattern (builders, renderers, etc) that are
required for a full implementation. It is my job, or part therof, to
make a serious attempt at easing the burden of learning these concepts.
One way to do that is to encourage you to make those concepts non-burdensome
in the first place.

In the current design (and I accept that exploiters of the layout engine may
be more responsible than designers of that engine), some aspects of layout
are quite confusing to beginners/learners. One sentence of truisms: no
programmer successfully enters the Mozilla diaspora without learning;
and Mozilla technology benefits the most from well-informed programmers.

The first confusing aspect is that of (my terminology) extrinsic
versus intrinsic layout elements. The property pattern used to specify
layout properties (in CSS) is a flat hierarchy, and it is non-obvious
which of these properties are of interest to XUL/CSS/JS programmers, and
which are implementation details handily exposed. A big ask to you all is
to (A) somehow taxonomically group exposed parts of the layout engine
into "those bits XPFE programmers might conceivably use" and "those bits
that should stay buried".

For learners, we once had some grouping of this nature with pseudo-classes,
which look at intrinsic aspects of the layout (eg :first-letter digs into
a block/box constructed from more than one layout primitive), but various -moz
enhancements now also grant access to intrinsic-like aspects of the layout,
so the line is blurred. Could there be some grouping of those intrinsic
elements?

On top of all that, learners need an entry-level concept to cling to, and
that concept is currently the "frame". So another big ask is to (B) constrain
your re-architecting so that the concept of frame retains some approximately
correct meaning for learners.

Regardless of whether you believe that frames/MVC are discrete concepts or
not, and regardless of whether they are discrete from the nitty-gritty of
implementation, the pragmatic fact is that they are early rocks that
learners will cling to when attempting to access the technology. Anything
that gives that early clinging action validity in future problem solving
is good.

For example, should a frame become factored into two or more pieces, those
pieces taken as a lump and used crudely should still have some currency as
a frame (or as some new noun, but frame is the lingua franca and I vote
it is retained). Once programmers gain experience they can go from
confident/part-right to confident/accurate.  Without a frame concept, they
must go from uncertain/unknown to confident/accurate directly. That latter
transition is common for gifted thinkers, but the former is easier for
the majority. Please retain the concept of a frame, even if it is
retained in a somewhat abstract form.

A second confusing aspect is that of value-added uses of the layout
(kick me if I'm OT). In particular, layout builders, views and controllers
are exposed to XPFE programmers very patchily. It is not obvious that
every significant XML tag (say) has MVC status, since only <menupopup>,
<listbox>, <tree> and <template> expose anything to the DOM/XPCOM of the
internals, and only <tree> makes an attempt at exposing a reasonable part
of the pipework.

No one is saying the current layout engine has shortcomings, it is
just that this limited exposure of MVC plumbing poses significant problems
to learners.  It means that the big, ugly tags (<tree>, <template>) are
loaded with learning hurdles separate and in addition to their base
conceptual use. Trees are confidence defeating, because they are conflated
with MVC concepts not yet seen elsewhere in the XUL widget (or frame) set.

No one would advocate padding out the layout engine so that the M,V,C,builder,
renderer, etc of each tag is exposed via the DOM/XPCOM. For
consistency, however, some uniformity is required if the functionality
of the design is to make rational sense. A big ask to you all is to (C) make
some rules about this exposure that will apply to ALL DOM/XPCOMed tags, and
then reduce away most of the complexity everywhere except where necessary.
If that means contructing layout APIs that encourage uniform design for
specific XML rendering systems (HTML, XUL), then so be it.

The first corollory to this last point is that there should be a demo
frame (and suitable tag) that learners can use as a test-bed before they
attack the <tree> tag or templates. That frame should be exercisable so
that the programmer can control the layout *process* in code, thus revealing
the dynamics of the engine, and preparing the way for more complex uses.
Such a beginner's sample is common to most development environments, and
in Mozilla would go a long way to helping the user understand MVC, renderers
and builders. That is big ask (D). You may well point me towards <iframe>
or similar, but of course, the layout process is conflated with document
management concepts in that case. <iframe> is not a simple learning testbed.

The second corollary is that of interface-object relationships. To an XPFE JS
programmer, it is not currently clear (or at least not as clear as it might be)
what the relationships are between various concepts. It is a quagmire for the
learner. On the one hand, there is the theory of MVC. This theory does not
map 1-1 (or neatly) to the architectural concepts of builders, renderers
or (sigh, [command] controllers). Then there is the matter of underlying,
possily XPCOM'ed, objects. Neither theory or architecture map neatly to
those objects. Just listen to anyone frothing at the mouth trying
to differentiate the content builder from the template or tree builder,
or trying to understand why a tree-and-template combination uses the
tree builder not the template builder, when a box-and-template combination
uses the template builder (from memory, which just shows my point).

Again, it is unfair to land this problem strictly on layout, but layout
provides the primitives that these higher systems (except for the
controllers, sigh) are at least partially built out of.

Great as it currently is, "controller" as jargon should be bought back by
layout. The Command pattern underlying the controllers in the DOM should
provide an alternate property name that better reflects command jargon
(eg "commandFacades"). If MVC is to be a useful crutch for learners, then
controllers should appear in the layout jargon/implementation. That is big
ask (E).

Here is a simple example of the cognitive disharmony that can result
from mis-matched theory vs implementation. Look at the filesystem directory
XPCOM objects, which I hope you are unfamiliar with. To explain this
little system, one must explain the theory (providers are registered with
queried directory services), and then match it to the technology
(@mozilla.org/file/directory_service;1 and friends). All is well, until,
dismayingly, this XPCOM object also contains a provider, one
that is used directly and not via a service it is to be registerd with. This
exception is digestable once pointed out but not easily discoverable.  It is
also at the expense of the initially touted model, which is now either useless
or distractingly special-cased, depending on your point of view.

Alas, the layout system combined with the various semi-exposed objects is
far more special-cased than the filesystem services.

In summary, this note is a request that you appreciate that one measure
of a successful re-architecture is the maximum ply of the implied
learning path.
Laying out a tag like <DIV> or <box> is a one-ply (one step) learning
path, well supported by Mozilla. <UL> is a two-ply learning path, because
both <UL> and <OL> must be comprehended before virtuosity is achieved.
The more complex parts of the layout process: lists, trees, and templates,
have very high plys currently, and are obstacles to the gaining of that
sense of virtuosity. And yet trees and templates are almost mandatory tech
for XPFE folk.

No doubt the layout engine should be able to silently and seamlessly solve
the display of a given XML document using nothing but content and CSS hints,
but in practical terms programmer will interact with it more than that.

My biggest ask, therefore, is (E) that XPFE (application, JS) programmers be
put abreast of embedded C/C++ programmers. In the long term they will be more
numerous and on average less gifted than the embedders, since most software
is application software.  Perhaps they will never be as numerous as Web
developers, but they are more likely to aid this community.  Please add
their learning needs to your considerations.

regards, Nigel.
As far as I can tell, comment 19 is completely off-topic for this bug and now
makes it so much harder to read (since it's so tall) that I'm considering
refiling this entire bug as a separate bug.
And I don't see how any of the decisions in this bug relate to how users of the
XUL toolkit understand it.  I think most of comment 19 is about other parts of
Mozilla.
This is the best existing bug with keyword=arch for this feedback.
My remarks bear directly on architectural issues. Comment 8 shows that
some consideration of related matters (XUL) is germaine. I have followed
that lead . The re-design of the layout engine might assist
in relieving these matters.

I'll post comment 19 to n.p.m.layout and n.p.m,xpfe , in case it
is too cross-module in scope for here. But it remains the case that
changes to layout will likely impact these matters one way or the other.
Being advised in advance is only sensible.
More thoughts in my "cleaning up layout" post on mozilla-layout a few months ago:
http://groups.google.com/groups?selm=20030917230123.GA4593%40darby.dbaron.org
I've run across this bug more than once.

Is there a timetable (granted nothing follows a timetable very well) for
implementing all these changes?

Is it worth waiting on bugs like bug 217527, hoping this bug takes care of it. 
Or assume this bug is years away?

From what I understand (not to much of the fancyspeak), it looks like a massive
undertaking, but will be a great improvement to Gecko.
Using bugzilla for things like this isn't going to work.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → INVALID
Verifying as seeming-unworkable-with-bugzilla.  Perhaps some future version of
bugzilla will allow read-only access to everyone except a group maintained by
the bug reporter.
Status: RESOLVED → VERIFIED
Another wacky idea (since I still don't have another place for this):

Perhaps we could do line breaking at frame construction time -- maintain the
invariant that a text frame can never have a break in the middle of it.  Doing
this would reduce the number of data structures needed.  The text frame would
still need to store whether it was breakable at the ends, whether the last
character(s) should be trimmed when breaking (for whitespace or &shy;), perhaps
the width of the trimmed characters, etc.  However, it would allow what I
described in the post in comment 23 without introducing (multiple) additional
levels of data structures.
That sounds like a pretty large expansion in footprint. Given that we already
have this complex, but efficient, setup, I'd rather try to make that as rational
as possible.
You need to log in before you can comment on or make changes to this bug.