51666 - [trk] tasks in gecko for footprint/bloat/leak/stability work

Status: RESOLVED FIXED

(Core :: Layout, defect, P3)

Description

[rbs]

Tracking bug. Putting this list of tasks in bugzilla, a good number of
which are for Gecko 1.x ... Cc:ing those referenced in the list.


Subject: List of tasks in gecko for footprint/bloat/leak/stability work
Date: 14 Jun 2000 06:19:07 GMT
From: [e-mail] (Steve Clark)
Organization: Another Netscape Collabra Server User
To: [e-mail]
Newsgroups: netscape.public.mozilla.layout,netscape.public.mozilla.porkjockeys

Last week, I asked the engineers on the layout team for a list of task 
that would be appropriate for the "make the code behave" effort.  Here's 
what I came up with.

1. Memory Leak Bugs

We only have 9 open memory leak bugs.  I'm sure we're in much worse 
shape than that.  So a thorough examination for memory leaks is in 
order.

2. Performance Bugs

We have 50 open performance bugs. Some of these require an intimate 
knowledge of the code.  But lots of them just require in-depth analysis 
to determine what is really causing the problem.  Having done this, it 
should be easy to work with the owner of the offending code to fix the 
problem. What I hear most from folks is that they don't have the time to 
do the analysis to figure out where to start.

3. Brainstorming Ideas

Here is a long list of tasks, largely unfiltered, that folks thought 
could use a little TLC.  I've prefaced each suggestion with the 
engineer's name, so you can contact that person to get more details.  
Most of these need bugs submitted for tracking.  Thanks to the layout 
guys for taking the time to write these up!

([e-mail]) - JST and I have discussed factoring out some of the
DOM manipulation routines into a service, so that the bulk of the
implementation could be written *once*, and then (e.g.) the XUL DOM could
just delegate to those routines.

([e-mail]) - hyatt and I discussed pulling a bunch of sparse
data that is "inlined" into XUL content model elements out into per-document
tables (e.g., broadcaster-observer relationships, template builder pointer,
RDF resource pointer).

([e-mail]) - I know that evaughan and hyatt need to make a pass
over the XUL frames (like troy did to layout frames) to do space
minimization.

([e-mail]) - vidur and I discussed (a while ago) doing
heuristic atomization of HTML attribute values (XUL attributes already do
this). vidur may have already bucked up and made this happen. From looking
at CSS over the last few days (cf. sfraser's recent posting on string
distribution) it seems like there could be some easy wins like this in the
style system ("courier", "true", "sans-serif", etc.)

([e-mail] and [e-mail]) - brendan, hyatt, and I 
have been kicking around an idea to implement leaf-level form controls 
in XBL and JS, using XPConnect to directly wrap the object. This could 
potentially get rid of the DOM glue and native implementations (big code 
savings), but there are some, uh -- details, left to be worked out. 
Apparently  he and rods have been playing around with this for <select>.  
[While this is open for discussion, [e-mail] should be included 
in those discussions.  The summary of his position is while this is a 
good long- term goal, it is not worth attempting for FCS.]

([e-mail]) Bug 8065 - fix the crash in the case of infinite 
frameset/iframe recursion - this bug has been marked later but should be 
a quick fix.

([e-mail]) Bug 38180 - gdk critical warning on Linux

([e-mail]) Bug 15320 One leak-ish thing is that when 
multipart forms are posted, temp files are written out to disk and never 
deleted (a la Nav 4.x)  It's not memory leaks, but disk space leaks, 
which seems just as bad.

([e-mail]) Memory Leak Bug  37434: Loading netscape.com 
causes the document to leak ( which in turn causes the parser, dtd, sink 
etc. to leak ). This could probably be high up in the docshell/webshell 
level. Should be very interesting to track down!

([e-mail])  Input data stream being converted to unicode ( 
in the scanner ) by default.  Should it be the default?  If no, how 
efficiently can this process be delayed?

([e-mail])  Currently, when the tokenizer runs out of input 
data, i.e., if the input data hasn't been received completely (from the 
stream ), we discard the tokens that were created for that round..and 
then start ( when lot more data becomes available ) tokenizing from the 
beginning ( of the buffer ) instead of  from the point were we left.  
This can definitely be optimized.

([e-mail]) Suggestion: post purify/quantify logs on a weekly 
basis for a given set of tests.  Maybe other bloat/memory tool logs as 
well.  Keep an archive so we can track our progress over time.

([e-mail]) DLL factoring work. Duplicate code elimination  
(there are some interesting tools available to do this).  Contact Rick 
for details about what tools he has in mind.

([e-mail]) reduce modular cohesion (I'm spending time on this 
myself) (and Rick's written some pretty nifty analysis tools to help 
with this!)

([e-mail]) Perf:  The text measurement/transform code still 
does multiple transforms of the content before rendering.

([e-mail]) Perf: The parsing engine may want to be optimized 
for 7bit ascii (the common case)

([e-mail]) Perf: the style system resolution code is still too 
expensive (per quantify)

([e-mail]) Perf: Reflow command coalescing needs more tuning

([e-mail]) Perf: Push/pop clip states in rendering system are
inefficient

([e-mail]) Perf: Excess reflow in widget code (see rods test 
page with 1000's of form elements)

([e-mail]) Perf: Character type-in speend in ender is 
unimpressive. I expect it's layout doing too much reflow. (also, we 
don't do any look-ahead in the event queue to pull out pending keyboard 
events.  If we did, we could service multiple input characters with a 
single transaction/reflow.)

([e-mail]) Get rid of the mContent pointer in nsGenericElement 
(layout/base/src/nsGenericElement.{h,cpp}), mContent is a pointer to the 
outer content element but all the calls to nsGenericElement go through 
the outer content object so we know the outer content at the time of 
each method call so we can simply pass that as an argument to all the 
method calls to nsGenericElement. I did the same thing some time back 
for nsGenericDOMDataNode. The result of doing this is 4 bytes smaller 
content objects.

([e-mail]) Look into removing mRangeList from 
nsGenericDOMDataNode, this is a bit trickier to do, but I think we could 
remove the direct pointer and put a DST in the document (or some place 
like that) and access the rangelist from the DST when needed. We might 
need to, for performance reasons, dedicate one bit (a low bit in one of 
the remaining pointers maybe) that tells if there is a rangelist for a 
node, if the bit is set we get the actual list from the DST. Not sure if 
this is possible to do, if it's possible the same thing could be done 
for mRangeList in the DOMSlots used in nsGenericElement. The result of 
this would be 4 bytes smaller content and text objects

([e-mail]) Make the mDocument and mParent in 
nsGenericDOMDataNode into one shared pointer, if the node has a parent, 
access the document through the parent, if the node doesn't have a 
parent, make the pointer point to the document. One of the low bits in 
the pointer can be used to tell if it's a document or parent pointer. 
The result of this would be 4 bytes smaller text objects

([e-mail]) Make content objects access their document through 
the nodeinfo pointer, this is a bit screwy, but it's possible and it 
would save us 4 bytes per content object. If this is done we need to 
make sure the nodeinfo in a content object is always a nodeinfo from the 
right document, today this is not true but fixing that should be quite 
easy.

([e-mail]) Remove mNameSpace (and all users of it) from 
nsGenericXMLElement and from nsXULElement. The nsINameSpace is IMO only 
needed when the document is constructed (ie in the content sink) so 
there's no need to store a pointer to it in the content objects. 
nsINameSpace might still be used in some places but I think that's 
wrong, those places should be fixed.

([e-mail]) Look into removing mCapturer from nsDOMSlots in 
nsGenericElement, currently it's not even used but it looks like it was 
added by [e-mail] (ie check with him before removing it) for 
some more optimized event capturing...

([e-mail]) I've handed off several perf tasks to waterson 
already for block/inline reflow.  I'm sure he could use some help.

([e-mail]) Bloat: Reflow peak memory usage.  We use way too 
much memory during a reflow.  There are oppurtunities to shrink the 
various reflow data structures even further.  More importantly, we need 
to examine those data structures and see what data is really needed, and 
what could be cheaply computed as needed instead.

([e-mail]) Perf: Reflow recalculates a ton of information 
every time through.  Some of that information could easily be stored 
between reflows.  The issues here are: a) identifying the data that is 
useful to keep cached b) identifying the cases that need to cause a 
flush of this cached data (incremental reflow is a good trigger, for 
example) c) design a tuning knob where we only cache up to a certain 
amount of information, so we don't introduce unreasonable additional 
bloat.

Comment 1

[buster]

all P3 bugs are Future, will not make RTM of NS6.

Comment 2

[Kevin McCluskey (gone)]

Build reassigning Buster's bugs to Marc.

Comment 3

[Håkan Waara]

This bug being almost 2 years old, I think it's safe to close it down. Or is it
still useful? jst, waterson?

Comment 4

[rbs]

(It would be instructive to see what has been resolved and the overall progress
that has been made ever since the compilation of that list.)

Cc:ing [e-mail] who has been compiling another similarly
challenging list.

Comment 5

[Juan José Mata]

Do we want Alec to take over this as part of his footprint work?  Alec?

Comment 6

[Michael Buckland]

Bulk adding topembed keyword.  Gecko/embedding needed.

Comment 7

[saari (gone)]

meta, removing topembed (children tasks may be topembed)

Comment 8

[Alec Flett]

I'm starting to go through this list and do "pre-triage", to pick out the bugs
that have yet to be fixed, what needs to be investigated, etc. I'm going to take
this bug but eventually I'd like to close this as FIXED or something, and file a
new bug as a tracking bug for the still-open issues.

Starting with harishd, rickg, and jst's suggestions.

([e-mail])  Input data stream being converted to unicode ( 
in the scanner ) by default.

-> need a bug on this

([e-mail])  Currently, when the tokenizer runs out of input 
data, i.e., if the input data hasn't been received completely (from the 
stream )

-> I think this was fixed with sliding strings - need to talk to harishd about this

([e-mail]) Suggestion: post purify/quantify logs on a weekly 
basis for a given set of tests. 

-> not a bug

([e-mail]) DLL factoring work.

-> I do this on a constant basis. not a real bug.

([e-mail]) reduce modular cohesion (I'm spending time on this 
myself) 

([e-mail]) Perf:  The text measurement/transform code still 
does multiple transforms of the content before rendering.

-> no idea. need to ask around

([e-mail]) Perf: The parsing engine may want to be optimized 
for 7bit ascii (the common case)

-> need to talk to jkeiser and harishd about this one.. there has been talk
before about how the parser expands everything to unicode by default, and that
this is kind of icky.

([e-mail]) Perf: the style system resolution code is still too 
expensive (per quantify)

-> this has been rewritten at least once since this bug was posted, so ignore this

([e-mail]) Perf: Reflow command coalescing needs more tuning

-> too generic, ignore this one

([e-mail]) Perf: Push/pop clip states in rendering system are
inefficient

-> need more investigation, probably an old issue. sounds like the view manager?
not sure.

([e-mail]) Perf: Excess reflow in widget code 

-> pretty sure this is "fixed"

([e-mail]) Perf: Character type-in speend in ender is 
unimpressive. 

-> generic editor problem. lets not consider it as part of this bug

([e-mail]) Get rid of the mContent pointer in nsGenericElement 
(layout/base/src/nsGenericElement.{h,cpp}),

-> looks like this is done

([e-mail]) Look into removing mRangeList from 
nsGenericDOMDataNode, 

-> done

([e-mail]) Make the mDocument and mParent in 
nsGenericDOMDataNode into one shared pointer, 

-> need some minor investigation, looks like mPArent is now mParentPtrBits and
already stores 2 bits of data, it will be hard to merge this into mDocument if
the bits are used up

([e-mail]) Remove mNameSpace (and all users of it) from 
nsGenericXMLElement and from nsXULElement. 

-> done

([e-mail]) Look into removing mCapturer from nsDOMSlots in 
nsGenericElement,

-> done

([e-mail]) Bloat: Reflow peak memory usage.  We use way too 
much memory during a reflow.  There are oppurtunities to shrink the 
various reflow data structures even further.  

-> need to investigate reflow structures

([e-mail]) Perf: Reflow recalculates a ton of information 
every time through.  Some of that information could easily be stored 
between reflows. 

-> a huge task, need to do investigation and talk with other layout people

Comment 9

[rbs]

> ([e-mail]) Perf:  The text measurement/transform code still
> does multiple transforms of the content before rendering.
> 
> -> no idea. need to ask around

A possible can of worms. Some related comments:
http://groups.google.com/groups?selm=38534BBF.1D64DBDA%40netscape.com
http://groups.google.com/groups?selm=3853E9BF.FD601CF%40netscape.com
 
> ([e-mail]) Perf: the style system resolution code is still too
> expensive (per quantify)
> 
> -> this has been rewritten at least once since this bug was posted, so ignore
> this

Done: bug 78695.

> ([e-mail]) Perf: Reflow command coalescing needs more tuning
> 
> -> too generic, ignore this one

Done: bug 129115.

> ([e-mail]) Perf: Character type-in speend in ender is
> unimpressive.
> 
> -> generic editor problem. lets not consider it as part of this bug

Might be bug 75511 which seems the root cause of other bugs such as 
bug 83205.

Comment 10

[Markus Hübner]

Recommending to consider bug 157681, bug 67752, bug 123191, bug 149216, bug 
170330

Comment 11

[louis bennett]

all of this bug's dependencies have been resolved.

hence -> FIXED (per comment 8)