Large/long pages won't load

VERIFIED DUPLICATE of bug 29641

Status

()

Core
Layout
P3
normal
VERIFIED DUPLICATE of bug 29641
19 years ago
18 years ago

People

(Reporter: Colin Blake, Assigned: Nisheeth Ranjan)

Tracking

({arch})

Trunk
Future
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [nsbeta2-] [nsbeta3-])

(Reporter)

Description

19 years ago
I have a test file that is 100,000 lines of

This is line 1 of 100000<br>
This is line 2 of 100000<br>
etc

Its loads fine using Nav 4.6 on Windows (takes about 40 seconds).

On Mozilla Linux (M12), it loads in about 200 seconds, but when you scroll to
the bottom of the window there's only about 30,000 lines present.

On Mozilla OpenVMS (M12), it loads in about 140 seconds, but when I try to
scroll to the bottom of the windows it stack dumps on me (its dying in GTK).

Haven't tried it on Windows.

I can send you the HTML file if you wish, but you can probably make your own
just as quick.

Updated

19 years ago
Assignee: leger → rickg
Component: Browser-General → Layout
QA Contact: leger → gerardok

Comment 1

19 years ago
Moving over to layout...

Updated

19 years ago
Assignee: rickg → nisheeth

Comment 2

19 years ago
Nisheeth -- can you please take a look.
(Assignee)

Comment 3

19 years ago
Setting milestone to M13 and accepting bug...
(Assignee)

Updated

19 years ago
Status: NEW → ASSIGNED
(Assignee)

Updated

19 years ago
Target Milestone: M13 → M14
(Assignee)

Comment 4

19 years ago
Moving the last two M13 bugs to M14 because they aren't M13 show stoppers.
(Assignee)

Comment 5

19 years ago
I tried this on windows 98 and it hung up the computer.  I had to reboot.  Does 
this qualify as a beta blocker?  Marking it beta1 so that the PDT team can 
decide...

CCing troy and vidur...
Keywords: beta1
(Assignee)

Comment 6

19 years ago
I tried this on NT and the mem usage went up to around 123 MB and the cpu usage 
was 100%.  The browser took close to 200 seconds to display the page.  During 
those 200 seconds, the UI was frozen.  Navigator 4.7 loads up the same page from 
disk in around 10 seconds.  We definitely need to become more incremental, 
responsive and fast for large pages like these.
(Assignee)

Comment 7

19 years ago
Attaching the test case...

Comment 8

19 years ago
The lines in the example vary from about 24 characters per line, to roughly 30
characters per line. With 100,000 characters, the file size is roughly 3 megs.

Due to either overhead, or fragmentation, or both, Nisheeth comments that we use
123MB.  IF we are generous, and assume a footprint of 23 before the load, then
roughly 100MB are taken by this 3 meg file. This means roughly a factor of 30
overhead.  (or maybe fragmentation cost has pushed us a bit higher).

First off: on Win 95, with restricted virtual memory, we'd surely hit a wall in
a Bad Way.  At best we could hope for a graceful termination.  Worst case, the
OS may come to its knees (permanently) before we get an out of memory condition :-(

The fact that scrolling induces a stack dump would suggest (to me) that the
overhead rises as display lines are (probably lazily) constructed (and hopefully
destroyed).  This is sad.. but believable.

Can this be fixed by beta1/M14 freeze (2 weeks)?  I'd say no way, don't even
dream about it.  I believe that a lot of the modularity and flexibility that we
bought/installed came at the (current) expense of objects with overhead for
small units of data.  We might help this a little (factor of 2 :-/ ), but then a
200,000 line file will kill us.

We need a spec for a "reasonable" size file.  We should try not to be subject to
attacks courtesy of a "large" file.  We could even say (at the front end, while
loading html files), that "some size" is too darn large.  That we could do for
beta... but it would probably be worse for the product than helpful.  Maybe we
can track memory used for a single page of display... or even total memory that
we're willing to allocate for the app... and be restrictive there.

Of the many bugs I've seen for beta that will impact many customers... I'm not
convinced that this should be a "Beta stopper" (re: beta1 candidate).  PDT will
make the final decision... but I personally wouldn't hold up a beta ship for
this bug.

Comment 9

19 years ago
Putting on PDT- radar for beta1.  Won't hold beta (would release note) for beta.
Whiteboard: [PDT-]
(Assignee)

Comment 10

19 years ago
Moving to M15 because this has been marked PDT-
Target Milestone: M14 → M15

Comment 11

19 years ago
Looks like a dup of 25509.

I think this is a really important bug to fix. My time spent with Quantify in 
25509 suggests that the layout algorithm is O(n^2) for the number of lines. 
Consequently, if this bug is affecting large files (rendering them impossible to 
lay out), then this bug is actually affecting _all_ layout performance. Could 
explain why we look so bad relative to 4.x performance. 

I suggesting making this the highest priority M14 bug, PDT+. Performance is 
presumably the second highest priority next to the ship date.

Comment 12

19 years ago
Warren, please don't make blanket statements like "the layout algorithm is 
O(n^2) for the number of lines". It's a meaningless and misleading statement. 
Maybe Kipp has a specific algorithm in this case that is not optimal, but that 
certainly doesn't generalize to all of layout...

It's really frustrating to always have everyone assume that every peformamnce 
issue is because of layout...

Comment 13

19 years ago
Two things worth noting:
1. this test case is not representative of normal Web pages, because all of the 
content is directly a child of the BODY tag. Typically there are child elements 
like P tags or DIV tags that contain a majority of the content

2. there is a known issue of how the block code computes text runs and numbers 
counters in a very non-incremental way. This is the code from 
nsBlockFrame::Reflow()

  if (eReflowReason_Resize != aReflowState.reason) {
    RenumberLists(aPresContext);
    ComputeTextRuns(aPresContext);
  }

That's especially a problem in cases like this where all of the running matter 
is a child of the BODY

Comment 14

19 years ago
Troy: Let me revise my statement. Layout is O(n^2) for laying out text files, 
and probably for any documents that call nsTextNode::AppendData.

Comment 15

19 years ago
Stop saying "Layout is O(n^2)". It may be true that "the block frame code has a 
O(n^2) algorithm in its reflow code when laying out text", but it certainly is 
not true that all of the layout algorithms in all of the frame classes are 
O(n^2). And it certainly isn't true that the basic reflow process is O(n^2)...

Comment 16

19 years ago
nsTextNode::AppendData is a potential problem because of the way it's used by 
the content sink - the content sink appends to a text node for every netlib 
chunk of data within that node. Kipp and I spent time discussing this and my 
impression was that he cut down the redundancy in subequent layout. 
Specifically, we are no longer laying out the entire text run for each append.  
We still do a realloc() (and hence a copy) in nsTextNode::AppendData() that is 
O(n*m) where n is the number of bytes in the node and m is the number of netlib 
chunks the data spans.

I suppose we could restrict the size of text nodes in the content sink. This 
would not be correct accoding to the DOM spec, but I could live with it for now.

Comment 17

19 years ago
N bytes times m network chunks would certainly explain it. My cpu pegs seemingly 
indefinitely for the tinderbox logs.

Can we agree that this needs to be fixed for M14?

Comment 18

19 years ago
Let me take that back. Because of the <BR>s in the text case, there should be no 
long text nodes and, hence, no calls to nsTextNode::AppendData. Did you really 
see that method on in your Quantify data?

Comment 19

19 years ago
I managed to get Quantify to give me the data one time. I think the data for 
this run is growing too large and overflowing what my machine can handle. I did 
see nsTextNode::AppendData as significant though after factoring out all the 
wait and i/o time.

You can see what happens to your cpu by bringing up the task manager while 
loading a tinderbox log. Just break anytime after the cpu pegs. You'll usually 
have that call in the stack.
(Reporter)

Comment 20

19 years ago
I don't know if someone fixed this, but in my M14 code I can now load really big 
pages. Its not fast, but I don't expect it to be.

Comment 21

19 years ago
Nisheeth improved things in nsTextNode::AppendData a while back, but they're 
not totally fixed yet.
(Assignee)

Comment 22

19 years ago
Actually, it was Vidur who did that!  But, hey, I'll take the credit for it... 
:-)

Updated

19 years ago
Summary: Large pages won't load → Large/long pages won't load
*** Bug 30650 has been marked as a duplicate of this bug. ***
(Assignee)

Comment 25

19 years ago
The performance on large pages is much better now.  Is there consensus among us 
to mark this bug fixed?

Comment 26

18 years ago
I've gone to this page with the latest build (on linux) and it never
finishes.  I have to kill it.. It does update the title, but never repaints
anthing.  It's a 2mb file. netscape 4.72 takes about 16 seconds to render it
with my 128mb pentium 233mhz

 http://www.informix.com/answers/english/ierrors.htm
(Assignee)

Comment 27

18 years ago
Moving bugs out by one milestone...
Target Milestone: M15 → M16
(Assignee)

Comment 28

18 years ago
Scheduling for M17...
Target Milestone: M16 → M17

Comment 29

18 years ago
*** Bug 37409 has been marked as a duplicate of this bug. ***
*** Bug 36285 has been marked as a duplicate of this bug. ***

Comment 31

18 years ago
*** Bug 37824 has been marked as a duplicate of this bug. ***

Comment 32

18 years ago
can't load this one at all (1.9 mb - only text)
http://bugs.gnome.org/db/pa/lgnome-core.html
It stops rendering the page pretty quick, and after having downloaded 95% of it
it freezes. Tried several times, deleted cache and all - no go. Up to 100% CPU
usage and full hang. Had to kill it in the end - nothing more was happening.
M16 Linux 2000-052809

Comment 33

18 years ago
Same build: another one..no freeze, just never loads, while spinning "forever"

http://www.rpmfind.net/linux/RPM/GByName.html

BTW: seems bug 29461 with dups is about same thing?

Comment 34

18 years ago
Cc'ing Waterson.
Keywords: arch, nsbeta2

Comment 35

18 years ago
[nsbeta2-], nominating for nsbeta3, when we are scheduled to tackle performance 
problems.
Keywords: nsbeta3
Whiteboard: [PDT-] → [nsbeta2-]

Comment 36

18 years ago
typo above: it seems bug 29641 is a dup (not 29461)

Comment 37

18 years ago
*** Bug 43648 has been marked as a duplicate of this bug. ***

Comment 38

18 years ago
Sorry: Bug 43648 is not a dup - it's a crasher. Reopened it.

Comment 39

18 years ago
Seems much better now, thanks to Waterson's changes. nsbeta3-, ->future

*** This bug has been marked as a duplicate of 29641 ***
Status: ASSIGNED → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → DUPLICATE
Whiteboard: [nsbeta2-] → [nsbeta2-] [nsbeta3-]
Target Milestone: M17 → Future

Comment 40

18 years ago
Verified duplicate.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.