85381 - Pathologically large ftp directory listing pegs CPU for 2 minutes

Status: RESOLVED FIXED

(Core Graveyard :: Networking: FTP, defect)

Description

[John Morrison]

If you go to ftp://ftp.sourceforge.net/pub/, and then click on sourceforge/,
mozilla 06/11 builds on mac/linux/win32, the browser pegs the cpu, and the UI
is locked up (seemingly dead) for >2minutes on ~500MHz machines.

The document returned to ftp 'ls' is about 500K in size.

Of course, the same ftp listing locks up Nav4.x, and a ftp listing that large
is really a bad idea any way you look at it, but the worst case behaviour 
should be that while it may take a long time to retrieve, the browser is 
still usable during that time (including allowing a graceful cancel).

In the end, this may just turn out to be a layout bug, but I thought I would 
pass it by ftp first (to see if there is something about the text to HTML 
conversion that is wrong).

Comment 1

[Bradley Baetz (:bbaetz)]

Right (the xul viewer, even with the timeout, has the same problem, BTW).

However, the 881K project Gutenberg text of oliver twist doesn't have this
problem (and displays quite quickly...)

its probably layout's problem, but I'm not sure. dougt?

Comment 2

[Doug Turner (:dougt)]

Attachment: long directory listing source

Comment 3

[Doug Turner (:dougt)]

The file protocol has the same problem. However, when downloading this data to
disk (to save it), there was no problems.  This is a necko client problem...
over to layout... I am sure that this is a dup of something..

Comment 4

[anthonyd]

why are necko bugs being kicked to layout?
doesnt matter anyway, WORKSFORME.

Comment 5

[John Morrison]

Well, it is a layout bug in the category of "large pages slow; UI starved".

But there are a collection of those already, I suppose. I also suppose that 
necko could make some simple changes to at least mitigate the problem when 
ftp gets handed a pathologically large directory listing: First, it should 
put a limit on the number of rows in the table(s) that it generates. 
Chopping the attachment into 500-row tables (instead of one big table) cut the 
time down from about 71 seconds, to 23 seconds (win32/500MHz/128MB/loaded from 
disk). Yes, you want all the rows to have the same width, but at 500 row 
intervals, does it matter? 

Second, necko could add 'width="100%" style="table-layout:fixed;"' to 
each <table> emitted; that cut the 23 second time down to 20.5 seconds.

While this ftp listing is an extreme example, these changes make sense for 
even a "medium" size ftp listing, in my opinion. Since we're not necessarily 
gonna get 3 times faster from layout changes, and we control the format of the 
html of the ftp listing, I'll give this back to necko to fix.

Comment 6

[John Morrison]

-> necko, possibly to take the changes I outlined above.

Comment 7

[Bradley Baetz (:bbaetz)]

Attachment: patch

I don't like the look of width=100%. This does the rest, though

Comment 8

[Doug Turner (:dougt)]

Comment on attachment 60458 [details] [diff] [review]
patch

workaround at best.  is there a layout bug number that we can add to the
comments in the patch?

Comment 9

[Bradley Baetz (:bbaetz)]

Well, this was it...

Listing /dev took > 5 minutes, and view source on that page took so long that I
gave up after 10.

Comment 10

[Darin Fisher]

Comment on attachment 60458 [details] [diff] [review]
patch

>Index: nsIndexedToHTML.cpp

>+
>+    if (++mRowCount > ROWS_PER_TABLE) {
>+        pushBuffer.Append(NS_LITERAL_STRING("</table>\n") + tableHeading);
>+        mRowCount = 0;
>+    }

it'd be nice to add a comment here to explain why you are doing this.

with that, sr=darin

Comment 11

[Bradley Baetz (:bbaetz)]

Checked in

Comment 12

[Asa Dotzler [:asa]]

*** Bug 122308 has been marked as a duplicate of this bug. ***

Comment 13

[Cryo]

I'd like to request that this bug be reopened.

ftp://ftp.hp.com/pub/printers/software/

shows that this problem still exists.

see Bug 122308

Comment 14

[benc]

I opened the sourceforge and hp links mentioned in the two bugs, and it took
about 5-10 minutes each time. I used "System Monitor" in Win 98, and watched my
CPU usage hit 100%.

Communicator 4 suffers a siimilar fate (and why would anyone put so many files
in a single directory anyhow?! Some fool that always sends the output of ls -l
to grep..?)

I'm flagging this as perf, and will resolve if dougt, bbaetz or jrgm says it is
okay.

Comment 15

[benc]

*** Bug 118172 has been marked as a duplicate of this bug. ***

Comment 16

[Tracy Walker [:tracy]]

mass remove verifyme requests greater than 4 months old