126731 - Performance problems loading any web page on this site

Status: RESOLVED WORKSFORME

(Core :: DOM: Core & HTML, defect)

Description

[David D. Kilzer (ddk)]

USER_AGENT: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.8) Gecko/20020205
NOTE: This browser was compiled locally from the 0.9.8 SRPM on
my Red Hat 6.2 system running a 2.2.19 kernel.

All pages on this web site load very slowly (especially when
compared to NS 4.77; see below).  NOTE:  Strict JavaScript
warnings are OFF in this case, and the pages still loaded
very slowly.

Originally I thought this might be related to Bug 104250, but
there are not a "massive number of images" (in my opinion) on
the web page, and the tables are "only" nested 6 or 7 levels
deep.

The apparent cause of the problem (from what I can guess
through observation) is the sheer number of JavaScript
warnings generated whether or not "strict JavaScript warnings"
are enabled.  After turning this option on in Mozilla 0.9.8,
memory usage jumped from 22MB to 156MB (using "top") while
loading this one page with a JavaScript debugger window open.
The first warnings complained about assigning values to
undeclared variables.  The last warnings complained about
missing properties from the 'window' object.

The first time I tried this (after having Mozilla running
for a couple days), Mozilla crashed (or the kernel killed it).
The second time I tried it (as described above when Mozilla
went from a 22MB to a 156MB memory footprint), I only had
the one web page from www.midwestsupplies.com open with a
JavaScript console, and the browser did not crash.

Note that all of these pages were generated with Microsoft
FrontPage 5.0, so my guess is that its generating the poor
JavaScript and including it on every web page.  If FrontPage
5.0 is the cause of this, the problem should be fixed or
worked around since there will mostly like a lot of web
pages using it to publish web sites.

I will attach a test case for posterity.  Note that it makes
no difference if the page is loaded locally or from a web
server.

Related bugs I found during a quick search:

Bug 78179  - mozilla becomes unavailable because the javascript
             console is flooded with javascript warnings
Bug 81279  - JS console sucks down memory and CPU
Bug 104549 - Too many duplicate warnings in javascript console
             on referencing an undefined variable or object
             property with strict warnings enabled

REGRESSION:  The Netscape Communicator 4.77 browser will load
this web page with a slight pause (probably for table layout),
but is otherwise uninhibited by the JavaScript in the page.
USER_AGENT: Mozilla/4.77 [en] (X11; U; Linux 2.2.19 i686)

I'm running Red Hat 6.2 on a Dell Optiplex GX110 with 256MB
RAM, 128MB swap, 867 MHz Pentium III (Coppermine).

Comment 1

[David D. Kilzer (ddk)]

Attachment: Source of http://www.midwestsupplies.com/index.htm (plus BASE tag)

This is the full HTML source of the index.htm web page for
www.midwestsupplies.com with the addition of a BASE tag to
make URLs on the web page work properly.

Comment 2

[David D. Kilzer (ddk)]

Attachment: Save "web page complete" of http://www.midwestsupplies.com/index.htm

Used Mozilla 0.9.8 to save "web page complete" of the index.htm file
on the www.midwestsupplies.com web site.

Comment 3

[David D. Kilzer (ddk)]

NOTE: Attachment 70549 [details] is a .tar.gz of the HTML file plus the 
local support files saved by Mozilla 0.9.8.

Comment 4

[Phil Schwartau]

I will attach some stack traces I got while trying to load this site
in a debug Mozilla build 2002-02-17 WinNT, and interrupting the load
manually in the Visual C++ debugger - 

Comment 5

[Phil Schwartau]

Attachment: Stack traces from manual interrupts of page load on WinNT

Comment 6

[Phil Schwartau]

Confirming slow load on WinNT, too. OS: Linux ---> All.

The interrupts all pick up Layout stack traces, so reassigning there
for the time being. I'm not exactly sure what the problem is.

cc'ing Bob Clary, in case he may be familiar with the templates
this site is using. As the reporter notes, they use Microsoft Frontpage,
and I noticed external JS files using a menu template called 
"DHTML QuickMenu" from an outfit called opencube. Are these known 
Evangelism problems? 

I can't use the JS Debugger on the site very usefully, because the
DHTML menu JS is obfuscated: it's all on ONE line!

David: would you happen to know if this site ever worked in Mozilla?
If so, how recent was that build?

Note: with JavaScript warnings turned off, I got no errors from this 
site in the JavaScript Console. My instincts tell me this is probably
not a problem of JS Engine performance, but rather DHTML performance
(e.g. DOM Level 0) or Layout performance problems due to reflow.

Please reassign back if anyone disagrees -

Comment 7

[David D. Kilzer (ddk)]

I do not have any regression information on whether this page ever
worked on older versions of Mozilla.  I can test with a locally
built version of 0.9.7 at home tonight, though.

Added 'perf' keyword its definition.

Comment 8

[Bob Clary [:bc] (inactive)]

Phil, I am not familiar with this site or script and will look into it right
away however if you choose pretty print from the venkman menus and click on a
particular function it should be displayed in a readable format. I will get back
to you on the other issues.

Comment 9

[Bob Clary [:bc] (inactive)]

This DQM script contains a number of poorly written constructs from the view of
performance. 

Eval is used extensively which is of itself a performance hit. 

In addition, Evals, and other expressions are not cached but are executed each
time they are needed. For example in function q32 the same expressions are
eval'd over and over and the same string.toLowerCase() is compared. Perhaps
other browsers cache the results of such repeated evals and string operations. I
would say DQM is a prime Evang candidate for performance unless you wish to
consider improving our perfomance on this type of code.

Comment 10

[Bob Clary [:bc] (inactive)]

using the venkman trick 

evald dd(console.sourceView.childData.lines.join("\n"));

to copy source from a pretty printed window:

    function q32(index) {
        if (!eval("window.DQM_url" + index)) {
            return;
        }
        if (eval("window.DQM_urltarget" + index)) {
            where = eval("DQM_urltarget" + index);
        } else {
            where = DQM_urltarget;
        }
        if ((where.length < 1) || (where.toLowerCase() === "_self")) {
            window.location.href = eval("DQM_url" + index);
        } else {
            if (where.toLowerCase().indexOf("_parent") > -1) {
                eval("parent.window.location=DQM_url" + index);
            } else {
                if (where.toLowerCase().indexOf("_new") < 0) {
                    eval("parent." + where + ".location=DQM_url" + index);
                } else {
                    tu = eval("DQM_url" + index);
                    window.open(tu);
                }
            }
        }
    }

Comment 11

[David D. Kilzer (ddk)]

Same problem occurs in Mozilla 0.9.7:

  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20020105

Comment 12

[Phil Schwartau]

The site uses at least these three external JS files:

      view-source:http://www.midwestsupplies.com/midwest_data.js
      view-source:http://www.midwestsupplies.com/dqm_script.js
      view-source:http://www.midwestsupplies.com/dqm_ie.js

The last one is no typo: Mozilla falls into their "IE5" codepath,
and the so the dqm_ie file is document.written by the dqm_script file. 
The midwest_data file is simply a list of constants used in
the menu creation.

Thanks to Bob, I was able to pretty-print the functions in the 
dqm_script and dqm_ie files; I will attach these below.

Comment 13

[Phil Schwartau]

Attachment: dqm_script.js (pretty-printed and breakpointable)

Comment 14

[Phil Schwartau]

Attachment: dqm_ie.js (pretty-printed and breakpointable)

Comment 15

[Phil Schwartau]

Attachment: HTML file loading the pretty-printed JS files; can debug

Comment 16

[Phil Schwartau]

OK, using the debuggable HTML page attached above in Comment #15,
you can set breakpoints in the Mozilla JS Debugger ("venkman").

When the JS files were obfusctated and all on one line, as they
are at the parent site, you couldn't do that. Thanks to Bob
for the pretty-printing advice! (View | Pretty Print in venkman). 

My findings from using the JS Debugger:


if(!q73)
{
  if(!window.DQM_cancel_onload)
    onload=DQM_handleOnload; <--- window.onload handler set to DQM_handleOnload

   etc.


function DQM_handleOnload()
{
  if(q61)
    return;
      
  q18=0;
  
  while (eval("document.menu"+q18))
    q18++;
    
  q31(); <----------------------------- takes, say, 40 seconds on my WinNT box
  
  i=0;
  while(eval("window.DQM_rollover_image"+i)) <--this GENERATES onload events!!!
  {
    q20=q20.concat(new Array(new Image()));
    q20[i].src=eval("DQM_rollover_image"+i);
    q21=q21.concat(new Array(q16("menu"+i).src));
    i++;
  }
  
  q61=true;
  eval(DQM_onload_statement);
}


function q31()  <----------------------------------- the function called above
{
  for(i=0;i<q18;i++)
  {
    if(eval("window.DQM_subdesc"+i+"_0"))
    {
      q1(i,true);
    }
  }
  
  for(i=0;i<q19.length;i++)
    q1(q19[i],false);
}

Comment 17

[Phil Schwartau]

COMMENTS:

1. Note their onload handler seems to generate onload events itself!
   I say this because the JS Debugger call stack shows the function
   calling itself. A DOM expert will need to see why this is happening.

2. I don't know why the q31() function takes so long; I didn't spend
   as much time on that one. I believe the counter variable q18 goes
   up to 26 before the function completes. That's not all that high -
   As Bob said above, the use of eval() is going to slow things, but...

3. Notice the use of the |concat| method of JS arrays, as in the line

           q20=q20.concat(new Array(new Image()));

   This potentially could be a JS Engine bottleneck, but I didn't 
   find it to be so in the JS Debugger. Stepping over that line was
   pretty quick. 


SUMMARY: 

The slow performance can be due to at least these four reasons:

A) Inefficient coding at the site (but then why does IE6 load it OK?)
B) Possible JS Engine performance issues (but I'm not seeing them)
C) Possible DHTML performance issues
D) Possible Layout reflow issues


I've come to the end of my knowledge, and I want to send this one along
to jst to see if he thinks the main problem might be C) or D) above.

Reassigning to DOM Level 0; thanks -

Comment 18

[Phil Schwartau]

*** Bug 159965 has been marked as a duplicate of this bug. ***

Comment 19

[Phil Schwartau]

Some time has elapsed since the above analysis, and I no longer
see any references to the "DHTML QuickMenu" template from OpenCube
at the given site: http://www.midwestsupplies.com/

However, there are other sites that use it, so let's keep this bug
open. In particular, the "DHTML QuickMenu" link at the parent site,
http://www.opencube.com, and the site at the duplicate bug 159965.

I also see a change in the filename for the external JS file that
gets loaded for Mozilla/Netscape. At the original site above, this
used to be dqm_ie.js (see Comment #12 above); but at the site in
the duplicate bug 159965, it is now called dqm_ns6.js.

                  site: http://www.polycom.com

I used the Mozilla's Tools > Web Development > JavaScript Debugger
("Venkman") to do some profiling at this site: 


                     Venkman Profile Report

Created .......... Mon Jul 29 2002 15:00:45 GMT-0700 (Pacific Daylight Time)
User Agent ....... Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.1b)                    
Gecko/20020725
Debugger Version . Venkman 0.9.24 [Mozilla rv:1.1b/20020725]
Sorted By ........ Total Time


================================================================================  
1 <http://www.polycom.com/dropdown/files/dqm_ns6.js>

  dqm_ns6.js: 2500 - 1000000 milliseconds
    Function Name: q1()  (Lines 2 - 3)
    Total Calls: 35 (max recurse 0)
    Total Time: 10218 (min/max/avg 93/797/291.94)
  
-------------------------------------------------------------------------------

  dqm_ns6.js: 750 - 2500 milliseconds
    Function Name: q0()  (Lines 2 - 3)
    Total Calls: 35 (max recurse 0)
    Total Time: 2296 (min/max/avg 15/172/65.6)

-------------------------------------------------------------------------------

  dqm_ns6.js: 250 - 750 milliseconds
    Function Name: q15()  (Lines 2 - 3)
    Total Calls: 239 (max recurse 0)
    Total Time: 563 (min/max/avg 0/16/2.36)
  
-------------------------------------------------------------------------------

  dqm_ns6.js: 100 - 500 milliseconds
    Function Name: q16()  (Lines 2 - 3)
    Total Calls: 111 (max recurse 0)
    Total Time: 298 (min/max/avg 0/16/2.68)

    Function Name: q17()  (Lines 2 - 3)
    Total Calls: 6 (max recurse 0)
    Total Time: 278 (min/max/avg 46/47/46.33)

  


================================================================================
2 <http://www.polycom.com/dropdown/files/dqm_script.js>
 
  dqm_script.js: 2500 - 1000000 milliseconds
    Function Name: DQM_handleOnload()  (Lines 6 - 7)
    Total Calls: 1 (max recurse 0)
    Total Time: 10265 (min/max/avg 10265/10265/10265)

    Function Name: q31()  (Lines 6 - 7)
    Total Calls: 1 (max recurse 0)
    Total Time: 10218 (min/max/avg 10218/10218/10218)

  
-------------------------------------------------------------------------------
 
  dqm_script.js: 75 - 250 milliseconds
    Function Name: getLevelVal()  (Lines 8 - 9)
    Total Calls: 105 (max recurse 0)
    Total Time: 107 (min/max/avg 0/16/1.02)

    Function Name: q33()  (Lines 8 - 9)
    Total Calls: 64 (max recurse 0)
    Total Time: 0 (min/max/avg 0/0/0)

    Function Name: q29()  (Lines 8 - 9)
    Total Calls: 1 (max recurse 0)
    Total Time: 0 (min/max/avg 0/0/0)
  
================================================================================

Comment 20

[Phil Schwartau]

Once again, we see the q31() function takes a long time. Further
analysis will be required to answer the questions in Comment #17 -

Comment 21

[Johnny Stenback (:jst)]

Mass-reassigning bugs to [e-mail]

Comment 22

[Andrew Schultz]

This does not seem to be a problem with current trunk (2004021108). 
http://www.opencube.com/ loads in <1 second.
http://www.polycom.com/home/ loads in ~2 seconds
attachment 70667 [details] loads in ~2 seconds.

polycom and attachment 70667 [details] load in ~3 seconds with Mozilla 1.1b (what Phil was
using in comment 19).

but perhaps my computer is too fast.  Athlon XP 2700+.  :P

Comment 23

[Simon Bohlin]

(In reply to comment #22)
> but perhaps my computer is too fast.  Athlon XP 2700+.  :P

I tried this on Windows 98, 96MB, Celeron 400MHz using a regular
Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.7.12) Gecko/20050915

http://www.opencube.com/ loads in ~4 seconds.
http://www.polycom.com/home/ loads in ~8 seconds.
attachment 70667 [details] loads in ~8 seconds.
Time was measured by opening the page, closing and opening the same page again.

The attachment seems to have take all its javascript from bugzilla attachments, and since I don't get anything like 40 seconds delays, I guess this problem is solved. (I didn't investigate further to see if maybe opencube has cleaned up their code.)

Preferably someone who previously saw this bug could check again and possibly close this bug?

Comment 24

[David D. Kilzer (ddk)]

This is still a problem in Firefox 1.5 (Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5).  If you open the original attachment for this bug, Attachment 70548 [details], you will see a noticable pause (with corresponding CPU spike) before the page is rendered.  Try reloading the page a couple times to negate effects of DNS lookups.

If you open the JavaScript console, you'll note a number of error messages stating:

Error: Error in parsing value for property 'text-decoration'. Declaration dropped.
Source File: https://bugzilla.mozilla.org/attachment.cgi?id=70548   Line:0

In my opinion, Firefox should not pause that long while rendering the page.

Comment 25

[Ryan VanderMeulen [:RyanVM]]

Can anybody reproduce this anymore? The site has changed considerably and the testcase doesn't really work anymore. I'm inclined to resolve this INCOMPLETE.

Comment 26

[Mihai Morar, (:MihaiMorar)]

No performance issue on Latest Nightly 25.