Closed Bug 482714 Opened 15 years ago Closed 8 years ago

Major Regression in CSS Rendering Performance

Categories

(Core :: CSS Parsing and Computation, defect)

defect
Not set
normal

Tracking

()

RESOLVED INVALID

People

(Reporter: jeresig, Unassigned)

References

()

Details

Based upon some of the tests created by Steve Souders (which can be found here, along with results):
http://www.stevesouders.com/blog/2009/03/10/performance-impact-of-css-selectors/

We're seeing a 4x performance regression from Firefox 3 to Firefox 3.1b2.

Now, it's not clear what exactly these tests are measuring - CSS rule matching may be a part of it, but that's not clear. Regardless - seeing a number like that jump 4x is definitely a huge warning sign.
That's a big regression range; getting it narrower is probably the most useful next step.
If what I'm supposed to see is that the numbers printed near the top of 
http://stevesouders.com/tests/css-selectors/baseline.php
are significantly different between 3.0 and mozilla-central, then I'm not seeing the problem.
DavidB - You're looking at the test page. If you want to see the results you should read the blog post ( http://www.stevesouders.com/blog/2009/03/10/performance-impact-of-css-selectors/ ).

There's some info in the blog post, but I'll provide more detail here. There
are five test pages (baseline, tag selector, 20K class selectors, 20K child
selectors, 20K descendant selectors), but it doesn't matter which one you use -
all of them are 4x slower on FF3.1b2 than FF 3.0.5.

For each test I loaded the page once to put it in cache. (Even though network
latency should be outside the scope of the timers in the page, because the page
is so large it's possible chunked encoding could affect the results, so instead
the page has a 2028 expires date so it's always read from cache. I watched my
access logs to make sure no requests were made except this first one.) After
caching, I loaded each page nine times and took the median. You can download
the raw data in csv (
http://stevesouders.com/tests/css-selectors-my-2000-tests-data.csv ).

The time measurement was done with a script block at the top (above the style
block and 20K anchors) and another script block at the bottom (right before the
close body tag).

I'd be happy to provide more information.
I need to be able to run the test that slowed down.  After reading the blog post, I thought comment 3 was the way to do that.  I can't tell if comment 4 is telling me I was wrong, or not.

How do I take a build of Firefox and see if it's fast or slow on the test that slowed down?
And to be clear, there are a *lot* of test pages there... ones with various numbers, etc.  Could you give me the URL to *one* that you know slowed down, and tell me how you measured it?
Also, what platform(s) were you testing on?  Windows?  Mac?  Linux?  What version?
For what it's worth, I also cannot reproduce this (on Mac; I assume David was testing on Linux as usual).  For the baseline page and n=2000, for example, I consistently get times of around 60ms from top of body and 70-80ms from top of head in both current trunk and Firefox 3.0.7.

In addition to David's questions, did you make sure you were testing both with clean profiles?  Some extensions can significantly affect various aspects of pageload time (Firebug is a prime example).
There are only five test pages. In my comment above, I already provided instructions and said you can pick any page (they're all consistently slower). I tested on Windows XP version 5.1. I disabled all add-ons, but did not use a clean profile.
I removed the 1000, 2000, test links to avoid confusion.

I tested on more of our Windows boxes to see if hardware makes a difference. When I test on the exact same hardware, 3.1b2 is only 25% slower across all the tests. This is obviously much less than 300% slower. 

I'll investigate more about the differences in the test installations that caused the issue. All our test boxes are supposed to have the same image and hardware, but the 3.0.5 box was faster. Once I find out why, I'll post the data.
OK, I just tried again on Mac.  The baseline.php page seems to have changed slightly (e.g. the numbered links are gone), but I shift-reloaded in all relevant browsers to make sure I had the most recent version.

Results:
1)  Times are basically identical in trunk, Firefox 3.1b2 and Firefox 3.0.7.
2)  There is a huge difference between reloading by hitting Cmd-R or clicking
    the Reload button and just focusing the url bar and hitting enter.  The
    latter is at least 5x faster.  Looking into that now.  I assume this is a
    cache issue, which can't explain Steve's results, since he was watching
    access logs.
Steve, I'd really appreciate you updating your blog post as you find out more here!
Ah, yes.  On reload we do cache validation, which means we make a request to the server and see what it says.  In this case, the original server response has no Last-Modified header, so we can't send If-Modified-Since and has no ETag, so we can't send If-None-Match.  We go ahead and do the GET, and the server responds with a "200 OK", so we do end up getting all the data from the server.   Best-case response we could hope for for a reload here would be a "304 Not Modified" response, which would allow us to use our cached data.

In any case, the file is 100KB, so on my network connection it should take approximately 300ms to download it.  That matches the times I was seeing for head-to-end times, and also explains the big difference I was seeing between head-to-end and body-to-end in this case: it's all network-limited.

Which leaves me with still seeing pretty identical behavior between the three versions I was testing, though there's enough noise and the times are small enough that 10% error or so could easily not be noticeable at first glance; I haven't done repeated timing and averaging, since I was looking for a 300% regression.
The PC I used to test FF 3.1 and IE 8 was slower (I think due to less memory). I re-ran on more consistent hardware and updated the blog post, chart, and csv with the new data. I ran both FF 3.0 and 3.1 on the same PC, and FF 3.1 came out only ~20% slower than FF 3.0.7.

I got distracted from the main point of the post - that CSS selectors have little impact on page load. Thanks for pointing out the discrepancy and sorry for the red herring.
Ah, ok.  That makes sense.

I've been trying to reproduce the 20% thing, and the noise on my machine is just so high that I'm not making much headway on it...  Certainly in terms of pinning it down to a particular nightly.  

Martiijn, can you reproduce that issue?
Flags: wanted1.9.1-
Flags: blocking1.9.1?
Flags: blocking1.9.1-
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.