The default bug view has changed. See this FAQ.

JavaScript speed regression

RESOLVED INCOMPLETE

Status

()

Core
JavaScript Engine
RESOLVED INCOMPLETE
11 years ago
9 years ago

People

(Reporter: Simon Howes, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

11 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20060804 Minefield/3.0a1
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20060804 Minefield/3.0a1

I have been benchmarking the speed of javascript over recent months on trunk, and I have found that it has regressed somewhat.


Testcase: http://www.24fun.com/downloadcenter/benchjs/benchjs.html

Regressed from 12secs, to 15secs, it did go up for a while to 13secs, and a couple of months later it has gone down to ~25secs.

This test takes 6secs in Safari, 7secs in Opera.

Call for review?

Reproducible: Always
Can you put any dates againts those numbers?
Assignee: nobody → general
Severity: major → normal
Component: General → JavaScript Engine
Product: Firefox → Core
QA Contact: general → general
Version: unspecified → Trunk
Also, running individual tests, eliminating DOM variables that may confound this bug being in the Core:JavaScript Engine product:component, are all important.  Is the test source easily viewable?  Is it possible to run individual tests and find which ones regressed?  How about reduced testcases that do not depend on the DOM or browser objects?

Just changing the GC scheduling and memory pressure machinery could make some synthetic benchmarks regress a lot.  If there is one particular test that concatenates strings and uses up all available VM, that will skew all the results into uselessness.

/be
(Reporter)

Comment 3

11 years ago
I do not know of any exact dates.  It has becomes slower over time.  I haven't ran any tests for a few weeks, I go to test and find this huge regression.

I have been a follower of trunk builds, however I have tried Bonecho, and it seems that has regressed just as far. 

Most natable tests from the link I gave, test 2 VM size for Minefield goes from 350MB to 550MB when it is running, and test 3 is one part that has slowed down the most.  Test 3 for Firefox 1.5 worked out at 8 secs, 16secs for Bonecho and Minefield builds.

I'm no programmer or web designer, so I will not be able to make test cases myself.
Summary: Javascript speed regression → JavaScript speed regression
I ran the tests four times, each with 1.5.0.6 and latest trunk, and the most significant differences were for two tests: "Replace images" and "Text manipulations". Average results were:

Test "Replace images": 1.5.0.6 scored 0.894 sec. and trunk 1.181 sec.
Test "Manipulate text": 1.5.0.6 scored 0.868 sec. and trunk 1.175 sec.

I think because of cairo and the spell checker.
 
(Reporter)

Comment 5

11 years ago
Cheers, still a noticeable difference between official release and trunk.

A huge difference in fact between our set-ups Ria.  What OS was that on?
(In reply to comment #5)
> 
Windows XP.

Comment 7

11 years ago
Some results for cairo trunk : 
http://bakkap.free.fr/FireFox/BenchJS.htm
It regressed today.

Comment 8

11 years ago
(In reply to comment #7)
> Some results for cairo trunk : 
> http://bakkap.free.fr/FireFox/BenchJS.htm
> It regressed today.
> 

Forgot what I said,there is a huge boost in perf in latest hourlies : especially in test 6 (about 100% faster),and a small improvement in test 3 and 4.

Comment 9

9 years ago
Simon:  Can this bug be closed out?  (WFM?)  Are you still experiencing this issue?  What about with a Firefox3 beta?

Comment 10

9 years ago
Marking WFM until more data is provided.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → WORKSFORME

Comment 11

9 years ago
Resolving as INCOMPLETE instead of WFM.  Looking forward to more info, though.
Resolution: WORKSFORME → INCOMPLETE

Comment 12

9 years ago
I'm not sure if this can be closed or not,but I've put back the test online here : 
http://bakkap.free.fr/Firefox/Benchs/benchjs/benchjs.html

What kind of regression range is useful ?
I have a lot of numbers for this tests.

Comment 13

9 years ago
Well, it would be nice to know how we compare in a recent Fx3beta nightly versus the shipping version of Fx2, for one thing.  Let's start with that before we start hunting a regression-range.

Thanks.
(Reporter)

Comment 14

9 years ago
Well whatever we had 'bug' back when this bug was first started, has been reversed.

Trunk 2008-01-28:

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?{%223d-cube%22:[483,382,386,376,398],%223d-morph%22:[608,615,615,609,606],%223d-raytrace%22:[293,285,288,290,295],%22access-binary-trees%22:[126,127,127,130,117],%22access-fannkuch%22:[384,381,383,386,383],%22access-nbody%22:[435,441,433,435,479],%22access-nsieve%22:[156,163,162,159,160],%22bitops-3bit-bits-in-byte%22:[161,173,157,157,155],%22bitops-bits-in-byte%22:[159,165,159,163,164],%22bitops-bitwise-and%22:[664,736,763,757,748],%22bitops-nsieve-bits%22:[195,189,190,188,193],%22controlflow-recursive%22:[80,79,81,83,84],%22crypto-aes%22:[203,224,226,225,220],%22crypto-md5%22:[130,117,114,114,114],%22crypto-sha1%22:[120,119,117,120,123],%22date-format-tofte%22:[364,362,368,368,364],%22date-format-xparb%22:[198,197,194,197,197],%22math-cordic%22:[343,331,325,327,329],%22math-partial-sums%22:[392,362,363,365,382],%22math-spectral-norm%22:[157,155,154,152,151],%22regexp-dna%22:[338,338,339,338,336],%22string-base64%22:[368,338,339,341,332],%22string-fasta%22:[323,323,331,351,325],%22string-tagcloud%22:[264,263,267,243,238],%22string-unpack-code%22:[436,436,437,476,485],%22string-validate-input%22:[157,161,184,149,154]}




2.0.11

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B594,559,546,555,560%5D,%223d-morph%22:%5B1134,1126,1138,1132,1118%5D,%223d-raytrace%22:%5B319,304,281,303,301%5D,%22access-binary-trees%22:%5B125,159,156,157,164%5D,%22access-fannkuch%22:%5B249,259,252,260,262%5D,%22access-nbody%22:%5B583,600,603,591,616%5D,%22access-nsieve%22:%5B178,180,178,178,178%5D,%22bitops-3bit-bits-in-byte%22:%5B223,226,228,226,222%5D,%22bitops-bits-in-byte%22:%5B207,211,211,209,206%5D,%22bitops-bitwise-and%22:%5B2742,2727,2722,2727,2816%5D,%22bitops-nsieve-bits%22:%5B258,246,247,249,257%5D,%22controlflow-recursive%22:%5B88,87,85,88,86%5D,%22crypto-aes%22:%5B215,220,215,216,217%5D,%22crypto-md5%22:%5B188,172,173,177,174%5D,%22crypto-sha1%22:%5B187,191,189,188,189%5D,%22date-format-tofte%22:%5B595,588,595,603,600%5D,%22date-format-xparb%22:%5B1108,1087,1091,1085,1094%5D,%22math-cordic%22:%5B587,521,562,517,533%5D,%22math-partial-sums%22:%5B380,391,400,390,384%5D,%22math-spectral-norm%22:%5B252,232,233,234,238%5D,%22regexp-dna%22:%5B458,466,459,464,458%5D,%22string-base64%22:%5B520,511,520,526,510%5D,%22string-fasta%22:%5B413,433,410,412,440%5D,%22string-tagcloud%22:%5B425,378,403,407,382%5D,%22string-unpack-code%22:%5B702,702,691,693,668%5D,%22string-validate-input%22:%5B263,267,262,258,263%5D%7D
(Reporter)

Comment 15

9 years ago
I could try giving the correct URL

Trunk - 

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?{%223d-cube%22:[483,382,386,376,398],%223d-morph%22:[608,615,615,609,606],%223d-raytrace%22:[293,285,288,290,295],%22access-binary-trees%22:[126,127,127,130,117],%22access-fannkuch%22:[384,381,383,386,383],%22access-nbody%22:[435,441,433,435,479],%22access-nsieve%22:[156,163,162,159,160],%22bitops-3bit-bits-in-byte%22:[161,173,157,157,155],%22bitops-bits-in-byte%22:[159,165,159,163,164],%22bitops-bitwise-and%22:[664,736,763,757,748],%22bitops-nsieve-bits%22:[195,189,190,188,193],%22controlflow-recursive%22:[80,79,81,83,84],%22crypto-aes%22:[203,224,226,225,220],%22crypto-md5%22:[130,117,114,114,114],%22crypto-sha1%22:[120,119,117,120,123],%22date-format-tofte%22:[364,362,368,368,364],%22date-format-xparb%22:[198,197,194,197,197],%22math-cordic%22:[343,331,325,327,329],%22math-partial-sums%22:[392,362,363,365,382],%22math-spectral-norm%22:[157,155,154,152,151],%22regexp-dna%22:[338,338,339,338,336],%22string-base64%22:[368,338,339,341,332],%22string-fasta%22:[323,323,331,351,325],%22string-tagcloud%22:[264,263,267,243,238],%22string-unpack-code%22:[436,436,437,476,485],%22string-validate-input%22:[157,161,184,149,154]}



I wish bugzilla would truncate links.

Comment 16

9 years ago
(In reply to comment #13)
> Well, it would be nice to know how we compare in a recent Fx3beta nightly
> versus the shipping version of Fx2, for one thing.  Let's start with that
> before we start hunting a regression-range.
> 
> Thanks.
> 

Ok,I compared firefox 2.0.0.11 vs current trunk on windows.
I ran all tests in safemode and averaged 4 runs :
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 

average total runtime : 8.28025 s
average time for test 1 : 1.586 s
average time for test 2 : 1.648 s
average time for test 3 : 0.686 s
average time for test 4 : 0.28825 s
average time for test 5 : 0.24575 s
average time for test 6 : 2.24025 s
average time for test 7 : 1.586 s

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3pre) Gecko/2008012901 Minefield/3.0b3pre ID:2008012901 

average total runtime : 6.28525 s
average time for test 1 : 1.563 s
average time for test 2 : 1.16725 s
average time for test 3 : 0.606 s
average time for test 4 : 0.54925 s
average time for test 5 : 0.221 s
average time for test 6 : 1.946 s
average time for test 7 : 0.23275 s

test 1 is giving about the same numbers between firefox 2.0.0.11 and current firefox trunk.
test 2 is a lot faster on windows firefox trunk.
test 3 is faster on windows firefox trunk.
test 4 is a lot slower on windows firefox trunk.
test 5 is faster on windows firefox trunk.
test 6 is faster on windows firefox trunk.
test 7 is a lot faster on windows firefox trunk.

In conclusion,we are a lot faster overall but also a lot slower on test 4.

Comment 17

9 years ago
ps : hardware used was 
Pentium D915 (2.8 GHz)
Intel Q965/Q963 Express Chipset Family 

Test 4 manipulates a long text in different ways.

Hope this is useful :)

Comment 18

9 years ago
Yani:  This is great information.  Test 4 does a few different things and is very benchmark-y (for example, the "spreadingtext()" routine could be, I think, implemented more effectively in other ways (perhaps split/join?).  I'm cc:ing jresig for his thoughts on these benchmarks.

It would be interesting to break this testcase down into its individual parts and find out which specific aspects of it are dramatically slower now.

Comment 19

9 years ago
Thank you Brian , I'll try to get reduced and splitted testcases.
You need to log in before you can comment on or make changes to this bug.