Closed Bug 249546 Opened 20 years ago Closed 8 years ago

Results of tests of JSBench

Categories

(Core Graveyard :: Tracking, defect)

Product:
Core Graveyard
Component:
Tracking
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: zbraniecki, Unassigned)

References

(
URL
)

Details

(Keywords: meta, perf) 

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040625 Firefox/0.9
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040625 Firefox/0.9

This bug is similar to bug 140789 but it's about other testcase. This one (URL)
shows two things. We're very slow with JavaScript, and we're getting slower.

Testing machine: Athlon 1.8 Ghz, GeForce 4, Linux Slackware 10.0, Kernel 2.6.7,
256 MB of DDR Ram.

All results are best-of-3.

Firefox 0.9 Linux - 50.84
TEST 1 time: 26.461 sec.
TEST 2 time: 2.131 sec.
TEST 3 time: 0.838 sec.
TEST 4 time: 0.895 sec.
TEST 5 time: 0.393 sec.
TEST 6 time: 21.675 sec.
TEST 7 time: 2.754 sec.

Firefox 0.8 Linux - 33.21
TEST 1 time: 12.347 sec.
TEST 2 time: 2.175 sec.
TEST 3 time: 1.242 sec.
TEST 4 time: 0.896 sec.
TEST 5 time: 0.426 sec.
TEST 6 time: 13.434 sec.
TEST 7 time: 2.685 sec.


Konqueror 3.2.3 Linux - 51.62
TEST 1 time: 14.709 sec.
TEST 2 time: 1.156 sec.
TEST 3 time: 4.752 sec.
TEST 4 time: 11.666 sec.
TEST 5 time: 7.18 sec.
TEST 6 time: 5.422 sec
TEST 7 time: 6.732 sec.

Opera 7.51 QT Static Linux - 8.87 sec
TEST 1 time: 1.301 sec.
TEST 2 time: 1.207 sec.
TEST 3 time: 0.539 sec.
TEST 4 time: 2.525 sec.
TEST 5 time: 0.18 sec.
TEST 6 time: 2.644 sec.
TEST 7 time: 0.476 sec.

Also notice that in Test 6 Firefox didn't show any animation. instead it
displayed final stage at the end. 

I tested only Linux machines, so meaby someone could check Windows, MacOSX with
all possible engines and Gecko 1.6 and Gecko 1.7 to compare?

Reproducible: Always
Steps to Reproduce:
> This one (URL) shows two things

Sorry, no.  It shows about a dozen things, of which you saw 1.

> We're very slow with JavaScript

No.  Of the 7 tests, only 1 actually tests Javascript speed.  The rest test all
sorts of things but NOT Javascript.

> Also notice that in Test 6 Firefox didn't show any animation.

That's bug 249061.

As for the tests...

> TEST 1 time: 26.461 sec.

This is bug 242036

> TEST 6 time: 21.675 sec.

This is covered by a large number of bugs I filed sometime around the end of
May.  See bug 243726 with all its dependancies.  Also, the fix for bug 230170
speeds this up a good bit.  My current development tree is easily twice as fast
as trunk is on this test.

I also wonder about your numbers a bit -- on my machine (P3 733) test 6 runs in
about 18 seconds on current trunk SeaMonkey builds.  I don't know why yours is
so much slower (when accounting for your faster processor).  Could also be gtk1
vs gtk2 trouble.  Similarly, I'm not sure at the perf difference on test 6
between 0.8 and 0.9 -- equivalent SeaMonkey builds show no such differences and
there were no real changes in that date range that should have caused such
differences.

Note also that on an unevenly loaded system best-of-3 is a really bad metric...
average or median would work much better.
Component: DOM: Level 0 → Tracking
Depends on: 230170
Keywords: meta
Summary: [META] Results of tests of JSBench → Results of tests of JSBench
(In reply to comment #1)
> No.  Of the 7 tests, only 1 actually tests Javascript speed.  The rest test all
> sorts of things but NOT Javascript.

You're obviously right. But we can say that such benchmarks sticks better to
real web pages than pure JS test. And thats what people call "JavaScript in use".
 
> > Also notice that in Test 6 Firefox didn't show any animation.
> 
> That's bug 249061.

Roc said it's fixed on 22th. I checked it with nightly 20040725 (0.9 branch) and
it's not :/ Should I comply there 

> My current development tree is easily twice as fast
> as trunk is on this test.

Great! Is there any chance to have them in Fx 1.0?
 
>I don't know why yours is so much slower (when accounting for your faster
processor).  Could also be gtk1 vs gtk2 trouble.

Could be. I'm using nightly with gtk2. 

>  Similarly, I'm not sure at the perf difference on test 6 between 0.8 and 0.9
-- equivalent SeaMonkey builds show no such differences and
> there were no real changes in that date range that should have caused such
> differences.

I checked 0.9 branch nightly from today and it's really 0.9-like. 19 sec on Test
6. I'll check corresponding Seamonkey in a moment.
(In reply to comment #2)
> Roc said it's fixed on 22th. I checked it with nightly 20040725 (0.9 branch) and
> it's not :/ Should I comply there 

no. test gecko changes on trunk.
> And thats what people call "JavaScript in use"

I don't much care what people call it; in this bug database, please use terms of
art correctly; it reduces confusion.

> Great! Is there any chance to have them in Fx 1.0?

Absolutely not.  They would have had to be checked in 6 months ago for that to
happen.

As Christian says, it's not much worth testing the gecko in the branch builds
for this sort of thing.  It's pretty out of date at this point.
Results for Mozilla 1.7alpha (Fx 0.8) and 1.7 final (0.9) are different. I bet
the reason is gtk1/gtk2 diff.

Mozilla 1.7 alpha
TEST 1 time: 25.655 sec.
TEST 2 time: 3.102 sec.
TEST 3 time: 2.397 sec.
TEST 4 time: 1.208 sec.
TEST 5 time: 0.494 sec.
TEST 6 time: 9.067 sec.
TEST 7 time: 1.083 sec.

Mozilla 1.7 final
TEST 1 time: 23.44 sec.
TEST 2 time: 2.947 sec.
TEST 3 time: 1.415 sec.
TEST 4 time: 1.118 sec.
TEST 5 time: 0.444 sec.
TEST 6 time: 8.101 sec.
TEST 7 time: 1.041 sec.

Test 6 worked "ok" (but very chopped animation) but test 1 was strange. In both
- 1.7a and 1.7 the red bar dissappeared during animation. I think it's same bug
as the one with Test 6 in Firefox, but why i didn't see it in Fx?
Depends on: 242036, 243726
I'm somewhat curious as to what results you'll see on test 6 with tomorrow's
trunk builds (which will have bug 230170 fixed).
AthlonXP 1.8, GeForce 4 MX

Firefox (0.9 branch) 20040807 Linux Kernel 2.6.7 GCC 3.3.4:
TEST 1 time: 25.011 sec
TEST 2 time: 3.466 sec.
TEST 3 time: 0.939 sec.
TEST 4 time: 0.949 sec.
TEST 5 time: 0.4 sec.
TEST 6 time: 20.326 sec.
TEST 7 time: 2.717 sec.


Firefox (0.9 branch) 20040811 Linux Kernel 2.6.7 GCC 3.3.4:
TEST 1 time: 1.403 sec
TEST 2 time: 2.245 sec.
TEST 3 time: 0.914 sec.
TEST 4 time: 0.856 sec.
TEST 5 time: 0.394 sec.
TEST 6 time: 25.573 sec.
TEST 7 time: 2.716 sec.

Test 1 speeded up a lot(!), Test 6 is still invisible.
Um.. did you read my comment?  TRUNK builds.  Don't test branch.  It's pointless.
uhm. can't launch trunk: bug 255246
Try a SeaMonkey build?
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a3) Gecko/20040811 (gtk2)

TEST 1 time: 14.577 sec
TEST 2 time: 3.186 sec.
TEST 3 time: 1.322 sec.
TEST 4 time: 1.194 sec.
TEST 5 time: 0.407 sec.
TEST 6 time: 4.435 sec.
TEST 7 time: 1.009 sec

Test 6 - nice, little bit choppy and slower than in Opera or KHTML, but quite
fine for me.
I know it's still a little slower than Opera; see bug 243726 for more work being
done on it.

Good to hear that things are a little better now, though.  :)
a lot better if we'd compare invisible animation to visible one :)

And why Test 1 was so fast on todays branch-0.9? I can't see bug 242036 fixed
there, but the difference is huge.
> And why Test 1 was so fast on todays branch-0.9?

No idea.  Was updating of the status bar turned off in prefs?
> Was updating of the status bar turned off in prefs?

bingo. dunna know how or why since i never used it, but i'm trying to reproduce
that speed and only turning it off gives it. Sorry for spam
Depends on: 258819
Depends on: 345560
Depends on: 360484
Assignee: general → nobody
QA Contact: ian → chofmann
Marking all tracking bugs which haven't been updated since 2014 as INCOMPLETE.
If this bug is still relevant, please reopen it and move it into a bugzilla component related to the work
being tracked. The Core: Tracking component will no longer be used.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → INCOMPLETE
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.