Regression in Encog Silverlight test from FF6 to FF7

RESOLVED WORKSFORME

Status

()

Core
Plug-ins
RESOLVED WORKSFORME
7 years ago
3 years ago

People

(Reporter: jesup, Unassigned)

Tracking

({perf, regression})

7 Branch
x86
Windows 7
perf, regression
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [qa?], URL)

Attachments

(1 attachment)

(Reporter)

Description

7 years ago
Tom's Hardware has a test called "Encog Silverlight"; they reported a significant regression in FF performance from FF6 to FF7

"Firefox 7 shows a significant performance drop from version 6, moving down to fourth place."
(Reporter)

Updated

7 years ago
Keywords: regressionwindow-wanted
Whiteboard: [qa?]

Comment 1

7 years ago
Have you reproduced? Are there instructions for running this test?
Status: NEW → UNCONFIRMED
Ever confirmed: false
(Reporter)

Comment 2

7 years ago
Good point, though I imagine it isn't a mistake.  (Bugzilla defaults to NEW for me.)

As for instructions, you should contact Tom's Hardware, but I imagine it's the test shown by the top Google hit:

http://www.heatonresearch.com/node/1201

My AthlonX4 (fairly fast for a year or two ago) got 40.17 on nightly (with a lot of tabs open!  (~200, 1.1GB RSS 1.3GB VSS, WinXP)

Comment 3

7 years ago
I tried to reproduce the issue, to look for regressions in behavior from previous Firefox builds.
I checked for Fx6, Fx7 and Fx8Beta5. In the attachment you can see that, indeed the performance for this test run is poorer for Fx 7 than Fx6 but it gets back to the same results for Fx8. So, there is no real evidence that there is a regression in behavior.

Comment 4

7 years ago
Created attachment 570235 [details]
Comment 3 results
(Reporter)

Comment 5

7 years ago
From their "Methodology" page:

"Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variations."

How many runs of each did you do?  How much run-to-run variance was there?  I think a single run is not enough to confirm or deny here.

I will note they list the test as 'dated', which means they may be looking for a replacement for it.
I get the same Result (~ 7,39...) for all of the major NPAPI Browsers (Firefox Trunk, Opera Next, Chrome 18) on the first Run.

Thus resolvable?
(Reporter)

Comment 7

7 years ago
Given the previous comment on it getting worse in 7 and better again in 8beta, I think this can be closed if the latest Tom's Hardware test either removed it or we scored ok on it.
Resolving per previous Comments.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WORKSFORME
Keywords: regressionwindow-wanted
You need to log in before you can comment on or make changes to this bug.