659467 - TI: Test from Opera blog is significantly slower than in Firefox 4

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[Timvde]

User-Agent:       Mozilla/5.0 (X11; Linux i686 on x86_64; rv:6.0a1) Gecko/20110524 Firefox/6.0a1
Build Identifier: 

The blog is about slow executing of Javascript in Webkit browsers. While the results aren't as bad, Firefox regressed on this micro benchmark. In Firefox 4 I was consistently getting 10-12ms, in 6.0 TI builds, results lie between 45 and 50 ms.

Reproducible: Always

Steps to Reproduce:
1. Go to the attached link
2. Scroll down to "webkit hotspot test-case"
3. Click and see higher numbers in TI builds than in Firefox 4

Actual Results:  
The script executed slower than in Firefox 4

Expected Results:  
It should have executed equally fast (or faster)

Last revision set at this moment: 29f08848d96c

Comment 1

[Boris Zbarsky [:bzbarsky]]

For what it's worth, this test runs in about 8ms here on non-TI builds, which is the JM time.  If I turn off JM and just use TM, it's closer to 4ms.

Comment 2

[Boris Zbarsky [:bzbarsky]]

And it's too bad there's no link to the full testcase we're claimed to be 10x slower than Opera on.

Comment 3

[Timvde]

If I understand his post correctly, he's not talking about a stripped down testcase there, but about running his entrire unzip (http://jsunzip.googlecode.com/svn/trunk/source/jsunzip.js) on the provided testfile (http://corpus.canterbury.ac.nz/resources/cantrbry.zip).

Comment 4

[Boris Zbarsky [:bzbarsky]]

Ah, perhaps.  If we can get that into a runnable form, might be worth filing a bug on that too.

Comment 5

[Brian Hackett [Laid off!]]

The microbenchmark is stressing string ADD and string GETELEM, neither of which we have fast paths for in JM / JM+TI.  TM has them, and they would be pretty simple to add now that we have good type information (no need to IC, etc.)  IIRC Tom Schuster was interested in adding the GETELEM path at least, I don't know if there's a bug on it.

JM+TI is going to be a lot slower than plain JM because it pays a large penalty when taking stub calls for property access ops like GETELEM --- instead of just getting the string element, we have to go and check that the string value being pushed is in the type set associated with the result of the GETELEM (this overhead could be improved, but it's better to just wipe out the stub call).

The second interesting thing about the microbenchmark is that for ADD, even if we fast path the rope creation we still could do better.  This is a great example for the idea of doing linear type analysis (more tractable now that we have SSA) and combining that with type information to append to the 'dest' string in place rather than making a new rope node each iteration.  Pretty sure Luke is interested in this.

I used to be a little skeptical about doing linear types (worries about type information changing under you while the string goes from its def to single use), but I think I was being dumb and/or paranoid and we should really be able to do this.  Can follow the model set by the analyses we have for ignoring integer overflows and determining 'new' script properties, which run into similar issues.

I'll add the microbenchmark to the AWFY assorted page, should really get a shell testcase for the whole unzip and add it too.

Comment 6

[Timvde]

He has updated his jsunzip. The new version runs on all browsers including IE9 and is faster. The old file (at which we are 10 times slower than Opera) can be found at http://jsunzip.googlecode.com/svn-history/r2/trunk/source/jsunzip.js.

On the last revision, he states IE9 to be the fastest by a 25% margin. As I'm on Linux, I can't directly compare, but on my pc TI is twice as fast as Chrome and Opera, so it should outperform IE9. Anyone on Windows willing to confirm? Test case is at http://files.myopera.com/emoller/blog/performance.html.

Comment 7

[David Mandelin [:dmandelin]]

I get 3-5 ms on the WebKit hotspot link in today's TI nightly. Is that a WFM?

Comment 8

[Timvde]

David Mandelin: still about 50ms here. Are you sure you are still on the TI nightly? If you have been automatically updating on for a while, you might have been hit by bug 660616. No WFM at my side, anyway.

Comment 9

[David Mandelin [:dmandelin]]

(In reply to comment #8)
> David Mandelin: still about 50ms here. Are you sure you are still on the TI
> nightly? If you have been automatically updating on for a while, you might
> have been hit by bug 660616. No WFM at my side, anyway.

OK, we'll keep it open then. I'm fairly sure I'm still on the TI nightly, as I downloaded a fresh one today (which I had to, because the Windows builds had been timing out for the last 2 months) and I switched the typeinference pref back and forth to test both ways.

Comment 10

[Timvde]

Toggling the preference doesn't seem to have an immediate effect, in both ways. Probably the old code is still in memory. I can get reproducible results by:

- Open the test (quick link: http://files.myopera.com/emoller/blog/webkit.html)
- Toggle the TI preference
- Close the test tab
- Open about:memory and click the "Minimize memory usage" button
- Reopen the test for different results

In case it would make a difference: I'm on 64 bit Linux.

Comment 11

[Jan de Mooij [:jandem]]

Attachment: Shell testcase

Here's a shell version of the most recent version (revision 5) of jsunzip. I added the cantrbry_less_than_150k.zip file from the repository, the cantrbry.zip file is probably too large to embed in the file.

js -m -n    : 115 ms
d8          : 143 ms
js -m -j -p : 158 ms
js -j       : 161 ms
js -m       : 170 ms

Tomorrow I'll add this to awfy-assorted, but first I want to check whether the numbers for the full zip file are similar.

Comment 12

[Jan de Mooij [:jandem]]

Attachment: Shell testcase

Added this testcase to awfy-assorted (the one in assorted uses a separate data file though so we don't benchmark array literal parsing).

Comment 13

[Timvde]

Note sure since when this is fixed, but I'm on the 24/08/2011 build (changeset 78a56e48dd3c) and the test case at http://files.myopera.com/emoller/blog/webkit.html seems to be getting good results (10-11 ms). Not for very long, as I've tested it just a few weeks ago with bad results.

This bug is WFM now and can be closed :)

If wanted, I can bisect to the specific build so you can add the right commit and can resolve it as FIXED instead of WORKSFORME, if it matters.

Comment 14

[Jan de Mooij [:jandem]]

(In reply to Tim from comment #13)
> This bug is WFM now and can be closed :)

Yes, latest Nightly is at least as fast as FF 7 (and much faster than Chrome and Opera).