638235 - Analyze BenchJS performance

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[David Mandelin [:dmandelin]]

There are currently 7 tests. Compared to Chrome 10, we are faster on test 5, but slower on the rest (times in seconds):

 test    Fx4      Cr10
 1     2.009     0.794
 2     7.989     3.198
 3     5.722     3.716
 4     0.199     0.088
 5     0.1       0.351
 6     0.821     0.771
 7     0.201     0.057

      17.04      8.98

Aggregation is by sum, so test 2 is the biggest problem right now.

Comment 1

[Boris Zbarsky [:bzbarsky]]

For what it's worth, I've run this test before (in an earlier incarnation that included window.open and window.status).  Except for maybe test 7, it's not js-bound as I recall.

In particular, test 1 and test 2 are measuring across timeouts; simply dropping my timeout clamp from 10ms to 4ms drops my runtimes by a factor of 2.5, to basically the same as Chrome.  Dropping it to 1ms drops them by another factor of 3.  So these tests are completely useless and I wouldn't worry about them too much.

Test 3, as I recall, is a more or less fair DOM + painting test, sorta, testing an operation whose performance doesn't matter much.

Test 4 might need measuring.

Test 5 is all DOM and CSS, last I checked.  So is test 6.  Can't hurt to double-check, I guess.

Comment 2

[David Mandelin [:dmandelin]]

That was quick. :-) Do you think it's worth getting in touch with the creators about the timeout issue, or should we just wait until ours gets fixed?

Comment 3

[Boris Zbarsky [:bzbarsky]]

I mailed whoever was maintaining this test about some issues back in 2003 several times, and was summarily ignored...

We could try contacting these people, especially if they're different.  For the sake of all the wasted CPU cycles if nothing else. ;)

Comment 4

[Boris Zbarsky [:bzbarsky]]

Attachment: Locally runnable version with no cleanup for your delectation

Comment 5

[Boris Zbarsky [:bzbarsky]]

Attachment: Better locally runnable

Comment 6

[Boris Zbarsky [:bzbarsky]]

OK, so comparing times for Firefox with timeout clamp at 4ms and Chrome 10:
 
 Test      Fx        Chrome
   1     1.052       0.948
   2     3.856       3.535
   3    10.378       4.299
   4     0.239       0.15
   5     0.16        0.395
   6     0.909       0.888
   7     0.055       0.056

Some data on tests 1, 2, 3, 4, 6.

Comment 7

[Boris Zbarsky [:bzbarsky]]

Er, some data coming up.

Comment 8

[Boris Zbarsky [:bzbarsky]]

test 1:

30% of the time is running the script that's triggered by the timer firing on the string (!) that was passed to setTimeout.  The time there is almost entirely under js::mjit::stubs::SetName calling js::proxy_SetProperty, calling JSWrapper::set, etc.  The vast majority of this is ending up under xpconnect, so this is the window.status set the page does.  I wonder whether we should quickstub that.. ;)

65% of the time is painting.  Some of this will get better with painting/invalidation off the refresh driver, but I don't quite understand why we end up with the ridiculously deep stacks we get here for BildDisplayListForStackingContext, with lots of scrollframes and table stuff on the stack.  :(

Comment 9

[Boris Zbarsky [:bzbarsky]]

Test 2:

35% reflow before painting 
39% painting
 5% running the script it's ostensibly testing

lots of event loop overhead here.

Again, this will be helped by invalidation off the refresh driver.

Comment 10

[Boris Zbarsky [:bzbarsky]]

Test 3:

17% painting
16% event loop gunk
 3% running JS
 6% attribute-setting code in DOM
 3% CanLoadImage security check
 5% Converting strings to nsIURI objects
 8% imgLoader::LoadImage gunk
13% imgRequestProxy::OnStopRequest calling nsDocLoader::OnStopRequest which does
    stupid stuff.  Bug 642551 filed.
16% image loads completing and calling Invalidate() from
    nsImageFrame::OnStopDecode.  Again, bug 598482 might help.
 6% dispatching image load events.

Comment 11

[Boris Zbarsky [:bzbarsky]]

Test 4:

12% reflow before painting
24% painting
41% spellchecking (which chrome may or may not be running on this text before the
    test finishes, depending on their setup)
 8% what looks like a timer triggering the session save part of session restore
 6% GC
 3% running the JS, setting the values of the textareas, etc.

Comment 12

[Boris Zbarsky [:bzbarsky]]

Test 6:

38% painting
 3% cycle collection
 3% session restore stuff
15% restyle processing (ReResolveStyleContext, CalcStyleDifference,
    ResolveStyleFor, etc)
 7% reflow
10% setting inline style.
 5% getting inline style
 3% eval() crud (more on this below).
 3% other JS

The eval comes from this gem:

  var thisspan=eval("document.getElementById('span'+i).style")

that runs for every piece of text for every move.

I looked at tests 5 and 7 briefly too.  Test 5 performance is dominated by the innerHTML setter actually parsing the HTML for us, and a bit of GC.  Test 7 seems to be 31% session restore saving the session, 12% GC, and most of the remainder js_Date, date_getDay, js_ConcatStrings, js_StringToNumber, that sort of thing.

Comment 13

[Guilherme Lima]

Test 3 is ridiculously slow on Nightly!
The others are fine.

Comment 14

[Guilherme Lima]

Ok, tested correctly after the results from others on Mozillazine Forums.
TEST 3 68.514 sec. (adblock plus + noscript)
TEST 3 32.135 sec. (only noscript)
TEST 3 9.216 sec. (without both)

Chrome: TEST 3 4.443 sec.

Comment 15

[Giorgio Maone [:ma1]]

(In reply to Guilherme Lima from comment #14)

> TEST 3 68.514 sec. (adblock plus + noscript)
> TEST 3 32.135 sec. (only noscript)
> TEST 3 9.216 sec. (without both)

Almost the only thing NoScript and Adblock Plus have in common about this test is being called by CanLoadImage(), with NoScript's implementation usually slightly faster especially for images.

"Test 3 replaces 1500 images as fast as possible. It repeats the procedure 60 times."

The result of turning an exclusively C++ almost NOP into a full fledged XPCOM->JavaScript call possibly doing a lot of regular expressions matching, and repeating this call 90000 times, is surprisingly fast IMHO.

Comment 16

[Guilherme Lima]

Thanks Maone. Probably there's nothing useful to be done here, as it's not something we see on regular websites?
I see the problem with ABP was filled last year - bug 756805, so I'm removing the needinfo, sorry for bothering.

Comment 17

[Gregory Pappas [:gregp]]

Firefox Nightly 112.0a1 (2023-02-26):
TEST 1 time: 0.852 sec.
TEST 2 time: 3.416 sec.
TEST 3 time: 2.526 sec.
TEST 4 time: 0.04 sec.
TEST 5 time: 0.039 sec.
TEST 6 time: 0.581 sec.
TEST 7 time: 0.023 sec.

Chromium 113.0.5616.0 (2023-02-25):
TEST 1 time: 0.847 sec.
TEST 2 time: 3.369 sec.
TEST 3 time: 3.562 sec.
TEST 4 time: 0.071 sec.
TEST 5 time: 0.032 sec.
TEST 6 time: 0.8 sec.
TEST 7 time: 0.019 sec.

Seems good now.