1428972 - RexBench is slower than Chrome/WebKit

Status: NEW

(Core :: JavaScript Engine, defect, P2)

Description

[Tetsuharu OHZEKI [:tetsuharu] (UTC+9)]

WebKit has RexBench to benchmark RegExp performance (https://github.com/WebKit/webkit/tree/master/PerformanceTests/RexBench).
However, SpiderMonkey is slower than V8/JSC. This is the serious problem.

## Environment

- Benchmark revision: https://github.com/WebKit/webkit/tree/1ccfc658a1e7ea0245f34b8eeebc67437f33061a/PerformanceTests/RexBench
- My Laptop spec
    - macOS Sierra 10.12.6 
    - CPU: 2.8 GHz Intel Core i7
    - RAM: 16 GB 2133 MHz LPDDR3
- Firefox: https://hg.mozilla.org/mozilla-central/rev/ca379fcca95b1f4a3744242ea8647004b99b3507
- Chrome (Canary): Version 65.0.3315.0
- WebKit (Nightly): r226612


## Benchmark Result (Overall)

- Firefox: 31.02 ±1.99ms 
- Chrome: 20.21 ±0.66ms
- WebKit: 19.76 ±0.45ms

Comment 1

[Tetsuharu OHZEKI [:tetsuharu] (UTC+9)]

WebKit folks introduced RexBench by https://bugs.webkit.org/show_bug.cgi?id=175225

Comment 2

[Nicolas B. Pierron [:nbp]]

André, do you know if this could be an integration issue caused by the recent update of the Regexp engine?

Comment 3

[André Bargull [:anba]]

No changes for the update (bug 1367105) have landed as of yet.

Comment 4

[Jean Gong :jgong]

Kannan, do you know what Engineer on JS would be able to work on this.

Comment 5

[Kannan Vijayan [:djvj]]

we use a fork of the V8 regex engine Irregexp, which was brought into our codebase a couple years back.  I assume that while the trunk regex handling code improved, our fork has not.  The first thing to investigate is just to pull in a fresh version of Irregxp and integrate it.

Since Brian Hackett did the fist Irregexp integration, he's the first to come to mind for doing the refresh.  In the meantime, we're aware that this is not going to get done in the immediate term, so is not viable for 61 release.  Moving this to 64.

Brian, can you find time this year to do this?

Comment 6

[Kannan Vijayan [:djvj]]

Forgot to update QF triage field and ping Brian.  Brian, please see comment 5.

Comment 7

[André Bargull [:anba]]

RexBench also uses derived classes, so we see lots of IonAborts when running the test suites, leading to disabling Ion compilation for scripts in OfflineAssembler, FlightPlanner, and FlightPlanner-Unicode.
 
---
andre@VBdev:~/svn/webkit/PerformanceTests/RexBench$ IONFLAGS=aborts ~/hg/mozilla-inbound/js/src/build-debug-opt-obj/dist/bin/js -e "readFile=read; runString=s=>{var g=newGlobal(); g.evaluate(s); return g};" -f cli.js
RexBench 0.93
summary:            0ms
firstIteration:     0ms
averageWorstCase:   0ms
steadyState:        0ms
Running... Flight Planner - Unicode ( 6  to go)
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86143
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86310
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86249
[IonAbort] AbortReason::PreliminaryObjects
[IonAbort] aborted @ @evaluate:85566
[IonAbort] AbortReason::PreliminaryObjects
[IonAbort] aborted @ @evaluate:85741
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86277
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86172
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86210
[IonAbort] AbortReason::PreliminaryObjects
[IonAbort] aborted @ @evaluate:85618
[IonAbort] AbortReason::PreliminaryObjects
[IonAbort] aborted @ @evaluate:85544
[IonAbort] AbortReason::PreliminaryObjects
[IonAbort] aborted @ @evaluate:85741
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86052
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86039
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:85965
[IonAbort] Disabling Ion compilation of script @evaluate:85963
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86068
[IonAbort] Disabling Ion compilation of script @evaluate:86066
[IonAbort] Unsupported opcode: superfun
[IonAbort] aborted @ @evaluate:86143
...
---

Comment 8

[André Bargull [:anba]]

When I only run Regex DNA, Octane RegExp, and UniPoker, so the tests where Ion is not disabled because of missing class jsop support, I get about the same time for Firefox 60 (27ms) and Chromium 66 (25ms). So maybe it's just another case where bug 1167472 is needed, similar to ARES-6.

Comment 9

[Brian Hackett [Laid off!]]

I doubt I will have time to work on refreshing our copy of irregexp this year.

Comment 10

[Steven DeTar [:sdetar]]

Changing QF Target to qf:p1:f67, we don't believe we can provide a fix in the FF64 time frame.