556837 - Multi-threaded background compiler and policy (cleaned up)

Status: VERIFIED WONTFIX

(Tamarin Graveyard :: Baseline JIT (CodegenLIR), enhancement, P2)

Description

[Simon Wilkinson]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6
Build Identifier: 

This patch is a super-cleaned-up version of the background compilation work.
It is, however, against a version of the vm with the old verifier semantics (before was 413522 applied).

Essentially it combines the Scanner/DiarizingVerifier (bug 540487), PostVerfiyCompiler (bug 545094) and VMThread (bug 555765) functionality into a framework for, and implementation of, multi-threaded background compilation.

When the vm is built with AVMFEATURE_BACKGROUND_COMPILATION enabled, the old entry-point on first invocation of an AS method (MethodInfo::verify()) is re-routed to a per-AvmCore CompilationManager. The following compilation/execution modes are then supported:

- Just-in-time verify then interpret (The old RM_interp_all mode)
- Just-in-time verify with pipelined compilation (The old RM_jit_all mode)
- Just-in-time verify with pipelined compilation, only interpret $init/clinit (The old RM_mixed mode)
- Just-in-time verify then interpret, plus immediate background compilation scheduled across N compiler threads.
- Just-in-time verify then interpret, plus deferred/hot-spotted background compilation scheduled across N compiler threads.

Note:

- InvokerCompilers run on the application thread, but can proceed concurrently with the compiler threads.
- The old style of pipelined verify/jit can be invoked concurrently with compiler threads (if, for example, we needed to back-out of pinning lots of GCObjects that we might need to perform a background compile).
- With a tiny bit of work, deferred compilation on the application thread will be possible.
- The new CompilationPolicy hierarchy included with this patch is intended to eventually subsume the current ad-hoc mechanisms. I'll add the options from Rick's JITPolicyMgr patch (528375) asap.

Build flags:

AVMFEATURE_SCANNER - Pre-req of AVMFEATURE_BACKGROUND_COMPILATION. If this is enabled without AVMFEATURE_BACKGROUND_COMPILATION then the old compilation framework is used, but with the Scanner-based verifier implementation used by background compilation.
AVMFEATURE_BACKGROUND_COMPILATION - enables new the CompilationManager.

Command line flags added:

[-bc_type=[serialjit | deferred | background]]
serialjit  - perform all compilation just-in-time on the application thread.
deferred   - DISABLED! perform all compilation as deferred on the application thread.
background - perform all compilation on background threads.
The default is 'background'.

[-bc_threads N]  use N compilation threads. The default is N=1.

[-bc_hotcount N] a method must be invoked N times to be compiled; for either deferred or background compilation. Use a value of zero for immediate scheduling of background compilation. The default is N=1.

[-bc_verbose] enable verbose output from the background compilation system. (AVMPLUS_VERBOSE builds only).

Reproducible: Always

Comment 1

[Simon Wilkinson]

Attachment: Initial patch (against bug 545094)

Comment 2

[Simon Wilkinson]

Superseded by bug 600723

Comment 3

[Trevor Baker]

bulk verifying resolved !fixed issues