Closed Bug 514492 Opened 15 years ago Closed 6 years ago

Run the VM in a (more) deterministic mode for profiling

Categories

(Tamarin Graveyard :: Virtual Machine, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX
Future

People

(Reporter: edwsmith, Unassigned)

References

Details

(Whiteboard: PACMAN, profiling-tools, Tracking)

for work-based benchmarks, the only source of noise is machine effects and timer jitter. if the GC uses a timer, then timer effects are magnified by changing the GC's behavior. extreme example: run a simple test on cachegrind, which is ~20x slower than "real time". each run can report different I1 reads (instructions executed) due to timer inputs into the GC.
Obvious fix is probably to change IncrementalMark to run for a fixed number of iterations rather than until its time slice runs out, in this kind of a mode. The number of iterations could be some useful default with a command line parameter to override it. For example -deterministic // use default iteration count -deterministic=n // set iteration count to n
fwiw, GC is not be the only source of entropy, so instruction-accurate repeatibility might be a little ways off. - must seed random # generator with something constant - tests must not use timer to affect behavior (v8 test do this) - QCache for traits & methodinfo uses random replacement - open addressing HT's use pointers as hashcodes, so OS page randomization affects hashtable probes, iteration order, size of jit'd code (short vs long branches), possibly more. each of these might be addressable in their own way. brainstorming: -deterministic (enables all defaults) -gc-incremental=N -random-seed=X
Come to think of it, since a switch along these lines would change the behavior of the GC, possibly in significant ways, would it not be equally good for most of your tests to use non-incremental GC? There already is a switch for that, -Dnoincgc (which is available in release builds as well). (For GC testing I could see us wanting to have something like -deterministic, though.)
Status: NEW → ASSIGNED
Assignee: lhansen → nobody
Status: ASSIGNED → NEW
Component: Garbage Collection (mmGC) → Virtual Machine
OS: Mac OS X → All
QA Contact: gc → vm
Hardware: x86 → All
Summary: It would be nice to run the GC in a deterministic mode for profiling → It would be nice to run the VM in a deterministic mode for profiling
Target Milestone: --- → Future
marking this PACMAN insofar as it would help make profiles more useful
Whiteboard: PACMAN
interesting methodology: if we can stamp out nondeterminism coming from inside tamarin (by not using ASLR, seeding the random # generator, and not using a timer in incgc, or not using incgc), THEN.. system noise can also be ignored by using cachegrind, and cg_diff (new diff tool) can be used to compare 2 cachegrind runs: http://blog.mozilla.com/nnethercote/2010/06/30/cg_diff-a-differential-profiling-tool/
Flags: flashplayer-qrb+
Whiteboard: PACMAN → PACMAN, profiling-tools
Summary: It would be nice to run the VM in a deterministic mode for profiling → Run the VM in a (more) deterministic mode for profiling
Converting to a tracker for running VM in more deterministic manner.
Depends on: 481931
Whiteboard: PACMAN, profiling-tools → PACMAN, profiling-tools, Tracking
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.