1472992 - support running v8 shell for jsshell-bench tests

Status: RESOLVED FIXED

(Testing :: General, enhancement, P1)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

currently we run some js shell benchmarks in automation.  We want to be able to compare against other js engines as well, specifically v8.  In looking into this it appears that if you build v8 (like you would jsshell/firefox), you end up with a binary called 'd8' (which is for debugging v8).  I see that is what we use in AWFY:
https://github.com/mozilla/arewefastyet/blob/71255695f7927453778135b8b190b09b75c062fb/slave/build.py#L313

possibly that code from v8builder:
https://github.com/mozilla/arewefastyet/blob/71255695f7927453778135b8b190b09b75c062fb/slave/build.py#L246

could be used for creating a fetch-task or dependent build that we can use.  As we only test on osx for shell builds, I think restricting this to the easiest build environment (osx or linux) would be ok.

Once we have the d8 binary, I would like to run our full set of existing tests we have against it so we have comparison data.

Comment 1

[Andrew Halberstadt [:ahal]]

It looks like we can get 'd8' from linux repositories via the v8-devel package (on Fedora) or libv8-dev (on Ubuntu). Do you think that would be good enough for now? I know we'd likely want to run against bleeding edge versions of v8 if at all possible, but that would involve a lot more infrastructure work.

Fwiw the version I get from the Fedora repos is the same (major and minor version but not patch) as the one currently shipping in Chrome stable.

Comment 2

[Andrew Halberstadt [:ahal]]

Hm, though the jsshell tasks don't use docker-worker, so installing packages either needs to be done at task runtime or we need to involve ops anytime we want to do it :/. Plus they don't run as root so maybe we can't even do it at runtime.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I would be fine with a hardcoded version for now (i.e. apt packages).  Maybe a fetchtask to download a binary and unpack it from google?  or do a google build in a fetch task?  I think getting something going now with basic instructions to update is adequate (lets say we update once/month).  A P2 is a better infrastructure for updating v8/d8 and google chrome.

Comment 4

[Andrew Halberstadt [:ahal]]

I wasn't able to find any pre-built binaries anywhere, though I guess we could build it locally, upload it somewhere, *and then* use a fetch task.

I wanted to avoid the need to build V8 in our CI, at least for now. If it's only once/month then build locally+upload seems like a good first step.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

checking in here on this bug, are there new updates or plans of action?

Comment 6

[Andrew Halberstadt [:ahal]]

I'm just starting now (still wrapping up the unity-webgl and my queue from PTO)

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1472992 - [jsshell] Run javascript shell benchmarks against Google V8, r=jmaher

This runs the jsshell benchmarks against Google's V8 engine in addition to spidermonkey.
Both shells will run in the same task to keep things simple and decluttered. Though we
could split them out to separate tasks at a later date if needed.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 9000023 [details]
Bug 1472992 - [jsshell] Run javascript shell benchmarks against Google V8, r=jmaher

Joel Maher ( :jmaher ) (UTC+2) has approved the revision.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2cc6ec2f7f0e
[jsshell] Run javascript shell benchmarks against Google V8, r=jmaher

Comment 10

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/2cc6ec2f7f0e