Closed Bug 1472992 Opened Last year Closed Last year

support running v8 shell for jsshell-bench tests

Categories

(Testing :: General, enhancement, P1)

enhancement

Tracking

(firefox63 fixed)

RESOLVED FIXED
mozilla63
Tracking Status
firefox63 --- fixed

People

(Reporter: jmaher, Assigned: ahal)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

currently we run some js shell benchmarks in automation.  We want to be able to compare against other js engines as well, specifically v8.  In looking into this it appears that if you build v8 (like you would jsshell/firefox), you end up with a binary called 'd8' (which is for debugging v8).  I see that is what we use in AWFY:
https://github.com/mozilla/arewefastyet/blob/71255695f7927453778135b8b190b09b75c062fb/slave/build.py#L313

possibly that code from v8builder:
https://github.com/mozilla/arewefastyet/blob/71255695f7927453778135b8b190b09b75c062fb/slave/build.py#L246

could be used for creating a fetch-task or dependent build that we can use.  As we only test on osx for shell builds, I think restricting this to the easiest build environment (osx or linux) would be ok.

Once we have the d8 binary, I would like to run our full set of existing tests we have against it so we have comparison data.
Priority: -- → P3
Assignee: nobody → ahal
Status: NEW → ASSIGNED
Priority: P3 → P1
It looks like we can get 'd8' from linux repositories via the v8-devel package (on Fedora) or libv8-dev (on Ubuntu). Do you think that would be good enough for now? I know we'd likely want to run against bleeding edge versions of v8 if at all possible, but that would involve a lot more infrastructure work.

Fwiw the version I get from the Fedora repos is the same (major and minor version but not patch) as the one currently shipping in Chrome stable.
Flags: needinfo?(jmaher)
Hm, though the jsshell tasks don't use docker-worker, so installing packages either needs to be done at task runtime or we need to involve ops anytime we want to do it :/. Plus they don't run as root so maybe we can't even do it at runtime.
I would be fine with a hardcoded version for now (i.e. apt packages).  Maybe a fetchtask to download a binary and unpack it from google?  or do a google build in a fetch task?  I think getting something going now with basic instructions to update is adequate (lets say we update once/month).  A P2 is a better infrastructure for updating v8/d8 and google chrome.
Flags: needinfo?(jmaher)
I wasn't able to find any pre-built binaries anywhere, though I guess we could build it locally, upload it somewhere, *and then* use a fetch task.

I wanted to avoid the need to build V8 in our CI, at least for now. If it's only once/month then build locally+upload seems like a good first step.
checking in here on this bug, are there new updates or plans of action?
Flags: needinfo?(ahal)
I'm just starting now (still wrapping up the unity-webgl and my queue from PTO)
Flags: needinfo?(ahal)
This runs the jsshell benchmarks against Google's V8 engine in addition to spidermonkey.
Both shells will run in the same task to keep things simple and decluttered. Though we
could split them out to separate tasks at a later date if needed.
Comment on attachment 9000023 [details]
Bug 1472992 - [jsshell] Run javascript shell benchmarks against Google V8, r=jmaher

Joel Maher ( :jmaher ) (UTC+2) has approved the revision.
Attachment #9000023 - Flags: review+
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2cc6ec2f7f0e
[jsshell] Run javascript shell benchmarks against Google V8, r=jmaher
https://hg.mozilla.org/mozilla-central/rev/2cc6ec2f7f0e
Status: ASSIGNED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Blocks: 1481245
You need to log in before you can comment on or make changes to this bug.