1465360 - add asm.js-apps shell benchmark to jsshell-bench

Status: RESOLVED FIXED

(Testing :: General, enhancement)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

another test we want to run as shell only is asm.js-apps.

This runs in arewefastyet.

The problem is the source is 16MB+, we should figure out if we want to check this in to the tree, or download it from tooltool.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:ahal, do you have thoughts on how to deal with a large volume of test data?  we currently have some tests that download artifacts from tooltool, not sure if that works in the 'source-test' model.

Comment 2

[Andrew Halberstadt [:ahal]]

We should be able to download it from tooltool with a source-test task no problem. There are several options:

1) Do it directly in the task's command (in the .yml file)
2) Have the mach command do it
3) Do it in the run-task script somewhere (more work, but will make it easier for future tasks)

For now option 1) would be easiest, but option 3) might be worthwhile (being able to download stuff from tooltool in the general case seems useful). I guess the main thing is making sure we have access to the 'tooltool.py' binary, but I think in other places we just wget it from hg.m.o, so we could do that here as well.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: add asmjs apps as a jsshell-bench

here is a try push showing the "profit" phase:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3f086fe473d3f2c19e2aae21595f29c699d8bc1d

Comment 4

[Phabricator Automation]

Attachment: Bug 1465360 - Use 'fetch' task to bootstrap asmjs-apps webkit benckmark

foobar

Comment 5

[Andrew Halberstadt [:ahal]]

Comment on attachment 8985420 [details] [diff] [review]
add asmjs apps as a jsshell-bench

Review of attachment 8985420 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me. I do have a WIP patch to support downloading the benchmarks using "fetch" tasks in CI. However, I can rebase my work on top of this and we can decide to land when/if the PerformanceTests directory becomes too huge. My patch doesn't yet solve the run locally case however.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/010e2e111043
add asm.js-apps shell benchmark to jsshell-bench. r=ahal

Comment 7

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/010e2e111043