Closed Bug 1465360 Opened Last year Closed Last year
.js-apps shell benchmark to jsshell-bench
another test we want to run as shell only is asm.js-apps. This runs in arewefastyet. The problem is the source is 16MB+, we should figure out if we want to check this in to the tree, or download it from tooltool.
:ahal, do you have thoughts on how to deal with a large volume of test data? we currently have some tests that download artifacts from tooltool, not sure if that works in the 'source-test' model.
We should be able to download it from tooltool with a source-test task no problem. There are several options: 1) Do it directly in the task's command (in the .yml file) 2) Have the mach command do it 3) Do it in the run-task script somewhere (more work, but will make it easier for future tasks) For now option 1) would be easiest, but option 3) might be worthwhile (being able to download stuff from tooltool in the general case seems useful). I guess the main thing is making sure we have access to the 'tooltool.py' binary, but I think in other places we just wget it from hg.m.o, so we could do that here as well.
here is a try push showing the "profit" phase: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3f086fe473d3f2c19e2aae21595f29c699d8bc1d
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8985420 - Flags: review?(ahal)
Comment on attachment 8985420 [details] [diff] [review] add asmjs apps as a jsshell-bench Review of attachment 8985420 [details] [diff] [review]: ----------------------------------------------------------------- Looks good to me. I do have a WIP patch to support downloading the benchmarks using "fetch" tasks in CI. However, I can rebase my work on top of this and we can decide to land when/if the PerformanceTests directory becomes too huge. My patch doesn't yet solve the run locally case however.
Attachment #8985420 - Flags: review?(ahal) → review+
Pushed by email@example.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/010e2e111043 add asm.js-apps shell benchmark to jsshell-bench. r=ahal
You need to log in before you can comment on or make changes to this bug.