Closed Bug 1465360 Opened Last year Closed Last year

add asm.js-apps shell benchmark to jsshell-bench

Categories

(Testing :: General, enhancement)

enhancement
Not set

Tracking

(firefox62 fixed)

RESOLVED FIXED
mozilla62
Tracking Status
firefox62 --- fixed

People

(Reporter: jmaher, Assigned: jmaher)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

another test we want to run as shell only is asm.js-apps.

This runs in arewefastyet.

The problem is the source is 16MB+, we should figure out if we want to check this in to the tree, or download it from tooltool.
:ahal, do you have thoughts on how to deal with a large volume of test data?  we currently have some tests that download artifacts from tooltool, not sure if that works in the 'source-test' model.
Flags: needinfo?(ahal)
We should be able to download it from tooltool with a source-test task no problem. There are several options:

1) Do it directly in the task's command (in the .yml file)
2) Have the mach command do it
3) Do it in the run-task script somewhere (more work, but will make it easier for future tasks)

For now option 1) would be easiest, but option 3) might be worthwhile (being able to download stuff from tooltool in the general case seems useful). I guess the main thing is making sure we have access to the 'tooltool.py' binary, but I think in other places we just wget it from hg.m.o, so we could do that here as well.
Flags: needinfo?(ahal)
here is a try push showing the "profit" phase:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3f086fe473d3f2c19e2aae21595f29c699d8bc1d
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8985420 - Flags: review?(ahal)
Attachment #8985664 - Attachment is obsolete: true
Comment on attachment 8985420 [details] [diff] [review]
add asmjs apps as a jsshell-bench

Review of attachment 8985420 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me. I do have a WIP patch to support downloading the benchmarks using "fetch" tasks in CI. However, I can rebase my work on top of this and we can decide to land when/if the PerformanceTests directory becomes too huge. My patch doesn't yet solve the run locally case however.
Attachment #8985420 - Flags: review?(ahal) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/010e2e111043
add asm.js-apps shell benchmark to jsshell-bench. r=ahal
https://hg.mozilla.org/mozilla-central/rev/010e2e111043
Status: ASSIGNED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.