Closed Bug 1542046 Opened 7 months ago Closed 2 months ago

Allow running JetStream2 via Raptor

Categories

(Testing :: Raptor, enhancement, P1)

Version 3
enhancement

Tracking

(firefox70 fixed)

RESOLVED FIXED
mozilla70
Tracking Status
firefox70 --- fixed

People

(Reporter: mgaudet, Assigned: marauder)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Jetstream2 is a new benchmark from Apple:

It would be nice if we could use it in the same manner as we can use Speedometer

Type: defect → enhancement

(If this needs a PI request, let us know how to make that the best way; the JIRA workflow documented on Mana doesn't seem to include test automation) .

Thank you Matthew. I was on PTO so just setting this now. NI'ing :davehunt re: PI request.

Flags: needinfo?(dave.hunt)

Whoops dropped the ball: sphilp filed a PI request for this here: https://moz-pi-test.atlassian.net/browse/PTE-90 (I can't actually see said link, but it's the link they shared :) )

Flags: needinfo?(dave.hunt)
Priority: -- → P3
Assignee: nobody → marian.raiciof
Status: NEW → ASSIGNED

I started the changes on my forked github repository and did a pull request:
https://github.com/mozilla/perf-automation/pull/20/commits/
Robert when you have some spare time, i need your review on this.
Thanks!

Flags: needinfo?(rwood)

Changed the required yml, ini, json, js and html files to add Jetstream2 benchmark test to Raptor.
The changes on the javascript files are on github.

Flags: needinfo?(rwood)
Priority: P3 → P1

(In reply to Marian Raiciof [:marauder] from comment #4)

I started the changes on my forked github repository and did a pull request:
https://github.com/mozilla/perf-automation/pull/20/commits/
Robert when you have some spare time, i need your review on this.
Thanks!

Great work thanks! Code looks good!

Tried it out locally - applied pull request, started a local webserver, and invoked via:

http://localhost:8081/JetStream2/index.html?raptor

The benchmark ran and completed fine. I merged the PR.

Hi Robert,
Dave created a release for JetStream2 because i didn't have permissions to do it.
Can you double check if all is fine ?
https://github.com/mozilla/perf-automation/releases/tag/jetstream2-v1
Thanks!

Flags: needinfo?(rwood)

(In reply to Marian Raiciof [:marauder] from comment #7)

Hi Robert,
Dave created a release for JetStream2 because i didn't have permissions to do it.
Can you double check if all is fine ?
https://github.com/mozilla/perf-automation/releases/tag/jetstream2-v1
Thanks!

Yep LGTM - you're almost there (see comments in phab) thanks for your hard work on this!

Flags: needinfo?(rwood)

Jetstream2 test did pass on linux machines but it was failing on Windows and OS X.
After i did several tests on a Windows 10 laptop, i realised that the time to get the metrics is terrible slow compared to linux.
I increased the time for a single pagecycle to 2000 seconds. And the overall time to 8000 seconds (4 pagecyles in total).

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=d89f94f89e4ed8c269c75ac27a40c6693929dbcf

Pushed by mraiciof@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6aff093ae164
Allow running JetStream2 via Raptor r=rwood,perftest-reviewers
Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
You need to log in before you can comment on or make changes to this bug.