1301791 - Add a script to run octane N times and aggregate results as a CSV, and another script to plot that CSV data

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Comment 1

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

Attachment: Add a script to run octane N times and aggregate results as a CSV, and another script to plot that CSV data

Comment 2

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

The plot-octane.R only plots the total score at the moment. Whoever wants breakdowns per-sub-suite can extend it in the future.

Comment 3

[Steve Fink [:sfink] [:s:]]

Comment on attachment 8789931 [details] [diff] [review]
Add a script to run octane N times and aggregate results as a CSV, and another script to plot that CSV data

Review of attachment 8789931 [details] [diff] [review]:
-----------------------------------------------------------------

That's more R than I ever knew. I'll take your word for it.

::: js/src/devtools/octane-csv.sh
@@ +28,5 @@
> +if [[ "$#" != "2" ]]; then
> +    usage_and_exit
> +fi
> +
> +# Get the absolute, normalized $JS path, and ensure its an executable.

you could also be lazy and say

JS=$(realpath $1)

but I won't fault you for being explicit and covering more cases. (And I don't know whether eg osx has realpath.)

@@ +32,5 @@
> +# Get the absolute, normalized $JS path, and ensure its an executable.
> +
> +JS_DIR=$(dirname $1)
> +if [[ ! -d "$JS_DIR" ]]; then
> +    echo "error: no such directory $JS_DIR"

echo "..." 1>&2

@@ +40,5 @@
> +
> +JS=$(basename $1)
> +cd "$JS_DIR" > /dev/null
> +JS="$(pwd)/$JS"
> +if [[ ! -e "$JS" ]]; then

-x

-e is "exists"

@@ +41,5 @@
> +JS=$(basename $1)
> +cd "$JS_DIR" > /dev/null
> +JS="$(pwd)/$JS"
> +if [[ ! -e "$JS" ]]; then
> +    echo "error: '$JS' is not executable"

1>&2 again

@@ +57,5 @@
> +# Run once as a warm up, and to grab the column headers. Then run the benchmark
> +# $ITERS times, grabbing just the data rows.
> +
> +echo "Warm up" 1>&2
> +"$JS" ./run.js | grep -v "\-\-\-\-" | cut -f 1 -d ':' | tr '\n' ','

personally, I would do grep -v -- "----", but same thing

Comment 4

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Steve Fink [:sfink] [:s:] from comment #3)
> you could also be lazy and say
> 
> JS=$(realpath $1)
> 
> but I won't fault you for being explicit and covering more cases. (And I
> don't know whether eg osx has realpath.)

Unfortunately, this sees through the symlinks and stuff and ends up with libmozglue not getting found/loaded and the shell not working.

> @@ +40,5 @@
> > +
> > +JS=$(basename $1)
> > +cd "$JS_DIR" > /dev/null
> > +JS="$(pwd)/$JS"
> > +if [[ ! -e "$JS" ]]; then
> 
> -x
> 
> -e is "exists"

D'oh, thanks

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/eab4e2f9adb4
Add a script to run octane N times and aggregate results as a CSV, and another script to plot that CSV data; r=sfink

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/eab4e2f9adb4