975403 - Support chunking in jit-test harness

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Dan Minor [:dminor]]

The jit-tests run slow on the linux test machines (bug 973946). I think the short term solution to this is to add chunking support to the harness.

What we've done for other harnesses is to add two additional options:
--this-chunk and --total-chunks. There harness will then split the list of all tests based upon the total number of chunks and only run the chunk specified by --this-chunk.

We'll eventually need this for the jit-tests on Android. Last I checked, we're now taking 158 minutes to run the tests there, and we're not going to magically get faster Android test machines.

Comment 1

[Dan Minor [:dminor]]

Attachment: Add chunking support.

Comment 2

[Terrence Cole [:terrence]]

Comment on attachment 8379748 [details] [diff] [review]
Add chunking support.

Review of attachment 8379748 [details] [diff] [review]:
-----------------------------------------------------------------

Neat! r=me

::: js/src/jit-test/jit_test.py
@@ +169,5 @@
> +    if options.total_chunks > 1:
> +        total_tests = len(test_list)
> +        tests_per_chunk = math.ceil(total_tests/float(options.total_chunks))
> +        start = int(round((options.this_chunk - 1)*tests_per_chunk))
> +        end = int(round(options.this_chunk*tests_per_chunk))

Add spaces around all of the * and / operators too.

@@ +171,5 @@
> +        tests_per_chunk = math.ceil(total_tests/float(options.total_chunks))
> +        start = int(round((options.this_chunk - 1)*tests_per_chunk))
> +        end = int(round(options.this_chunk*tests_per_chunk))
> +        if end > total_tests:
> +            end = total_tests

Python slices are explicitly smart enough to handle this case automatically.

>>> [0, 1, 2][1:100000]
[1, 2]

I think it's clear enough from context (e.g. the code in question will be actively used in production) that having the manual bounds adjustment is probably not necessary.

Comment 3

[Dan Minor [:dminor]]

Thanks! With comments addressed: https://hg.mozilla.org/integration/mozilla-inbound/rev/ca9e7deea2fa

Comment 4

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/ca9e7deea2fa