523654 - Enable JS RefTest Tests on Tracemonkey Maemo Builds

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect, P4)

Description

[cmtalbert]

We need to turn on the JS RefTest browser JS tests on for Maemo builds on the Tracemonkey tree to get some amount of Javascript testing coverage for ARM.  Currently the only JS tests we have are make check tests which must be run on the build system directly and cannot therefore be run on mobile devices.  

Setting up these tests should be nearly identical to running reftests.

See bug 519213 for the related but different effort to turn on these tests for normal x86 builds.

Comment 1

[Chris Cooper [:coop] (he/him)]

Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

Clint, is this still needed and important on Maemo or should this effort be directed to desktop or android build testing?

Comment 3

[cmtalbert]

We still need javascript testing on ARM chips.  Since Android is more of a focus at this point, I think it makes sense to press forward with these tests on the tegras.  But, I think jsreftest work is behind the ongoing talos and mochitest work on the tegras.  So, if we can do this parallel on maemo in the meantime, that'd be a win.  

So, yes, it's definitely needed.  I would venture to guess that having this testing on android is higher priority than having it on maemo, but if it's simple to switch on for maemo, we might as well throw the switch now and get something running in parallel while we bring up android support.

Comment 4

[John Ford [:jhford] CET/CEST Berlin Time]

Joel,  is it possible to get maemkit to run this?  If so, how would I launch these tests using it?

Is the list of valid --testtypes for maemkit-chunked.py specified in the maemkit.cfg file?

If so, do you think something like the following would be likely to work?

[jsreftest]
testroot = /builds/fennec/jsreftest/tests
utility-path = /builds/fennec/reftest
manifest = /builds/fennec/jsreftest/tests/jstests.list
log-file = /builds/fennec/log_jsreftest.txt
verbose = 1
no-pieces = true

Comment 5

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: patch to enable jsreftests once in maemkit

Waiting on maemkit harness support, but this should be what is required

Comment 6

[Aki Sasaki (not active)]

Comment on attachment 518225 [details] [diff] [review]
patch to enable jsreftests once in maemkit

Assuming it's called 'jsreftest', that looks right.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #6)
> Comment on attachment 518225 [details] [diff] [review]
> patch to enable jsreftests once in maemkit
> 
> Assuming it's called 'jsreftest', that looks right.

according to the bug that I filed to track the maemkit work, it will be :)

Comment 8

[Chris Cooper [:coop] (he/him)]

How far out is the maemkit work in bug 640404?

I would suggest that the tegras are green enough now that they would be a better target for this effort.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

I don't know if we are interested in turning on new tests for maemo.  The dev team is really focused on android and desktop linux this quarter.

Comment 10

[Chris Cooper [:coop] (he/him)]

(In reply to comment #9)
> I don't know if we are interested in turning on new tests for maemo.  The dev
> team is really focused on android and desktop linux this quarter.

...and if we wait any longer than that, Maemo may be a non-issue. OK, WONTFIXing.

There is debug testing work for tegras lining up in bug 649690, and that specifically mentions js reftests. Please let me know if we need to file a more targeted bug though.