519213 - Enable JavaScript jsreftests on Tracemonkey unittests

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[Bob Clary [:bc] (inactive)]

Please enable the JavaScript jsreftests on Tracemonkey unitest machines.

bug 469718 landed the jsreftests on tracemonkey. These tests operate in a very similar fashion to the layout reftests and the crash tests. 

They are invoked via make as:

make -C $(OBJDIR) jstestbrowser

The actual command being executed is:

python $(OBJDIR)/_tests/reftest/runreftest.py \
       --extra-profile-file=js/src/tests/user.js js/src/tests/jstests.list

The output is identical to that of the reftests and crash tests.

example opt log: http://bclary.com/log/2009/09/10/jstestbrowser-opt.log
example dbg log: http://bclary.com/log/2009/09/10/jstestbrowser-debug.log

The tests are packaged into jsreftest/tests/ along with the other tests via:

make -C $(OBJDIR) package-tests

Once this is merged into mozilla-central, we will want to enable them there as well. Medium term plans are to include this on mozilla-1.9.2. Longer term plans are to possibly include this on mozilla-1.9.1.

Comment 1

[Bob Clary [:bc] (inactive)]

expected run times are < 10 minutes for opt and < 20 minutes for debug.

Comment 2

[cmtalbert]

Catlee, these tests should be done exactly the same as reftests are currently done in buildbot.  What do you need to move this forward?  Should we try to write a patch or can you take it from here?

I'm happy to write a patch -- this seems super straight forward but I have no ability to test said patch, so I'm not sure how useful that would be.

Comment 3

[Chris AtLee [:catlee]]

This looks like it should be fairly straightforward, but I won't be able to get to it right away.  When do you need this by?  Any plans to have these tests running on trunk as well?

Comment 4

[Bob Clary [:bc] (inactive)]

The sooner these tests are running on the tracemonkey tinderbox the better for the js team.  I don't want to push too hard, but as soon as you possibly can would be great.

These tests will eventually get merged from tracemonkey to mozilla-central and we would want the tests running there as well when that happens.

In addition, the tests will eventually land on mozilla-1.9.2 and mozilla-1.9.1.

Comment 5

[Bob Clary [:bc] (inactive)]

catlee, how does P4 translate into a time frame?

Comment 6

[Chris AtLee [:catlee]]

Are these to be run on all 3 build types (optimized, debug, and refcounting-enabled optimized)?

Comment 7

[Bob Clary [:bc] (inactive)]

yes please.

Comment 8

[Chris AtLee [:catlee]]

actually, never mind, we're not yet running tests on optimized or debug builds on tracemonkey!

If that's something you'd like, please file a new bug against Release Engineering to have it turned on.

Comment 9

[Bob Clary [:bc] (inactive)]

Filed Bug 524176.

Comment 10

[Chris AtLee [:catlee]]

Attachment: Add support for jsreftest command

Comment 11

[Chris AtLee [:catlee]]

Attachment: Turn on jsreftests on tracemonkey

Comment 12

[Chris AtLee [:catlee]]

Attachment: Turn on jsreftests on tracemonkey

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 408501 [details] [diff] [review]
Add support for jsreftest command

http://hg.mozilla.org/build/buildbotcustom/rev/21bbaf623cbb

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 408521 [details] [diff] [review]
Turn on jsreftests on tracemonkey

http://hg.mozilla.org/build/buildbot-configs/rev/942bfd2f5eb8

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

pm and pm02 reconfig'd to enable jsreftest on
* Linux tracemonkey test everythingelse
* OS X 10.5.2 tracemonkey test everythingelse
* WINNT 5.2 tracemonkey test everythingelse
on the TraceMonkey-Unittest tree.

Leaving open for catlee to close out.

Comment 16

[Bob Clary [:bc] (inactive)]

catlee, I think we are ready to get this running on mozilla-central. Will be possible to enable this for all 3 build types including debug? Do you need a new bug?

Comment 17

[Chris AtLee [:catlee]]

(In reply to comment #16)
> catlee, I think we are ready to get this running on mozilla-central. Will be
> possible to enable this for all 3 build types including debug? Do you need a
> new bug?

That'd be great, thanks.

Comment 18

[Bob Clary [:bc] (inactive)]

filed Bug 527100