527100 - Enable JavaScript jsreftests on mozilla-central unittests

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Bob Clary [:bc] (inactive)]

+++ This bug was initially created as a clone of Bug #519213 +++

Please enable the JavaScript jsreftests on mozilla-central unit test machines for all three build types including debug.

Comment 1

[Chris Cooper [:coop] (he/him)]

Attachment: Enable jsreftests on mozilla-central

Comment 2

[Bob Clary [:bc] (inactive)]

This will be really great to have up and running.

It will probably cause some initial orange. Can you give me some heads up when you might deploy it so I can be sure to be available? We might also want to do it during a quiet time (relatively) so any initial orange can be stamped out quickly without interfering with everyone.

Comment 3

[Chris Cooper [:coop] (he/him)]

Bob: the most "quiet" times are generally early morning. I could push this live either tomorrow (Thursday) or Friday AM EST if that works for you.

Comment 4

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

After this suite is deployed, and green, on mozilla-central, should we then enable this unittest suite on other branches? (f.e., mozilla-1.9.2, places, electrolysis, tracemonkey, TryServer)

Also, approx how long does this take to run?

Comment 5

[Bob Clary [:bc] (inactive)]

It is already running on tracemonkey although it currently is only running with opt builds.

I expect opt to take around 10 minutes, and debug around 20 minutes. This agrees with the times on tracemonkey tests.

We will want this on mozilla-1.9.2 eventually, but the relevant code hasn't landed there yet.

It would be helpful for the developers if the TryServer ran the tests (opt and debug if possible) for those branches where jsreftests has landed, currently mozilla-central and tracemonkey.

I'm not familiar with places, but I don't think we will need it there.

If the normal reftests and crashtests run on electrolysis and electrolysis periodically is synced with mozilla-central, it may be desirable to run them there, but I don't think so. bsmedberg and jgriffin will know more about feasibility and desirability. 

I don't think we will want it on 1.9.1 due to the number of prerequisites that haven't landed there.

Comment 6

[Chris Cooper [:coop] (he/him)]

Comment on attachment 411743 [details] [diff] [review]
Enable jsreftests on mozilla-central

http://hg.mozilla.org/build/buildbot-configs/rev/9820c18fc2a6

Comment 7

[Chris Cooper [:coop] (he/him)]

This is live now, and I've triggered a bunch of everythingelse unittests on mozilla-central.

Comment 8

[Chris Cooper [:coop] (he/him)]

Attachment: Enable jsreftests on mozilla-central, v2

Comment 9

[Chris Cooper [:coop] (he/him)]

Comment on attachment 411977 [details] [diff] [review]
Enable jsreftests on mozilla-central, v2

http://hg.mozilla.org/build/buildbot-configs/rev/2411629cf564

Comment 10

[Chris Cooper [:coop] (he/him)]

Seems to be working now. I already see some passing results for OS X mozilla-central test everythingelse and WINNT mozilla-central test everythingelse, and I've triggered rebuilds for all the others.

Comment 11

[Jonathan Griffin (:jgriffin)]

We should enable these on e10s, after the next merge from mozilla-central.

Comment 12

[Chris Cooper [:coop] (he/him)]

One failed jsreftest so far on the WINNT debug everythingelse builder, but other platforms seem fine.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1258048503.1258054126.7653.gz

Specifically:

REFTEST TEST-UNEXPECTED-FAIL | file:///e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/jsreftest/tests/jsreftest.html?test=js1_5/extensions/regress-371636.js | Numeric sort performance Expected value '(tint/tstr < 3)=true', Actual value '(tint/tstr < 3)=false'  item 1

bc: do you want to track that failure in a separate bug, or just reassign this bug to yourself to work on it?

Comment 13

[Bob Clary [:bc] (inactive)]

coop, I'll track that in a separate bug. thanks!