508316 - [Buildbot] Should compile and run acceptance suite with -AS3

Status: RESOLVED FIXED

(Tamarin Graveyard :: Build Config, defect, P3)

Description

[Brent Baker]

Attachment: Failures from windows release

Should have deep testing for compiling and running the acceptance suite with -AS3. There are currently 173 known tests that do NOT pass when compiled with the AS3 class based object model (tests manipulate the prototype chain).

These failures should be accounted for in the testconfig and we should compile and test with the -AS3

Comment 1

[Trevor Baker]

Should do runs with and w/o -AS3

Comment 2

[Dan Schaffer]

moving target to future since the bug is not required for 10.1 release.

Comment 3

[Brent Baker]

I think this is being enabled in bug 523863, check patch https://bug523863.bugzilla.mozilla.org/attachment.cgi?id=408155

Comment 4

[Chris Peyer]

Patch mentioned above only adds -AS3 flag to performance suite when -optimize is specified.

Comment 5

[Trevor Baker]

status?

Comment 6

[Brent Baker]

QRB: re-target

Comment 8

[Chris Peyer]

Attachment: Changes to runtests framework

runtests.py: code cleanup and slight rearrangement - no change in functionality
runtestBase.py:

Added --addtoconfig command line switch that adds the given string to the generated config (in between os and vm args)

Added ability to specify negative args in --ascargs.  e.g: --ascargs=-no-AS3 will remove -AS3 from any compiler args if specified in an .asc_args file.

Comment 9

[Chris Peyer]

Attachment: update tests to run with -AS3

1. Compile all tests (expect ecma3 & spidermonkey) with -AS3 by using dir.asc_args files.
2. Add as3Enabled():Boolean to detect whether test was compiled with -AS3 to shell.as
3. Update as3/Vector/sort* to check for correct error depending on how it was compiled.
4. Update RuntimeErrors/Error1034TypeCoercionFailed to use code that consistently throws a 1034
5. Update testconfig for vector/map failures w/ -AS3

Comment 10

[Chris Peyer]

Attachment: buildbot: also test -ES run in deep

Buildbot changes to run -ES in deep and to run -ES in codecoverage

Comment 11

[Chris Peyer]

Attachment: updated buildbot patch

Updated buildbot patch based on brbaker feedback

Comment 12

[Chris Peyer]

Comment on attachment 482603 [details] [diff] [review]
update tests to run with -AS3

Reviewed with Brent over the phone - got the verbal r+ from him.

Comment 13

[Chris Peyer]

Pushed to redux:
http://hg.mozilla.org/tamarin-redux/rev/e523f4426e83

Comment 14

[Chris Peyer]

Attachment: Update ecma3 tests to run with -AS3

All the ecma3 tests now run with no error when compiled with -AS3.  Note that the ecma3/dir.asc_args has been removed so that we compile ecma3 with -AS3 by default.  The dir is tested in -ES mode in the deep phase (already present).

Comment 15

[Brent Baker]

Comment on attachment 485456 [details] [diff] [review]
Update ecma3 tests to run with -AS3

Can you type the as3Enabled var in the shell.as or is that going to cause issues trying to run -ES? 

- Scanned through the large patch [OK]
- Rebuilt locally                 [OK]
- Ran an acceptance pass          [FAIL]

When I recompiled all of the tests the following testcases failed with your patch, if I revert your change and rebuild the testcases pass:

as3/Definitions/Packages/IdentifierPackageNames.abc : captured output: ReferenceError: Error #1065
as3/Vector/sortInitializers.abc : captured output: Vector.sort(comparefn) | TypeError: Error #1006
as3/Vector/sort.abc : captured output: Vector.sort(comparefn) | TypeError: Error #1006


Please run patch through the sandbox for a sanity check to get all builds.

Comment 16

[Chris Peyer]

Attachment: Update ecma3 tests to run with -AS3

Updated patch that addresses comments.  Ran in sandbox and only had ppc64 jit issues which have been set to expectedfail in testconfig (Bug 608130 and Bug 608132).

Comment 17

[Brent Baker]

Comment on attachment 486780 [details] [diff] [review]
Update ecma3 tests to run with -AS3

I am going to remove the review request until it is understood what this change will do to the ecma3 suite with regards to the ability to run all of the ecma3 tests through eval() (bug #608474)

Comment 18

[Chris Peyer]

Attachment 486780 [details] [diff] does not add any as3 only constructs.  I would like to land this patch and we can then work on getting the ecma3 dir prepped for an eval run.

Comment 19

[Chris Peyer]

ecma3 patch pushed:

http://hg.mozilla.org/tamarin-redux/rev/793a96705159

Comment 20

[Tamarin Bot]

changeset: 5917:3dbacc7ffe77
user:      Brent Baker <[e-mail]>
summary:   Bug 508316: make sure to grab new precompiled test media after running an acceptance pass with -ES. (r=brbaker)

http://hg.mozilla.org/tamarin-redux/rev/3dbacc7ffe77