Open Bug 652780 (test262) Opened 9 years ago Updated 2 months ago

[meta] Tracking test262 issues

Categories

(Core :: JavaScript Engine, defect)

defect
Not set

Tracking

()

People

(Reporter: evilpie, Unassigned)

References

(Depends on 14 open bugs, Blocks 1 open bug, )

Details

(Keywords: meta)

Attachments

(3 files, 7 obsolete files)

No description provided.
Depends on: 635017
Alias: test262
Depends on: 636989
Depends on: 665835
Depends on: 668024
Attached file Reasonably-current failures list (obsolete) —
This was the set of failures I got with a near-pristine TM build yesterday.  It could use categorization.  So far I've been picking a test at a time to investigate and fix, so I don't know exactly how many issues this represents.  I'm reasonably confident most of the Object.defineProperty failures are due to [].length not being redefinable through that method, so it'd probably be best for anyone looking to categorize these to start with the non-O.dP tests first.
Depends on: 598996
Depends on: 668390
Depends on: 668934
Depends on: 668936
Attached file Failures list, now mostly categorized (obsolete) —
We have some bugs, the tests have some bugs (to be reported shortly, need to open up a browser that has my test262 bugzilla password saved), about as you'd expect.

I haven't fully investigated most of the tests that appear to be [].length redefinition, and arguments bizarreness oddities.  I don't think there's much to gain investigating the former -- we know what they are in general, and the few that aren't, we can deal with after fixing the bulk of them, with much less effort.  There's also a couple decodeURI tests that might be fixed by a reviewed patch to be landed soon for bug 660612, but given those tests' complexity I'm uninterested in investigating until I know that particular problem exists post-patch.  Other than that everything's investigated, with bugs filed.
Attachment #542595 - Attachment is obsolete: true
Some of the tests wrongly expect that U+0085 NEXT LINE is a space character.  I filed <https://bugs.ecmascript.org/show_bug.cgi?id=118> to cover that.
Depends on: 669766
Depends on: 669246
Depends on: 520095
S13.0_A5 checks that the engine doesn't implement destructuring parameters to function statements, which we do implement.  I filed https://bugs.ecmascript.org/show_bug.cgi?id=133 on getting that bit of extension-testing removed from the test.
Depends on: 331773
Depends on: 649567
Attached file Failures list, categorized and updated (obsolete) —
This updates waldo's list taking into account what we've fixed and what was fixed in test262 upstream.

I've removed these from waldo's categories, since all the tests that rely on them no longer fail:
bug 668024 (actually, this hasn't landed, but I tested with that patch included)
bug 652771
bug 635017
bug 668934
bug 660612
Buggy global object in our DOM (for undefined, NaN, Infinity writing/deletion)
Bug 331773 (isn't needed for test262; I guess bug 660612 fixed that.)


There are the tests which no longer fail:

11.13.1-4-27-s
11.13.1-4-3-s
11.4.1-4.a-8-s
15.2.3.6-4-354-13
15.2.3.6-4-354-4
15.2.3.6-4-360-3
15.2.3.6-4-360-7
15.2.3.6-4-531-13
15.2.3.6-4-531-4
15.2.3.6-4-538-3
15.2.3.6-4-538-7
15.2.3.7-6-a-24
15.5.4.20-3-2
15.5.4.20-3-3
15.5.4.20-3-4
15.5.4.20-3-5
15.5.4.20-3-6
S15.1.2.2_A2_T10
S15.1.2.3_A2_T10
S15.1.3.1_A2.4_T1
S15.1.3.2_A2.4_T1
S15.10.6.2_A12
S15.10.6.2_A1_T16
S15.10.6.3_A1_T16
S15.4.4.12_A2_T1
S15.4.4.12_A2_T4
S15.4.4.12_A3_T2
S15.4.4.12_A3_T3
S15.5.4.10_A1_T4
S15.5.4.10_A1_T6
S15.5.4.10_A1_T7
S15.5.4.10_A1_T8
S15.5.4.10_A1_T9
S15.5.4.12_A1_T4
S9.3.1_A2
S9.3.1_A3_T1
S9.3.1_A3_T2
Attachment #543562 - Attachment is obsolete: true
No longer depends on: 331773
Nice :>
Depends on: 413155
Attached file updated for recent test262 updates (obsolete) —
Test262 has fixed a lot of the problems with their tests, and we've fixed a few here, so the list is now a bit shorter.
Attachment #550813 - Attachment is obsolete: true
Depends on: 530650
Blocks: 793483
No longer blocks: 793483
Depends on: 793483
There's a regression, Firefox Nightly (21) fails a lot tests that Fx stable (18) didn't, errors say, citing: "eval within global execution context".
Since Chrome fails only 15 tests now, Safari - 8, IE10 - 8, Opera - 10 (all of them in stable versions) and Firefox Nightly fails 209 tests, I think this bug needs the parity-webkit and parity-ie flags (I'd also say parity-opera but since they're switching to WebKit it has probably lost significance).
With non-writable array length implemented (note that bug's not fixed, the patch there isn't ready, it still has issues to work through), the failure count drops to 68.

Looking at some of the tests that fail on the site, I'm not convinced all the test262 issues mentioned before have been resolved.  But investigating too much further probably should wait til bug 858381 is fixed.
Attachment #552094 - Attachment is obsolete: true
Depends on: 866222
Attached file Results on nightly post-858381 (obsolete) —
There are some new failures, that aren't present on Walden's attachment.
But there are some failures that are gone!

I have installed addons (I forgot to disable them, sorry). I hope they don't change the results.
All these Array.prototype.* ones are problems with the test suite. I reported them https://bugs.ecmascript.org/show_bug.cgi?id=1453.
A lot of new errors since last time. Ran with today's Nightly.

Latest released Chrome has 11 errors. Nightly has 73.

I also noticed that the tests ran a lot faster in Chrome than in Nightly.
By contrast 23.0.1 has 193 errors. So a good improvement compared to the latest release.
(In reply to Kai de Leeuw from comment #15)
> I also noticed that the tests ran a lot faster in Chrome than in Nightly.
I believe this is due to the test runner playing with iframes a lot rather than a raw JavaScript performance issue.
Tom's Hardware Guide's "Web Browser Grand Prix" (tracking bug 933949) includes test262.
Blocks: WBGP
Depends on: 688852
Firefox 29 and Nightly 32 fail 42 test cases. Chrome 35 fails only 5 test cases.
Assignee: general → nobody
Depends on: 603201
Firefox 33 now failing 53 test cases:

10.4.3-1-104	Strict mode should not ToObject thisArg if not an object. Strict equality operator should succeed.	Fail
10.4.3-1-106	Strict mode should not ToObject thisArg if not an object. Return type should be 'number'.	Fail
S12.1_A4_T2	Checking if execution of "y={x}" fails	Fail
S12.6.4_A15	Using block within "for-in" Expression	Fail
15.11.4.4-8-2	Error.prototype.toString return empty string when 'name' is empty string and 'msg' is undefined	Fail
15.2.3.4-1-4	Object.getOwnPropertyNames throws TypeError if 'O' is a boolean	Fail
15.2.3.4-1-5	Object.getOwnPropertyNames throws TypeError if 'O' is a string	Fail
15.2.3.4-1	Object.getOwnPropertyNames throws TypeError if type of first param is not Object	Fail
15.2.3.6-4-292-1	Object.defineProperty - 'O' is an Arguments object of a function that has formal parameters, 'name' is own property of 'O' which is also defined in [[ParameterMap]] of 'O', and 'desc' is data descriptor, test updating multiple attribute values of 'name' (10.6 [[DefineOwnProperty]] step 3 and 5.b)	Fail
15.2.3.6-4-293-2	Object.defineProperty - 'O' is an Arguments object of a function that has formal parameters, 'name' is own data property of 'O' which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown when updating the [[Value]] attribute value of 'name' which is defined as unwritable and non-configurable (10.6 [[DefineOwnProperty]] step 4 and step 5b)	Fail
15.2.3.6-4-293-3	Object.defineProperty - 'O' is an Arguments object of a function that has formal parameters, 'name' is own data property of 'O' which is also defined in [[ParameterMap]] of 'O', test TypeError is not thrown when updating the [[Value]] attribute value of 'name' which is defined as non-writable and configurable (10.6 [[DefineOwnProperty]] step 3 and step 5.b)	Fail
15.2.3.6-4-294-1	Object.defineProperty - 'O' is an Arguments object of a function that has formal parameters, 'name' is own data property of 'O' which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown when updating the [[Writable]] attribute value of 'name' which is defined as non-configurable (10.6 [[DefineOwnProperty]] step 4 and 5b)	Fail
15.2.3.6-4-295-1	Object.defineProperty - 'O' is an Arguments object of a function that has formal parameters, 'name' is own data property of 'O' which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown when updating the [[Enumerable]] attribute value of 'name' which is defined as non-configurable (10.6 [[DefineOwnProperty]] step 4 and step 5b)	Fail
15.2.3.6-4-296-1	Object.defineProperty - 'O' is an Arguments object of a function that has formal parameters, 'name' is own data property of 'O' which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown when updating the [[Configurable]] attribute value of 'name' which is defined as non-configurable (10.6 [[DefineOwnProperty]] step 4 and step 5b)	Fail
15.2.3.6-4-354-13	Object.defineProperty will update [[Value]] attribute of indexed property successfully when [[Configurable]] attribute is true and [[Writable]] attribute is false, 'O' is the global object (8.12.9 - step Note)	Fail
15.2.3.6-4-354-4	Object.defineProperty will update [[Value]] attribute successfully when [[Configurable]] attribute is true and [[Writable]] attribute is false, 'O' is the global object (8.12.9 - step Note)	Fail
15.2.3.6-4-360-3	ES5 Attributes - Updating data property 'P' whose attributes are [[Writable]]: false, [[Enumerable]]: true, [[Configurable]]: true to an accessor property, 'O' is the global object (8.12.9 - step 9.b.i)	Fail
15.2.3.6-4-360-7	ES5 Attributes - Updating indexed data property 'P' whose attributes are [[Writable]]: false, [[Enumerable]]: true, [[Configurable]]: true to an accessor property, 'O' is the global object (8.12.9 - step 9.b.i)	Fail
15.2.3.6-4-531-13	Object.defineProperty will update [[Get]] and [[Set]] attributes of indexed accessor property 'P' successfully when [[Configurable]] attribute is true, 'O' is the global object (8.12.9 step 11)	Fail
15.2.3.6-4-531-17	ES5 Attributes - Updating an indexed accessor property 'P' using simple assignment is successful, 'O' is the global object (8.12.5 step 5.b)	Fail
15.2.3.6-4-531-4	Object.defineProperty will update [[Get]] and [[Set]] attributes of named accessor property 'P' successfully when [[Configurable]] attribute is true, 'O' is the global object (8.12.9 step 11)	Fail
15.2.3.6-4-538-3	ES5 Attributes - Updating a named accessor property 'P' whose [[Configurable]] attribute is true to a data property is successful, 'O' is the global object	Fail
15.2.3.6-4-538-7	ES5 Attributes - Updating an indexed accessor property 'P' whose [[Configurable]] attribute is true to a data property is successful, 'O' is the global object	Fail
15.2.3.7-6-a-24	Object.defineProperties - 'O' is the global object which implements its own [[GetOwnProperty]] method to get 'P' (8.12.9 step 1 )	Fail
15.4.4.14-1-17	Array.prototype.indexOf applied to the global object	Fail
15.4.4.14-2-15	Array.prototype.indexOf - 'length' is property of the global object	Fail
15.4.4.14-9-b-i-23	Array.prototype.indexOf - This object is the global object	Fail
15.4.4.15-1-17	Array.prototype.lastIndexOf applied to the global object	Fail
15.4.4.15-2-15	Array.prototype.lastIndexOf - 'length' is property of the global object	Fail
15.4.4.15-8-b-i-23	Array.prototype.lastIndexOf - This object is the global object	Fail
15.4.4.16-2-15	Array.prototype.every - 'length' is property of the global object	Fail
15.4.4.16-7-c-i-23	Array.prototype.every - This object is an global object which contains index property	Fail
15.4.4.17-2-15	Array.prototype.some - 'length' is property of the global object	Fail
15.4.4.17-7-c-i-23	Array.prototype.some - This object is an global object which contains index property	Fail
15.4.4.18-2-15	Array.prototype.forEach - 'length' is property of the global object	Fail
15.4.4.18-7-c-i-23	Array.prototype.forEach - This object is an global object which contains index property	Fail
15.4.4.19-8-c-i-23	Array.prototype.map - This object is the global object which contains index property	Fail
15.4.4.20-2-15	Array.prototype.filter - 'length' is property of the global object	Fail
15.4.4.20-9-c-i-23	Array.prototype.filter - This object is the global object which contains index property	Fail
15.4.4.21-2-15	Array.prototype.reduce - 'length' is property of the global object	Fail
15.4.4.21-8-b-iii-1-23	Array.prototype.reduce - This object is the global object which contains index property	Fail
15.4.4.21-9-c-i-23	Array.prototype.reduce - This object is the global object which contains index property	Fail
15.4.4.22-2-15	Array.prototype.reduceRight - 'length' is property of the global object	Fail
15.4.4.22-8-b-iii-1-23	Array.prototype.reduceRight - This object is the global object which contains index property	Fail
15.4.4.22-9-c-i-23	Array.prototype.reduceRight - This object is an global object which contains index property	Fail
15.4.4.4-5-c-i-1	Array.prototype.concat will concat an Array when index property (read-only) exists in Array.prototype (Step 5.c.i)	Fail
S15.4.4.4_A3_T1	[[Prototype]] of Array instance is Array.prototype, [[Prototype] of Array.prototype is Object.prototype	Fail
S15.9.3.1_A5_T1	2 arguments, (year, month)	Fail
S15.9.3.1_A5_T2	3 arguments, (year, month, date)	Fail
S15.9.3.1_A5_T3	4 arguments, (year, month, date, hours)	Fail
S15.9.3.1_A5_T4	5 arguments, (year, month, date, hours, minutes)	Fail
S15.9.3.1_A5_T5	6 arguments, (year, month, date, hours, minutes, seconds)	Fail
S15.9.3.1_A5_T6	7 arguments, (year, month, date, hours, minutes, seconds, ms)	Fail

Test suite version: ES5 | Test suite date: 2014-09-18
(In reply to Alexandre Folle de Menezes from comment #20)
> Firefox 33 now failing 53 test cases:

This triage is at a quick skim, going from memory of previous investigation in many cases, tempered with inspection of cases where memory of the error message wasn't helpful.  So I may have gotten some of these wrong.  Still, I'm pretty confident it's generally accurate.  What it reveals is that test262 has a lot of tests that are broken, and a smallish handful that SpiderMonkey doesn't implement right, mostly in very edge-case behaviors.

These failures look bogus in light of ES6 additions (note that test262-the-site may well not be up-to-date with ES6 changes, nor even up-to-date with test262 on Github):

> S12.1_A4_T2	Checking if execution of "y={x}" fails	Fail
> S12.6.4_A15	Using block within "for-in" Expression	Fail
> 15.11.4.4-8-2	Error.prototype.toString return empty string when 'name' is
> empty string and 'msg' is undefined	Fail

These look bogus in light of ES6 changes to ES5 semantics:

> 15.2.3.4-1-4	Object.getOwnPropertyNames throws TypeError if 'O' is a boolean
> Fail
> 15.2.3.4-1-5	Object.getOwnPropertyNames throws TypeError if 'O' is a string
> Fail
> 15.2.3.4-1	Object.getOwnPropertyNames throws TypeError if type of first
> param is not Object	Fail
> S15.4.4.4_A3_T1	[[Prototype]] of Array instance is Array.prototype,
> [[Prototype] of Array.prototype is Object.prototype	Fail

These I think are bogus in a browser environment with the DOM's global object (which has interesting [[DefineOwnProperty]] and [[Set]] behaviors for "length" and indexed properties) -- they may fail, but they're not actual bugs:

> 15.2.3.6-4-354-13	Object.defineProperty will update [[Value]] attribute of
> indexed property successfully when [[Configurable]] attribute is true and
> [[Writable]] attribute is false, 'O' is the global object (8.12.9 - step
> Note)	Fail
> 15.2.3.6-4-360-7	ES5 Attributes - Updating indexed data property 'P' whose
> attributes are [[Writable]]: false, [[Enumerable]]: true, [[Configurable]]:
> true to an accessor property, 'O' is the global object (8.12.9 - step 9.b.i)
> Fail
> 15.2.3.6-4-531-13	Object.defineProperty will update [[Get]] and [[Set]]
> attributes of indexed accessor property 'P' successfully when
> [[Configurable]] attribute is true, 'O' is the global object (8.12.9 step
> 11)	Fail
> 15.2.3.6-4-538-7	ES5 Attributes - Updating an indexed accessor property 'P'
> whose [[Configurable]] attribute is true to a data property is successful,
> 'O' is the global object	Fail
> 15.2.3.6-4-531-17	ES5 Attributes - Updating an indexed accessor property 'P'
> using simple assignment is successful, 'O' is the global object (8.12.5 step
> 5.b)	Fail
> 15.4.4.14-1-17	Array.prototype.indexOf applied to the global object	Fail
> 15.4.4.14-2-15	Array.prototype.indexOf - 'length' is property of the global
> object	Fail
> 15.4.4.14-9-b-i-23	Array.prototype.indexOf - This object is the global
> object	Fail
> 15.4.4.15-1-17	Array.prototype.lastIndexOf applied to the global object	Fail
> 15.4.4.15-2-15	Array.prototype.lastIndexOf - 'length' is property of the
> global object	Fail
> 15.4.4.15-8-b-i-23	Array.prototype.lastIndexOf - This object is the global
> object	Fail
> 15.4.4.16-2-15	Array.prototype.every - 'length' is property of the global
> object	Fail
> 15.4.4.16-7-c-i-23	Array.prototype.every - This object is an global object
> which contains index property	Fail
> 15.4.4.17-2-15	Array.prototype.some - 'length' is property of the global
> object	Fail
> 15.4.4.17-7-c-i-23	Array.prototype.some - This object is an global object
> which contains index property	Fail
> 15.4.4.18-2-15	Array.prototype.forEach - 'length' is property of the global
> object	Fail
> 15.4.4.18-7-c-i-23	Array.prototype.forEach - This object is an global object
> which contains index property	Fail
> 15.4.4.19-8-c-i-23	Array.prototype.map - This object is the global object
> which contains index property	Fail
> 15.4.4.20-2-15	Array.prototype.filter - 'length' is property of the global
> object	Fail
> 15.4.4.20-9-c-i-23	Array.prototype.filter - This object is the global object
> which contains index property	Fail
> 15.4.4.21-2-15	Array.prototype.reduce - 'length' is property of the global
> object	Fail
> 15.4.4.21-8-b-iii-1-23	Array.prototype.reduce - This object is the global
> object which contains index property	Fail
> 15.4.4.21-9-c-i-23	Array.prototype.reduce - This object is the global object
> which contains index property	Fail
> 15.4.4.22-2-15	Array.prototype.reduceRight - 'length' is property of the
> global object	Fail
> 15.4.4.22-8-b-iii-1-23	Array.prototype.reduceRight - This object is the
> global object which contains index property	Fail
> 15.4.4.22-9-c-i-23	Array.prototype.reduceRight - This object is an global
> object which contains index property	Fail
> 15.4.4.4-5-c-i-1	Array.prototype.concat will concat an Array when index
> property (read-only) exists in Array.prototype (Step 5.c.i)	Fail

I suspect DOM spec choices make these tests invalid, but I haven't deeply investigated them to be certain:

> 15.2.3.6-4-354-4	Object.defineProperty will update [[Value]] attribute
> successfully when [[Configurable]] attribute is true and [[Writable]]
> attribute is false, 'O' is the global object (8.12.9 - step Note)	Fail
> 15.2.3.6-4-360-3	ES5 Attributes - Updating data property 'P' whose
> attributes are [[Writable]]: false, [[Enumerable]]: true, [[Configurable]]:
> true to an accessor property, 'O' is the global object (8.12.9 - step 9.b.i)
> Fail
> 15.2.3.6-4-531-4	Object.defineProperty will update [[Get]] and [[Set]]
> attributes of named accessor property 'P' successfully when [[Configurable]]
> attribute is true, 'O' is the global object (8.12.9 step 11)	Fail
> 15.2.3.6-4-538-3	ES5 Attributes - Updating a named accessor property 'P'
> whose [[Configurable]] attribute is true to a data property is successful,
> 'O' is the global object	Fail
> 15.2.3.7-6-a-24	Object.defineProperties - 'O' is the global object which
> implements its own [[GetOwnProperty]] method to get 'P' (8.12.9 step 1 )	Fail

These failures are valid:

> 10.4.3-1-104	Strict mode should not ToObject thisArg if not an object.
> Strict equality operator should succeed.	Fail
> 10.4.3-1-106	Strict mode should not ToObject thisArg if not an object.
> Return type should be 'number'.	Fail
> 15.2.3.6-4-292-1	Object.defineProperty - 'O' is an Arguments object of a
> function that has formal parameters, 'name' is own property of 'O' which is
> also defined in [[ParameterMap]] of 'O', and 'desc' is data descriptor, test
> updating multiple attribute values of 'name' (10.6 [[DefineOwnProperty]]
> step 3 and 5.b)	Fail
> 15.2.3.6-4-293-2	Object.defineProperty - 'O' is an Arguments object of a
> function that has formal parameters, 'name' is own data property of 'O'
> which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown
> when updating the [[Value]] attribute value of 'name' which is defined as
> unwritable and non-configurable (10.6 [[DefineOwnProperty]] step 4 and step
> 5b)	Fail
> 15.2.3.6-4-293-3	Object.defineProperty - 'O' is an Arguments object of a
> function that has formal parameters, 'name' is own data property of 'O'
> which is also defined in [[ParameterMap]] of 'O', test TypeError is not
> thrown when updating the [[Value]] attribute value of 'name' which is
> defined as non-writable and configurable (10.6 [[DefineOwnProperty]] step 3
> and step 5.b)	Fail
> 15.2.3.6-4-294-1	Object.defineProperty - 'O' is an Arguments object of a
> function that has formal parameters, 'name' is own data property of 'O'
> which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown
> when updating the [[Writable]] attribute value of 'name' which is defined as
> non-configurable (10.6 [[DefineOwnProperty]] step 4 and 5b)	Fail
> 15.2.3.6-4-295-1	Object.defineProperty - 'O' is an Arguments object of a
> function that has formal parameters, 'name' is own data property of 'O'
> which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown
> when updating the [[Enumerable]] attribute value of 'name' which is defined
> as non-configurable (10.6 [[DefineOwnProperty]] step 4 and step 5b)	Fail
> 15.2.3.6-4-296-1	Object.defineProperty - 'O' is an Arguments object of a
> function that has formal parameters, 'name' is own data property of 'O'
> which is also defined in [[ParameterMap]] of 'O', test TypeError is thrown
> when updating the [[Configurable]] attribute value of 'name' which is
> defined as non-configurable (10.6 [[DefineOwnProperty]] step 4 and step 5b)
> Fail

Memory suggests this is valid, and easily fixt if someone investigates it:

> S15.9.3.1_A5_T1	2 arguments, (year, month)	Fail
> S15.9.3.1_A5_T2	3 arguments, (year, month, date)	Fail
> S15.9.3.1_A5_T3	4 arguments, (year, month, date, hours)	Fail
> S15.9.3.1_A5_T4	5 arguments, (year, month, date, hours, minutes)	Fail
> S15.9.3.1_A5_T5	6 arguments, (year, month, date, hours, minutes, seconds)
> Fail
> S15.9.3.1_A5_T6	7 arguments, (year, month, date, hours, minutes, seconds,
> ms)	Fail
(In reply to Jeff Walden [:Waldo] (remove +bmo to email) from comment #21)
> Memory suggests this is valid, and easily fixt if someone investigates it:
> 
> > S15.9.3.1_A5_T1	2 arguments, (year, month)	Fail
> > S15.9.3.1_A5_T2	3 arguments, (year, month, date)	Fail
> > S15.9.3.1_A5_T3	4 arguments, (year, month, date, hours)	Fail
> > S15.9.3.1_A5_T4	5 arguments, (year, month, date, hours, minutes)	Fail
> > S15.9.3.1_A5_T5	6 arguments, (year, month, date, hours, minutes, seconds)
> > Fail
> > S15.9.3.1_A5_T6	7 arguments, (year, month, date, hours, minutes, seconds,
> > ms)	Fail

I think the date tests are broken outside of PDT time zone:

https://bugs.ecmascript.org/show_bug.cgi?id=293
https://github.com/tc39/test262/ is now really easy to run. It's somewhat annoying that we still don't run most the tests in our automation.

 - Ran 12134 tests
 - Passed 11606 tests (95.6%)
 - Failed 528 tests (4.4%)
s/2013-08-23/2015-03-08/
@evilpie what version are you running? I just ran Test262 on Firefox 36.0.1 on Windows and got

Total tests ran: 11552 | Pass: 11449 | Fail: 103 | Failed to load: 0

Is there a newer version than whats on http://test262.ecmascript.org/? You ran more test than I did.
The website uses the "es5-tests" branch (https://github.com/tc39/test262/tree/es5-tests).
(In reply to Tom Schuster [:evilpie] from comment #23)
> https://github.com/tc39/test262/ is now really easy to run. It's somewhat
> annoying that we still don't run most the tests in our automation.

Fodder for another SM job?
Flags: needinfo?(sphink)
Flags: needinfo?(sphink)
Depends on: 1231476
Depends on: 1292635
Depends on: 1317306
Depends on: 1317307
Depends on: 1317308
Depends on: 1317309
Depends on: 1317310
Depends on: 1317311
Depends on: 884561
Depends on: 944846
Depends on: 1317373
Depends on: 1317374
Depends on: 1317375
Depends on: 1317376
Depends on: 1317378
Depends on: 1317379
Depends on: 1317382
Depends on: 1317383
Depends on: 1317384
Depends on: 1317387
Depends on: 1317389
Depends on: 1317391
Depends on: 1317394
Depends on: 1317395
Depends on: 1317397
Depends on: 1317400
No longer depends on: 1112732
Depends on: 1317405
Depends on: 1317416
Depends on: 1286997
Depends on: globalThis
Depends on: 1285372
Depends on: 1318403
Depends on: 1297179
Depends on: 1338882
Depends on: 1225839
Depends on: 1336991
Depends on: 1338884
Depends on: 1079853
Depends on: 1050755
Depends on: 1308159
Depends on: 1341692
Depends on: 1342663
Depends on: 1339395
Depends on: 1354529
Depends on: 1355399
Depends on: 1374290
Depends on: 1398839
Depends on: 1407587
Depends on: 1407588
Depends on: 1019116
Depends on: 1414049
Depends on: 1415882
Depends on: 1380881
Depends on: 1296237
Depends on: 1434007
Depends on: 1440468
Depends on: 1462741
Depends on: 1462745
Depends on: 1471465
Depends on: 1406171
Depends on: 1472211
Depends on: 1334813
Depends on: 1321616
Depends on: 605515
Depends on: 1146532
Depends on: 1508684
Attachment #543599 - Attachment is obsolete: true
Attachment #734809 - Attachment is obsolete: true
Attachment #742525 - Attachment is obsolete: true
Depends on: 1515167
Depends on: 1508683
Depends on: 1556818
Depends on: 1560300
Depends on: 1560355
See Also: → 1569381
See Also: → 1569543
Depends on: 1570245
Depends on: 1570278
Depends on: 1590539
Depends on: 1590543
Depends on: 1590552
You need to log in before you can comment on or make changes to this bug.