551702 - Minefield fails more Sputnik tests compared to FX3.6

Status: RESOLVED INVALID

(Core :: JavaScript Engine, defect)

Description

[Marco Bonardo [:mak]]

I ran for curiosity the Sputnik test on current Minefield nightly, and it failed 265 tests.
The reference number they give for Firefox 3.6 is 259 failures, so looks like Minefield is failing 6 more tests.

Comment 1

[Marco Bonardo [:mak]]

Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a3pre) Gecko/20100311 Minefield/3.7a3pre

Comment 2

[Jim Jeffery not reading bug-mail 1/2/11]

My test run using latest hourly build, with dw & d2d enabled:

Total:	5246
Succeeded:	4985
Failed:	261

cset:  http://hg.mozilla.org/mozilla-central/rev/3f7f0781120f
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a3pre) Gecko/20100312 Minefield/3.7a3pre Firefox/3.6 ID:20100312053645 <- latest hourly

Comment 3

[Nickolay_Ponomarev]

I get 258 failures on 3.6.2, 261 on 3.7:

--- 551702-3.6.txt	2010-03-27 14:47:10.000000000 +0300
+++ 551702-3.7.txt	2010-03-27 14:41:34.000000000 +0300
@@ -1 +1 @@
-Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2
+Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.3a4pre) Gecko/20100326 Minefield/3.7a4pre
@@ -100,2 +99,0 @@
-S11.2.2_A4_T4 - Checking Global object case
-S11.2.3_A4_T4 - Checking Global object case
@@ -139,0 +138,5 @@
+S15.1.1.1_A2_T1 - Checking Boolean, Number, String Functions;
+S15.1.1.1_A2_T2 - Checking typeof Operator;
+S15.1.1.2_A2_T1 - Checking Boolean, Number, String Functions;
+S15.1.1.3_A2_T1 - Checking Boolean, Number, String Functions;
+S15.1.1.3_A2_T2 - Checking typeof Operator;

The new failures test things like |NaN=1; assert(Boolean(NaN) === true)|.

Comment 4

[Brendan Eich [:brendan]]

These tests should pass in any ES1-3 implementation and fail in ES5 (ECMA-262 Fifth Edition).

/be

Comment 5

[Jeff Walden [:Waldo]]

NaN is an immutable global binding in ES5 (likewise for Infinity/undefined), so if tests are relying on it being mutable, they're correct per ES3 but wrong per ES5.

Comment 6

[Jan de Mooij [:jandem]]

These tests are incorrect, so marking this invalid. FF4 does significantly better than 3.6, despite the invalid tests (from 259 to 175 failures). I believe these tests have also been removed from test262 (http://test262.ecmascript.org/).