Open Bug 671557 Opened 13 years ago Updated 8 days ago

Investigate remaining skip-if(Android) and fails-if(Android) jsreftests

Categories

(Core :: JavaScript Engine, task, P5)

Product:
Core
Component:
JavaScript Engine
Platform:
x86_64
Linux
Type:
task

Tracking

()

People

(Reporter: cjones, Unassigned)

References

(Blocks 1 open bug)

Details

They were wholesale disabled without bug numbers or comments noting why. It looks like some of them are stress-regression tests that are probably straining our current tegra2 boards harder than others. That won't always be the case so it's kind of a shame to have if(Android) than if([something-more-declarative]) but probably also not worth investing more time in. I tried turning of *all* of them with try push dfa5900fe51e. That didn't work out well. The process here would need to be, incrementally turn off skip-if/fails-ifs, if the tests still fail, figure out approximately why and then file bugs or add notes to the manifest. The payoff of this probably isn't huge (which is why I filed this bug ;). I would recommend we get rid of the "Android" jsreftest var unless there are really a whole lot of android-specific things causing tests to fail.
(In reply to comment #0) > I tried turning of *all* of them with try push dfa5900fe51e. That didn't > work out well. This _any_ of those test cases work? If so, we should turn them on unconditionally now, rather than wait for the resolution of the larger problem.
Almost certainly. Want to push to try O(2^N) times to determine which are OK? I don't have that kind of time right now.
The test logs (and TBPL) list the tests which fail (these are jsreftest-1). Presumably all the other ones are good. REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.28-1.js | TDATE = new Date(946684800000);(TDATE).setMinutes(1234567);TDATE.getTime() wrong value item 86 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.28-1.js | TDATE = new Date(946684800000);(TDATE).setMinutes(1234567);TDATE.valueOf() wrong value item 87 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.28-1.js | TDATE = new Date(946684800000);(TDATE).setMinutes(1234567);TDATE.getUTCHours() wrong value item 91 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.29-1.js | TDATE = new Date(946684800000);(TDATE).setUTCMinutes(1234567);TDATE.getHours() wrong value item 98 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.35-1.js | TDATE = new Date(0);(TDATE).setUTCMonth(5,4);TDATE.getHours() wrong value item 53 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62159356800001)).getMonth() wrong value item 73 Bug 630428 - Update WinXP slaves for 2007 daylight savings time changes (ecma/Date/15.9.5.8.js tests failing on XP) REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62156764800001)).getMonth() wrong value item 78 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62154086400001)).getMonth() wrong value item 83 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62151494400001)).getMonth() wrong value item 88 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62148816000001)).getMonth() wrong value item 93 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62146137600001)).getMonth() wrong value item 98 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62143545600001)).getMonth() wrong value item 103 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-62140867200001)).getMonth() wrong value item 108 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(7775999999)).getMonth() wrong value item 133 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(10367999999)).getMonth() wrong value item 138 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(13046399999)).getMonth() wrong value item 143 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(15638399999)).getMonth() wrong value item 148 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(18316799999)).getMonth() wrong value item 153 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(20995199999)).getMonth() wrong value item 158 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(23587199999)).getMonth() wrong value item 163 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(26265599999)).getMonth() wrong value item 168 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2201212800001)).getMonth() wrong value item 193 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2198620800001)).getMonth() wrong value item 198 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2195942400001)).getMonth() wrong value item 203 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2193350400001)).getMonth() wrong value item 208 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2190672000001)).getMonth() wrong value item 213 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2187993600001)).getMonth() wrong value item 218 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2185401600001)).getMonth() wrong value item 223 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(-2182723200001)).getMonth() wrong value item 228 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(954547199999)).getMonth() wrong value item 253 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(957139199999)).getMonth() wrong value item 258 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(959817599999)).getMonth() wrong value item 263 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(962409599999)).getMonth() wrong value item 268 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(965087999999)).getMonth() wrong value item 273 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(967766399999)).getMonth() wrong value item 278 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(970358399999)).getMonth() wrong value item 283 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(973036799999)).getMonth() wrong value item 288 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1112313599999)).getMonth() wrong value item 373 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1114905599999)).getMonth() wrong value item 378 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1117583999999)).getMonth() wrong value item 383 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1120175999999)).getMonth() wrong value item 388 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1122854399999)).getMonth() wrong value item 393 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1125532799999)).getMonth() wrong value item 398 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1128124799999)).getMonth() wrong value item 403 REFTEST TEST-UNEXPECTED-FAIL | http://10.250.48.210:30068/jsreftest/tests/jsreftest.html?test=ecma/Date/15.9.5.8.js | (new Date(1130803199999)).getMonth() wrong value item 408
Yes, but the entire suite didn't finish because of timeouts, so that approach doesn't work. I'm not arguing that what you're proposing isn't worth doing, I'm just saying that the payoff is not the worth the amount of time it would take me to do it right now. If it makes you feel any better, bug 669949 removed a bunch of fails-if()'s ;).
Assignee: general → nobody
Severity: normal → S3

I think there's value here in making sure we are only skipping tests on Android if we have a good reason.

It looks like there are only about 30 tests under js/ that have skip-if(Android), and no fails-if(Android) tests, so it should be pretty easy to check each test individually to see if it still fails on Android.

Blocks: sm-testing
Type: defect → task
Priority: -- → P5
You need to log in before you can comment on or make changes to this bug.