[e10s] Can't complete the test262 suite

NEW
Unassigned

Status

()

P3
major
3 years ago
5 months ago

People

(Reporter: alex.fdm, Unassigned)

Tracking

(Blocks: 1 bug, {triage-deferred})

50 Branch
x86_64
Windows 10
triage-deferred
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(e10s+)

Details

(URL)

(Reporter)

Description

3 years ago
I am trying to run the test262 suite hosted here:
http://v8.github.io/test262/website/default.html

I am using Firefox Developer Edition (v49.0a2), which cannot complete the test. halfway through the test it stops painting some areas of the tab running the test (they become black), and later the browser stops responding completely.
(Reporter)

Comment 1

3 years ago
To see the painting issue, minimize the window running the test and let it run for a while.  When the window is restored, there are a lot of black areas.  Moving the cursor around the page will make some elements to appear, but not all.
(Reporter)

Comment 2

3 years ago
I have now tried this with v50.0a2, and still have the same issue.  Actually, instead of hanging the browser it now says "A web page is slowing down your browser.  What you would like to do?".  Maybe this happens with 49.0a2 too, I left it running overnight and never saw it showing this before it stopped responding.
(Reporter)

Comment 3

3 years ago
And now I am running with the current stable version (v48.0) and everything seems fine.
What can be going on:
- Some issue with electrolisys, since it is off in the stable version but on in the Dev Edition;
- Some issue with the 64bit version (I am running 32bit for stable and 64bit for Dev Edition);
- Some other issue introduced in v49;

Can I do something else to help diagnosing this?
(Reporter)

Comment 4

3 years ago
I have now disabled Electrolysis on v50.0, and it is running fine.  So I am assuming this is an Electrolysis issue. Marking it as such.
Blocks: 516752
Summary: Can't complete the test262 suite → [e10s] Can't complete the test262 suite
tracking-e10s: --- → ?
There are definitely tests that will just hang in the shell, for example built-ins/Array/prototype/slice/S15.4.4.10_A3_T1

Updated

3 years ago
tracking-e10s: ? → +

Comment 6

2 years ago
Nightly completed locked up on the "result-get-matched-err" test. It made the content process use 3GB (maximum on my Win7 machine) and use 50% of the CPU total. Had to kill the process.
Keywords: triage-deferred
Priority: -- → P3
(Reporter)

Comment 7

5 months ago
This is still happening with 63.0a1.  I believe this is a major stability issue, in a high visibility test, which may hurt Firefox in a future review or browser comparison.
I think at this point the issue should be closed as WONT FIX, because the test262 version at <http://v8.github.io/test262/website/default.html> is too old to work with current browsers (it's test262 from 2016-03-23). Both, Firefox and Chrome, hang at 10% for me, which is a good indication at least one of the tests expects some behaviour which is no longer compliant with the current ECMAScript spec.
(Reporter)

Comment 9

5 months ago
I ran the tests with Chrome and Firefox, and both hang at the same place as you said:

Firefox 61: Tests To run: 16436 | Total tests ran: 1679 | Pass: 1617 | Fail: 62 | Failed to load: 0
Firefox 63: Tests To run: 16436 | Total tests ran: 1679 | Pass: 1617 | Fail: 62 | Failed to load: 0
Chrome 68:  Tests To run: 16436 | Total tests ran: 1679 | Pass: 1617 | Fail: 25 | Failed to load: 0

I even tried to turn Electrolisys off with <browser.tabs.remote.autostart=false>, which used to fix the issue on Firefox 50 (see comment 4), but I get the same exact result, regardless of its value.

But I disagree with you, I don't think a browser hang is a reasonable result, even if the test is buggy.

I also ran the test with IE and Edge, and both are able to finish it:

IE 11:   Tests To run: 16436 | Total tests ran: 16436 | Pass: 12465 | Fail: 3971 | Failed to load: 0 
Edge 16: Tests To run: 16436 | Total tests ran: 16436 | Pass: 15665 | Fail: 771  | Failed to load: 0
I haven't investigated which test (or tests) let the browser hang, or more correctly just take a long time to complete, but there have been tests in test262 which tested the wrap around behaviour of, for example ToUint32, in some spec algorithms. Later spec revisions then changed ToUint32 to ToLength, so the input value is no longer wrapped around, but instead clapped at 2^53-1. And if test262 tests now basically had `ToUint32(4294967296)` and expected the value is wrapped to 0, but newer specs now use `ToUint32(4294967296)`, it's inevitable that tests need longer for completion in implementations which implement the ToLength call. 

ToUin32: https://tc39.github.io/ecma262/#sec-touint32
ToLength: https://tc39.github.io/ecma262/#sec-tolength
You need to log in before you can comment on or make changes to this bug.