1585464 - Fix / unskip all browser chrome tests for Fission

Status: RESOLVED FIXED

(Remote Protocol :: Agent, task, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

There are a couple of browser chrome tests for the Remote protocol marked as skipped when Fission is enabled. We should get those unskipped, or fixed.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Pushed a try build with all fission skipped lines removed. Lets see how it looks like:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=adae5d249b73283fd194f8ef7a124c69c3968058

Comment 2

[Andreas Tolfsen ❲:ato❳]

Most of the tests pass with --enable-fission in both release and
debug builds, apart from these two which consistently fail in both:

• remote/test/browser/browser_page_frameNavigated.js
• remote/test/browser/browser_page_frameNavigated_iframe.js

The issue is to do with frame scripts not being reloaded when the
content browser undergoes a process/remoteness change, I think.
This has all sorts of other problems tied to it, such as the fact
that we don’t currently have any infrastructure for reloading/resuming
the state if the frame scripts were reloaded.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I doubt that this is the problem here. Note that this issue only manifests on Linux and Windows, but not MacOS. I will have to do further investigation on that.

In the meantime I will already push a revision to unskip all the other remote protocol tests which work fine with Fission enabled. We can get this already landed while investigating and fixing the navigation test issues.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

When having a look at the logs I can see the following:

On MacOS where the tests are passing we have the following results:

[task 2019-10-02T09:31:48.381Z] 09:31:48 INFO - TEST-OK | remote/test/browser/browser_page_frameNavigated.js | took 2699ms
..
[task 2019-10-02T09:31:50.876Z] 09:31:50 INFO - TEST-OK | remote/test/browser/browser_page_frameNavigated_iframe.js | took 2408ms

Means both tests take about 2.5s to run through all the individual steps.

On Linux and Windows where it times out I can see a delay of more than 1s while Firefox is starting up. Specifically after the DevTools WebSocket has been started listening:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=269388825&repo=try&lineNumber=1184-1202

[task 2019-10-02T09:38:48.037Z] 09:38:48     INFO - TEST-START | remote/test/browser/browser_page_frameNavigated.js
[task 2019-10-02T09:38:48.352Z] 09:38:48     INFO - GECKO(3516) | DevTools listening on ws://localhost:9222/devtools/browser/cb02eec1-0f11-448a-8158-a461b0429adf
[task 2019-10-02T09:39:33.054Z] 09:39:33     INFO - TEST-INFO | started process screenshot
[task 2019-10-02T09:39:33.114Z] 09:39:33     INFO - TEST-INFO | screenshot: exit 0
[task 2019-10-02T09:39:33.115Z] 09:39:33     INFO - Buffered messages logged at 09:38:48
[task 2019-10-02T09:39:33.115Z] 09:39:33     INFO - Entering test bound 
[task 2019-10-02T09:39:33.115Z] 09:39:33     INFO - CDP client instantiated
[task 2019-10-02T09:39:33.116Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | Page domain has been enabled - 
[task 2019-10-02T09:39:33.116Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | getFrameTree exposes one frame - 
[task 2019-10-02T09:39:33.116Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | getFrameTree reports no child frame - 
[task 2019-10-02T09:39:33.117Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | getFrameTree's frame has an id - 
[task 2019-10-02T09:39:33.117Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | getFrameTree's frame has the right url - 
[task 2019-10-02T09:39:33.117Z] 09:39:33     INFO - Test Page.navigate
[task 2019-10-02T09:39:33.118Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | A new page has been loaded - 
[task 2019-10-02T09:39:33.118Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | Page.navigate returned a frameId - 
[task 2019-10-02T09:39:33.118Z] 09:39:33     INFO - TEST-PASS | remote/test/browser/browser_page_frameNavigated.js | The Page.navigate's frameId is the same than getFrameTree's one - 
[task 2019-10-02T09:39:33.118Z] 09:39:33     INFO - Console message: [JavaScript Error: "The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol." {file: "http://example.com/browser/remote/test/browser/doc_page_frameNavigated.html" line: 0}]
[task 2019-10-02T09:39:33.118Z] 09:39:33     INFO - Buffered messages finished
[task 2019-10-02T09:39:33.118Z] 09:39:33     INFO - TEST-UNEXPECTED-FAIL | remote/test/browser/browser_page_frameNavigated.js | Test timed out - 

Because of this delay the test will be aborted with a timeout in the middle of its execution.

This gives me a feeling that whatever those navigation tests are doing during setup might be a problem here.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1585464 - [remote] Unskip passing browser-chrome tests with Fission enabled. r=#remote-protocol-reviewers

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As it looks like bug 1547961 suffers from the same problem. So adding it to the dependency list.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The navigation tests will actually be fixed via bug 1585630.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c9d3a971833f
[remote] Unskip passing browser-chrome tests with Fission enabled. r=remote-protocol-reviewers,ato

Comment 9

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/c9d3a971833f

Comment 10

[Chris Peterson [:cpeterson]]

Retroactively moving fixed bugs whose summaries mention "Fission" (or other Fission-related keywords) but are not assigned to a Fission Milestone to an appropriate Fission Milestone.

This will generate a lot of bugmail, so you can filter your bugmail for the following UUID and delete them en masse:

0ee3c76a-bc79-4eb2-8d12-05dc0b68e732