1197055 - [M-e10s(bc3)] Permafail on win10

Reporter

Description

•

9 years ago

Brad, can you help identify an owner for this?

On try, pushing with my one win10 machine (https://treeherder.mozilla.org/#/jobs?repo=try&revision=55ecdb02f01a&exclusion_profile=false) which was based off m-i rev at https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=cb6c34882e41&exclusion_profile=false

We're failing on win10 bc2 for e10s mochitests, starting at suite-startup with:

20:17:06 INFO - 0 INFO *** Start BrowserChrome Test Results ***
20:17:06 INFO - 1 INFO checking window state
20:17:06 INFO - TEST-INFO | started process screenshot
20:17:07 INFO - TEST-INFO | screenshot: exit 0
20:17:07 INFO - 2 INFO TEST-UNEXPECTED-FAIL | browser-test.js | Found an unexpected unknown window 
 20:17:07 INFO - TEST-INFO
20:17:07 INFO - 3 INFO TEST-INFO | (browser-test.js) | Console message: [JavaScript Error: "DEPRECATION WARNING: FUEL is deprecated, you should use the add-on SDK instead.
20:17:07 INFO - You may find more details about this deprecation at: https://developer.mozilla.org/Add-ons/SDK/
20:17:07 INFO - jar:file:///C:/slave/test/build/application/firefox/browser/omni.ja!/components/fuelApplication.js 1458 Application
20:17:07 INFO - jar:file:///C:/slave/test/build/application/firefox/browser/omni.ja!/components/fuelApplication.js 726 af_ci
20:17:07 INFO - chrome://mochikit/content/browser-test.js 248 Tester_start
20:17:07 INFO - chrome://mochikit/content/browser-harness.xul 254 createTester/</<
20:17:07 INFO - null 0 null
20:17:07 INFO - " {file: "resource://gre/modules/Deprecated.jsm" line: 79}] 

Which is also interesting, because bc1 and bc3 pass.

Then there is also the next failure which is interesting:
 20:17:37 INFO - 60 INFO TEST-START | browser/base/content/test/plugins/browser_plugin_reloading.js
20:17:37 INFO - Not taking screenshot here: see the one that was previously logged
20:17:37 INFO - 61 INFO checking window state
20:17:37 INFO - 62 INFO Entering test
20:17:37 INFO - 63 INFO Leaving test
20:17:37 INFO - 64 INFO Entering test
20:17:37 INFO - 65 INFO Wait tab event: load
20:17:37 INFO - 66 INFO Tab event received: load
20:17:37 INFO - 67 INFO Console message: [JavaScript Error: "The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol." {file: "data:text/html,<html></html>" line: 0}]
20:17:37 INFO - 68 INFO TEST-PASS | browser/base/content/test/plugins/browser_plugin_reloading.js | exception: null
20:17:37 INFO - 69 INFO *** loading new blocklist: http://127.0.0.1:8888/browser/browser/base/content/test/plugins/blockNoPlugins.xml
20:17:37 INFO - 70 INFO *** waiting on local load
20:17:37 INFO - 71 INFO *** waiting on remote load
20:17:37 INFO - 72 INFO *** blocklist loaded.
20:17:37 INFO - 73 INFO Leaving test
20:17:37 INFO - 74 INFO Entering test
20:17:37 INFO - 75 INFO Wait tab event: load
20:17:37 INFO - 76 INFO Tab event received: load
20:17:37 INFO - 77 INFO TEST-PASS | browser/base/content/test/plugins/browser_plugin_reloading.js | Test 1, Should have a click-to-play notification
20:17:37 INFO - 78 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/plugins/browser_plugin_reloading.js | Test 2, plugin fallback type should be PLUGIN_CLICK_TO_PLAY - Got 3, expected 8
20:17:37 INFO - Stack trace:
20:17:37 INFO - chrome://mochikit/content/browser-test.js:test_is:931
20:17:37 INFO - chrome://mochitests/content/browser/browser/base/content/test/plugins/browser_plugin_reloading.js:null:56
20:17:37 INFO - self-hosted:InterpretGeneratorResume:750
20:17:37 INFO - self-hosted:next:658
20:17:37 INFO - Handler.prototype.process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:934:23
20:17:37 INFO - this.PromiseWalker.walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:813:7
20:17:37 INFO - Promise*this.PromiseWalker.scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:744:11
20:17:37 INFO - this.PromiseWalker.schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
20:17:37 INFO - this.PromiseWalker.completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:711:7
20:17:37 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:319:13
20:17:37 INFO - promise callback*TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:388:7
20:17:37 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:322:13
20:17:37 INFO - promise callback*TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:388:7
20:17:37 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:322:13
20:17:37 INFO - TaskImpl@resource://gre/modules/Task.jsm:275:3
20:17:37 INFO - createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:249:14
20:17:37 INFO - Task_spawn@resource://gre/modules/Task.jsm:164:12
20:17:37 INFO - TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:381:1
20:17:37 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:322:13
20:17:37 INFO - Handler.prototype.process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:934:23
20:17:37 INFO - this.PromiseWalker.walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:813:7
20:17:37 INFO - Promise*this.PromiseWalker.scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:744:11
20:17:38 INFO - this.PromiseWalker.schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
20:17:38 INFO - this.PromiseWalker.completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:711:7
20:17:38 INFO - receiveMessage@resource://testing-common/ContentTask.jsm:94:7
20:17:38 INFO - Not taking screenshot here: see the one that was previously logged
20:17:38 INFO - 79 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/plugins/browser_plugin_reloading.js | Test 3, plugin should have started - Got 4, expected 2
20:17:38 INFO - Stack trace:
20:17:38 INFO - chrome://mochikit/content/browser-test.js:test_is:931
20:17:38 INFO - chrome://mochitests/content/browser/browser/base/content/test/plugins/browser_plugin_reloading.js:null:65
20:17:38 INFO - self-hosted:InterpretGeneratorResume:750
20:17:38 INFO - self-hosted:next:658
20:17:38 INFO - Handler.prototype.process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:934:23
20:17:38 INFO - this.PromiseWalker.walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:813:7
20:17:38 INFO - Promise*this.PromiseWalker.scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:744:11
20:17:38 INFO - this.PromiseWalker.schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
20:17:38 INFO - this.PromiseWalker.completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:711:7
20:17:38 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:319:13
20:17:38 INFO - promise callback*TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:388:7
20:17:38 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:322:13
20:17:38 INFO - promise callback*TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:388:7
20:17:38 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:322:13
20:17:38 INFO - TaskImpl@resource://gre/modules/Task.jsm:275:3
20:17:38 INFO - createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:249:14
20:17:38 INFO - Task_spawn@resource://gre/modules/Task.jsm:164:12
20:17:38 INFO - TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:381:1
20:17:38 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:322:13
20:17:38 INFO - Handler.prototype.process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:934:23
20:17:38 INFO - this.PromiseWalker.walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:813:7
20:17:38 INFO - Promise*this.PromiseWalker.scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:744:11
20:17:38 INFO - this.PromiseWalker.schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
20:17:38 INFO - this.PromiseWalker.completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:711:7
20:17:38 INFO - receiveMessage@resource://testing-common/ContentTask.jsm:94:7
20:17:38 INFO - 80 INFO TEST-PASS | browser/base/content/test/plugins/browser_plugin_reloading.js | Test 4, plugin node should not be activated
20:17:38 INFO - *************************
20:17:38 INFO - A coding exception was thrown and uncaught in a Task.
20:17:38 INFO - Full message: TypeError: Components.utils.waiveXrays(...).getObjectValue is not a function
20:17:38 INFO - Full stack: @chrome://mochikit/content/tests/BrowserTestUtils/content-task.js line 22 > eval:5:18
20:17:38 INFO - TaskImpl_run@resource://gre/modules/Task.jsm:314:40
20:17:38 INFO - TaskImpl@resource://gre/modules/Task.jsm:275:3
20:17:38 INFO - createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:249:14
20:17:38 INFO - Task_spawn@resource://gre/modules/Task.jsm:164:12
20:17:38 INFO - @chrome://mochikit/content/tests/BrowserTestUtils/content-task.js:24:5
20:17:38 INFO - *************************
20:17:38 INFO - Not taking screenshot here: see the one that was previously logged
20:17:38 INFO - 81 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/plugins/browser_plugin_reloading.js | Uncaught exception - TypeError: Components.utils.waiveXrays(...).getObjectValue is not a function
20:17:38 INFO - 82 INFO Leaving test
20:17:38 INFO - 83 INFO *** loading new blocklist: http://127.0.0.1:8888/browser/browser/base/content/test/plugins/blockNoPlugins.xml
20:17:38 INFO - 84 INFO *** waiting on local load
20:17:38 INFO - 85 INFO *** waiting on remote load
20:17:38 INFO - 86 INFO *** blocklist loaded.
20:17:38 INFO - MEMORY STAT | vsize 826MB | vsizeMaxContiguous 130001148MB | residentFast 296MB | heapAllocated 95MB
20:17:38 INFO - 87 INFO TEST-OK | browser/base/content/test/plugins/browser_plugin_reloading.js | took 495ms 

(then yet another plugin related fail)

afaict these tests don't fail on non-e10s.

Flags: needinfo?(blassey.bugs)

Justin Wood (:Callek)

Reporter

Comment 1

•

9 years ago

log was at https://treeherder.mozilla.org/logviewer.html#?job_id=10554338&repo=try

Justin Wood (:Callek)

Reporter

Comment 2

•

9 years ago

(In reply to Justin Wood (:Callek) from comment #0)
> 20:17:07 INFO - 2 INFO TEST-UNEXPECTED-FAIL | browser-test.js | Found an
> unexpected unknown window 

Ooo apparantly that is *also* a message that was in bc1 and bc3 despite being green results in treeherder. (and is not w10 specific) so lets concentrate this bug on the plugin fails.

Brad Lassey [:blassey] (use needinfo?)

Updated

•

9 years ago

Assignee: nobody → jmathies

Flags: needinfo?(blassey.bugs)

Justin Wood (:Callek)

Reporter

Comment 3

•

9 years ago

jmathies, so Tracing this down, I could use some help on any humans for this piece...


20:17:37 INFO - 78 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/plugins/browser_plugin_reloading.js | Test 2, plugin fallback type should be PLUGIN_CLICK_TO_PLAY - Got 3, expected 8

Looking in code that is https://dxr.mozilla.org/mozilla-central/rev/04b8c412d9f58fb6194c58dcaa66bf278bbd53cf/browser/base/content/test/plugins/browser_plugin_reloading.js?offset=0#56

  is(pluginInfo.pluginFallbackType, Ci.nsIObjectLoadingContent.PLUGIN_CLICK_TO_PLAY,
     "Test 2, plugin fallback type should be PLUGIN_CLICK_TO_PLAY");

And specifically based on the IDL at https://dxr.mozilla.org/mozilla-central/rev/04b8c412d9f58fb6194c58dcaa66bf278bbd53cf/dom/base/nsIObjectLoadingContent.idl#49,63

This test is failing because we're blocklisting the plugin here.
   const unsigned long PLUGIN_BLOCKLISTED          = 3;

Hope this helps us get to the bottom of it somehow.

Flags: needinfo?(jmathies)

Justin Wood (:Callek)

Reporter

Comment 4

•

9 years ago

Per https://treeherder.mozilla.org/#/jobs?repo=try&revision=c27e72521c3c looks like this shifted to bc3

Summary: [M-e10s(bc2)] Permafail on win10 → [M-e10s(bc3)] Permafail on win10

Jim Mathies [:jimm]

Comment 5

•

9 years ago

(In reply to Justin Wood (:Callek) from comment #3)
> jmathies, so Tracing this down, I could use some help on any humans for this
> piece...
> 
> 
> 20:17:37 INFO - 78 INFO TEST-UNEXPECTED-FAIL |
> browser/base/content/test/plugins/browser_plugin_reloading.js | Test 2,
> plugin fallback type should be PLUGIN_CLICK_TO_PLAY - Got 3, expected 8
> 
> Looking in code that is
> https://dxr.mozilla.org/mozilla-central/rev/
> 04b8c412d9f58fb6194c58dcaa66bf278bbd53cf/browser/base/content/test/plugins/
> browser_plugin_reloading.js?offset=0#56
> 
>   is(pluginInfo.pluginFallbackType,
> Ci.nsIObjectLoadingContent.PLUGIN_CLICK_TO_PLAY,
>      "Test 2, plugin fallback type should be PLUGIN_CLICK_TO_PLAY");
> 
> And specifically based on the IDL at
> https://dxr.mozilla.org/mozilla-central/rev/
> 04b8c412d9f58fb6194c58dcaa66bf278bbd53cf/dom/base/nsIObjectLoadingContent.
> idl#49,63
> 
> This test is failing because we're blocklisting the plugin here.
>    const unsigned long PLUGIN_BLOCKLISTED          = 3;
> 
> Hope this helps us get to the bottom of it somehow.

I understand this, but what I don't understand is why this fails on your win10 hardware and passes on everything else?

Flags: needinfo?(jmathies)

Justin Wood (:Callek)

Reporter

Comment 6

•

9 years ago

Well this test passes in non-e10s on the same hardware. This is the same physical hardware that runs our passing w8 tests.

This is an x64 build.

Help me help you, what do you need from me in order to triage this?

Flags: needinfo?(jmathies)

Jim Mathies [:jimm]

Comment 7

•

9 years ago

(In reply to Justin Wood (:Callek) from comment #6)
> Well this test passes in non-e10s on the same hardware. This is the same
> physical hardware that runs our passing w8 tests.
> 
> This is an x64 build.
> 
> Help me help you, what do you need from me in order to triage this?

Interesting, do you know if we run the test in 64-bit fx on some other platform and it works? Or is this the first time we've tried to get it working under 64-bit?

Flags: needinfo?(jmathies)

Justin Wood (:Callek)

Reporter

Comment 8

•

9 years ago

Our win8 coverage is based on the x64 build. And its passing there.

Flags: needinfo?(jmathies)

Jim Mathies [:jimm]

Comment 9

•

9 years ago

Lots of failures here, I count 55 in a local run of the plugin suite.

Flags: needinfo?(jmathies)

Brad Lassey [:blassey] (use needinfo?)

Updated

•

9 years ago

Blocks: e10s-tests

tracking-e10s: --- → +

Jim Mathies [:jimm]

Comment 10

•

9 years ago

You can disable e10s related test suites if need be to get this going.

Assignee: jmathies → nobody

Justin Wood (:Callek)

Reporter

Comment 11

•

9 years ago

(In reply to Jim Mathies [:jimm] from comment #10)
> You can disable e10s related test suites if need be to get this going.

Given that our w10 slaves will be replacing w8 slaves. I have a few questions to help evaluate priority:

* Is this indicative of a problem out in the wild for actual users?
  - If so is this something a "significant" amount of users would face degraded experience with?
* If we have this test live on integration branches on win8, would the absense of an e10s testsuite run for w10 block the win10 deployment (in that it would *replace* w8 machines entirely)


If the consensus is "turn off w10 e10s testing" && "w10 e10s testing disabled does not block us turning off windows 8 testing" then I have no problem just outright disabling the test, but if those *two* statements don't hold true, I don't feel comfortable turning off the suites without further discussion and bringing us to a situation where we feel this is neither a user-facing issue that we'd hurt if the test is disabled nor an issue that blocks us turning off w8 machines for.

Flags: needinfo?(jmathies)

Flags: needinfo?(blassey.bugs)

Jim Mathies [:jimm]

Comment 12

•

9 years ago

> * Is this indicative of a problem out in the wild for actual users?
>   - If so is this something a "significant" amount of users would face
> degraded experience with?

I doubt it considering windows 8 is running fine. My guess is this has something to do with hardware / /test harness / timing.. i.e. testing infra related.

> * If we have this test live on integration branches on win8, would the
> absense of an e10s testsuite run for w10 block the win10 deployment (in that
> it would *replace* w8 machines entirely)

Yes, most likely. That's why it is tracking+ for e10s.

Flags: needinfo?(jmathies)

Jim Mathies [:jimm]

Comment 13

•

9 years ago

(In reply to Jim Mathies [:jimm] from comment #12)
> > * Is this indicative of a problem out in the wild for actual users?
> >   - If so is this something a "significant" amount of users would face
> > degraded experience with?
> 
> I doubt it considering windows 8 is running fine. My guess is this has
> something to do with hardware / /test harness / timing.. i.e. testing infra
> related.
> 
> > * If we have this test live on integration branches on win8, would the
> > absense of an e10s testsuite run for w10 block the win10 deployment (in that
> > it would *replace* w8 machines entirely)
> 
> Yes, most likely. That's why it is tracking+ for e10s.

Sorry, re-reading that, what I meant was I don't think it would block win10 replacing win8 for test runs, but it would likely block e10s rollout from aurora to beta.

Justin Wood (:Callek)

Reporter

Updated

•

9 years ago

Flags: needinfo?(blassey.bugs)

Amy Rich [:arr] [:arich]

Comment 14

•

7 years ago

This bug was from 2015 and has been superseeded by the migration we've already done for talos tests to w10 and the taskcluster work that's ongoing.

Status: NEW → RESOLVED

Closed: 7 years ago

Resolution: --- → FIXED

Nobody; OK to take it and work on it

Assignee

Updated

•

6 years ago

Component: Mochitest Chrome → Mochitest

Bugzilla

Quick Search

[M-e10s(bc3)] Permafail on win10

Categories

(Testing :: Mochitest, defect)

Tracking

(e10s+)

People

(Reporter: Callek, Unassigned)

References

(Blocks 1 open bug)

Details

Crash Data

Security

(public)

User Story

Description

Comment 1

Comment 2

Updated

Comment 3

Comment 4

Comment 5

Comment 6

Comment 7

Comment 8

Comment 9

Updated

Comment 10

Comment 11

Comment 12

Comment 13

Updated

Comment 14

Updated