1591695 - Permafailing browser_pluginCrashReportNonDeterminism.js | Uncaught exception - Timed out waiting for plugin to be in crash report state - threw exception: TypeError: plugin is null when Gecko 72 merges to Beta on 2019-12-02

Status: RESOLVED WORKSFORME

(Core Graveyard :: Plug-ins, defect)

Description

[Natalia Csoregi [:nataliaCs]]

[Tracking Requested - why for this release]:
Central as Beta: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=f240cd4eb2e712483535f23a8d9175931fd388f1&searchStr=browser-chrome&selectedJob=273081796

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273081796&repo=try&lineNumber=2225

[task 2019-10-26T12:52:20.157Z] 12:52:20 INFO - TEST-START | browser/base/content/test/plugins/browser_pluginCrashReportNonDeterminism.js
[task 2019-10-26T12:52:20.476Z] 12:52:20 INFO - GECKO(2188) | [Parent 2188, Gecko_IOThread] WARNING: pipe error (113): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 358
[task 2019-10-26T12:52:20.553Z] 12:52:20 INFO - GECKO(2188) | ###!!! [Parent][MessageChannel::Call] Error: Channel error: cannot send/recv
[task 2019-10-26T12:52:20.553Z] 12:52:20 INFO - GECKO(2188) | ###!!! [Parent][MessageChannel] Error: (msgtype=0x6E001E,name=PPluginInstance::Msg_AsyncSetWindow) Channel error: cannot send/recv
[task 2019-10-26T12:52:20.569Z] 12:52:20 INFO - TEST-INFO | started process screentopng
[task 2019-10-26T12:52:21.216Z] 12:52:21 INFO - TEST-INFO | screentopng: exit 0
[task 2019-10-26T12:52:21.217Z] 12:52:21 INFO - Buffered messages logged at 12:52:20
[task 2019-10-26T12:52:21.217Z] 12:52:21 INFO - Entering test bound testChromeHearsPluginCrashFirst
[task 2019-10-26T12:52:21.217Z] 12:52:21 INFO - Buffered messages finished
[task 2019-10-26T12:52:21.217Z] 12:52:21 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/plugins/browser_pluginCrashReportNonDeterminism.js | Uncaught exception - Timed out waiting for plugin to be in crash report state - threw exception: TypeError: plugin is null
[task 2019-10-26T12:52:21.217Z] 12:52:21 INFO - Leaving test bound testChromeHearsPluginCrashFirst
[task 2019-10-26T12:52:21.218Z] 12:52:21 INFO - Entering test bound testContentHearsCrashFirst
[task 2019-10-26T12:52:21.218Z] 12:52:21 INFO - TEST-PASS | browser/base/content/test/plugins/browser_pluginCrashReportNonDeterminism.js | Found minidump -
[task 2019-10-26T12:52:21.218Z] 12:52:21 INFO - TEST-PASS | browser/base/content/test/plugins/browser_pluginCrashReportNonDeterminism.js | Found extra file -

Comment 1

[Johann Hofmann [:johannh]]

Are you sure you have the right bug? Bug 1422056 isn't really meddling with early beta vs. Nightly, and the pref it's changing seems really unrelated.

Comment 2

[Cosmin Sabou [:CosminS]]

The test is skipped if !crashreporter that's why is not failing anymore on beta sims. https://searchfox.org/mozilla-central/source/browser/base/content/test/plugins/browser.ini#103
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273656137&repo=try&lineNumber=1478

Green on the 25th: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=272948007&repo=try&lineNumber=2925
Failed on the 26th: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273081796&repo=try&lineNumber=2225
Green on the 27th: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273151652&repo=try&lineNumber=2775
Could it have anything to do with patch in bug 1591386? https://phabricator.services.mozilla.com/D50696 that's when it was imported and caused new failures in beta-sims. Since then it was all green.

The pushlog from 25th till 26th: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ad7a152bc66c0d411a6fb0b210d675abed9693c7&tochange=0b9d0a86ebfd89cfb89b8044a72e89ce743aeeee

Comment 3

[Johann Hofmann [:johannh]]

The test is skipped if !crashreporter that's why is not failing anymore on beta sims. https://searchfox.org/mozilla-central/source/browser/base/content/test/plugins/browser.ini#103

So are you saying that this bug is invalid? :)

Could it have anything to do with patch in bug 1591386? https://phabricator.services.mozilla.com/D50696 that's when it was imported and caused new failures in beta-sims. Since then it was all green.

Unfortunately I have little knowledge of this test or that bug, so I'm going to needinfo the author of the test, which seems to be ... mconley! Mike, do you have any idea what's going on here?

Comment 4

[Mike Conley (:mconley) (:⚙️)]

The many instances of

[task 2019-10-26T12:52:19.970Z] 12:52:19     INFO - GECKO(2188) | JavaScript error: resource:///actors/PluginChild.jsm, line 1038: InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable

before the test failure in the log seem suspicious.

Essentially, this test checks that we behave correctly if the parent process receives the message about the plugin process crash before the host content process does, and vice versa.

The test is failing because plugin is null here:

https://searchfox.org/mozilla-central/rev/1fe0cf575841dbf3b7e159e88ba03260cd1354c0/browser/base/content/test/plugins/browser_pluginCrashReportNonDeterminism.js#69

On inspection, this might be because we're not necessarily waiting for the test page to finish loading for either of the two cases in the test - example: https://searchfox.org/mozilla-central/rev/1fe0cf575841dbf3b7e159e88ba03260cd1354c0/browser/base/content/test/plugins/browser_pluginCrashReportNonDeterminism.js#69

Here, we're awaiting on browserLoaded, but we're not ensuring that the page that's loaded is CRASH_URL. Since this is a freshly opened tab, this means that we might be getting the load event for the initial about:blank.

Out of curiosity, why is the crashreporter disabled on Beta? This means we lose quite a bit of test coverage for crash scenarios...

Comment 5

[Cosmin Sabou [:CosminS]]

Out of curiosity, why is the crashreporter disabled on Beta? This means we lose quite a bit of test coverage for crash scenarios...

I cannot really explain this round of events:
It ran on the 25th: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=272948007&repo=try&lineNumber=2925
Failed on the 26th: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273081796&repo=try&lineNumber=2225
Skipped on the 27th: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273150490&repo=try&lineNumber=1467
and this line https://searchfox.org/mozilla-central/source/browser/base/content/test/plugins/browser.ini#103 was on central before it was finally merged to beta on the 21st.
Searched for when it was added to central code but didn't find anything in the relevant time frame:
https://hg.mozilla.org/mozilla-central/log?rev=crashreporter or https://hg.mozilla.org/mozilla-central/log?rev=browser%2Fbase%2Fcontent%2Ftest%2Fplugins%2Fbrowser.ini
what landed on between 25th and 27th on beta is https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=b403c8a57e2ae627e85c998c3290d9fe4b90e590&tochange=0dfa20097d27d51b6748eb04467a0d6a98764899
Maybe Ricky or Ryan have a better answer to your question.

Comment 6

[Ryan VanderMeulen [:RyanVM]]

ASAN builds don't have the crash reporter enabled.

Comment 7

[Cosmin Sabou [:CosminS]]

LE for comment 2: double checked the runs from 27th and indeed the test ran and it was green: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=273151652&repo=try&lineNumber=2775
Sorry for the mess up. This only points more towards issues from the patch in bug 1591386? https://phabricator.services.mozilla.com/D50696 as there were several browser chrome failures on that beta-sim that only happened on that day.

Comment 8

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Only seen for last Saturday's central-as-beta sim, likely from an imported patch.