1754334 - Intermittent org.mozilla.geckoview.test.ScreenshotTest#capturePixelsBeforeGpuProcessCrash | java.lang.AssertionError: Images are almost identical

Status: RESOLVED FIXED

(GeckoView :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mlaza [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=367152345&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/PedSmJ2IS-WGZqCq96wP0Q/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-02-09T00:22:52.961Z] 00:22:52     INFO -  TEST-START | org.mozilla.geckoview.test.ScreenshotTest#capturePixelsBeforeGpuProcessCrash
[task 2022-02-09T00:22:54.317Z] 00:22:54     INFO -  org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: numtests=875
[task 2022-02-09T00:22:54.317Z] 00:22:54     INFO -  org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: stream=
[task 2022-02-09T00:22:54.317Z] 00:22:54     INFO -  org.mozilla.geckoview.test | Error in capturePixelsBeforeGpuProcessCrash(org.mozilla.geckoview.test.ScreenshotTest):
[task 2022-02-09T00:22:54.317Z] 00:22:54     INFO -  org.mozilla.geckoview.test | java.lang.AssertionError: Images are almost identical
[task 2022-02-09T00:22:54.318Z] 00:22:54     INFO -  org.mozilla.geckoview.test | Expected: a value less than or equal to <1>
[task 2022-02-09T00:22:54.318Z] 00:22:54     INFO -  org.mozilla.geckoview.test |      but: <255> was greater than <1>
[task 2022-02-09T00:22:54.318Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
[task 2022-02-09T00:22:54.318Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.junit.Assert.assertThat(Assert.java:956)
[task 2022-02-09T00:22:54.318Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.junit.rules.ErrorCollector$1.call(ErrorCollector.java:65)
[task 2022-02-09T00:22:54.318Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.junit.rules.ErrorCollector.checkSucceeds(ErrorCollector.java:78)
[task 2022-02-09T00:22:54.319Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.junit.rules.ErrorCollector.checkThat(ErrorCollector.java:63)
[task 2022-02-09T00:22:54.319Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.checkThat(GeckoSessionTestRule.java:887)
[task 2022-02-09T00:22:54.319Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.mozilla.geckoview.test.BaseSessionTest.assertThat(BaseSessionTest.kt:136)
[task 2022-02-09T00:22:54.319Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.mozilla.geckoview.test.ScreenshotTest.assertScreenshotResult(ScreenshotTest.kt:88)
[task 2022-02-09T00:22:54.320Z] 00:22:54     INFO -  org.mozilla.geckoview.test | 	at org.mozilla.geckoview.test.ScreenshotTest.capturePixelsBeforeGpuProcessCrash(ScreenshotTest.kt:252)
...
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.876 W/webrender::device::gl(19063): Attribute VertexAttribute { name: "aUvRect2", count: 4, kind: F32 } is not found in the shader composite. Expected at 7, found at -1
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.876 E/eglCodecCommon(19063): glUtilsParamSize: unknow param 0x00008741
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.933 D/EGL_emulation(19063): eglMakeCurrent: 0x7f8a17fba440: ver 3 0 (tinfo 0x7f8a227bf120)
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.934 D/GeckoViewContent( 3476): handleEvent: DOMWindowClose
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.936 D/GeckoViewConsole( 3476): onEvent GeckoView:WebExtension:List null
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.937 D/GeckoViewProgressDelegate[C](19023): handleEvent: MozAfterPaint
[task 2022-02-09T00:22:54.434Z] 00:22:54     INFO -  02-09 00:22:53.939 D/GeckoViewProgressDelegate[C](19023): handleEvent: MozAfterPaint
[task 2022-02-09T00:22:54.435Z] 00:22:54     INFO -  02-09 00:22:53.939 E/GeckoSessionTestRule( 3476): test_end c5ee677f-bc83-49bd-9e28-2d35f3d0f059 capturePixelsBeforeGpuProcessCrash(org.mozilla.geckoview.test.ScreenshotTest)
[task 2022-02-09T00:22:54.435Z] 00:22:54  WARNING -  TEST-UNEXPECTED-FAIL | org.mozilla.geckoview.test.ScreenshotTest#capturePixelsBeforeGpuProcessCrash | java.lang.AssertionError: Images are almost identical
[task 2022-02-09T00:22:54.435Z] 00:22:54     INFO -  TEST-INFO took 1425ms

Comment 1

[Jamie Nicol [:jnicol]]

Hmm, so the problem seems to be that sometimes the RequestScreenPixels request comes in before webrender has built the scene including the content process' display list. So it just has the empty parent process display list to render, so the screenshot is empty.

I can reproduce this on one of my devices but not on the emulator. It's definitely just timing related whether the content process display list arrives before the capture screen pixels request. With some hacks I can fix this bug locally, by delaying taking the screenshot until we've built a scene containing 2 pipelines. But it's actually pretty tricky to do this properly - we don't know how many or which pipelines are required for the page to be correct.

I think we should instead just return an error if the GPU process crashes during a screenshot request. I'll write a patch

Comment 2

[Jamie Nicol [:jnicol]]

Attachment: Bug 1754334 - Return error when GPU process crash occurs during screen pixels request. r?agi

In bug 1750569 we attempted to ensure that following a GPU process
crash outstanding screen pixels requests would be fulfilled. While
this usually worked there was a race condition between sending the
request to the new compositor and the content process sending the
display list to the new compositor, which meant that sometimes we
would screenshot an empty screen instead of the page content.

As a GPU process crash is an extraordinary circumstance and
screenshots are non-critical, the best solution is to simply return an
error if a GPU process crash occurs while there is an outstanding
request (or if a new request is made whilst the GPU process is
restarting). This patch also updates the junit test to check for this
error rather than expecting a screenshot to be returned.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/41b4135023e3
Return error when GPU process crash occurs during screen pixels request. r=agi

Comment 4

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/41b4135023e3

Comment 5

[Intermittent Failures Robot]

7 failures in 3711 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 7

Platform and build breakdown:

• android-em-7-0-x86_64-qr: 4
  • debug-isolated-process: 1
  • opt: 3
• android-em-7-0-x86_64-lite-qr: 3
  • debug: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1754334&startday=2022-02-07&endday=2022-02-13&tree=all