1876538 - macOS Crash in [@ mozilla::wr::Moz2DRenderCallback]

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

There are 10 assertions/crashes from 9 installs of Firefox 123.0b1 and 123.0b2 on macOS, none for v122.

Crash report: https://crash-stats.mozilla.org/report/index/165feac7-a0fa-4c7c-80d4-142400240125

MOZ_CRASH Reason: MOZ_RELEASE_ASSERT(false)

Top 10 frames of crashing thread:

0  XUL  mozilla::wr::Moz2DRenderCallback  gfx/webrender_bindings/Moz2DImageRenderer.cpp:433
0  XUL  wr_moz2d_render_cb  gfx/webrender_bindings/Moz2DImageRenderer.cpp:474
1  XUL  webrender_bindings::moz2d_renderer::rasterize_blob::{{closure}}  gfx/webrender_bindings/src/moz2d_renderer.rs:615
1  XUL  objc::rc::autorelease::autoreleasepool  third_party/rust/objc/src/rc/autorelease.rs:29
1  XUL  webrender_bindings::moz2d_renderer::autoreleasepool  gfx/webrender_bindings/src/moz2d_renderer.rs:592
1  XUL  webrender_bindings::moz2d_renderer::rasterize_blob  gfx/webrender_bindings/src/moz2d_renderer.rs:613
2  XUL  core::ops::function::Fn::call  library/core/src/ops/function.rs:79
2  XUL  core::ops::function::impls::<impl core::ops::function::FnMut<A> for&F>::call_mut  library/core/src/ops/function.rs:272
2  XUL  core::ops::function::impls::<impl core::ops::function::FnOnce<A> for&mut F>::call_once  library/core/src/ops/function.rs:305
2  XUL  core::option::Option<T>::map  library/core/src/option.rs:1072

Comment 1

[Jeff Muizelaar [:jrmuizel]]

Critical error contains: |[0][GFX1-]: Replay failure: SetCurrentFilterNode PLAY (t=331.667)

Andrew, could this be related to your changes?

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Also maybe related to an existing problem: bug 1873322

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The bug is marked as tracked for firefox123 (beta). However, the bug still isn't assigned.

:bhood, could you please find an assignee for this tracked bug? If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit BugBot documentation.

Comment 4

[Andrew Osmond [:aosmond] (he/him)]

Volume is too high to be a coincidence. An existing crash should have the same signature since it would have been a replay failure, e.g.:
https://searchfox.org/mozilla-central/diff/9c65def36af441133c75a44b126e65184b039b2f/gfx/2d/RecordedEventImpl.h#4128

Comment 5

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1876538 - Backout changes from bug 1875364 to cache current filter for recordings.

Comment 6

[Andrew Osmond [:aosmond] (he/him)]

Reviewing the crash logs, there are no historical filter related crashes, so this must be a regression.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4641c3dcf0c3
Backout changes from bug 1875364 to cache current filter for recordings. r=gfx-reviewers,jrmuizel

Comment 8

[Andrew Osmond [:aosmond] (he/him)]

Comment on attachment 9376630 [details]
Bug 1876538 - Backout changes from bug 1875364 to cache current filter for recordings.

Beta/Release Uplift Approval Request

• User impact if declined: Low volume crash
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Just a backout reverting to historical behaviour, should apply cleanly to beta since the change was so recent
• String changes made/needed:
• Is Android affected?: Yes

Comment 9

[Iulian Moraru]

Backed out for causing mochitest-chrome failures on test_animation_observers_async.html.

• Push with failures

• Failure log

• Backout link

[task 2024-01-26T16:56:11.210Z] 16:56:11     INFO - TEST-START | dom/animation/test/chrome/test_animation_observers_async.html
[task 2024-01-26T16:56:11.341Z] 16:56:11     INFO - TEST-INFO | started process screentopng
[task 2024-01-26T16:56:11.564Z] 16:56:11     INFO - TEST-INFO | screentopng: exit 0
[task 2024-01-26T16:56:11.564Z] 16:56:11     INFO - TEST-UNEXPECTED-FAIL | dom/animation/test/chrome/test_animation_observers_async.html | tree_ordering: subtree - tree_ordering: subtree: assert_equals: records after finishing - number of records expected 13 but got 12
[task 2024-01-26T16:56:12.955Z] 16:56:12     INFO - TEST-PASS | dom/animation/test/chrome/test_animation_observers_async.html | Test chrome-only MutationObserver animation notifications (async tests) - Test chrome-only MutationObserver animation notifications (async tests): Elided 14 passes or known failures.
[task 2024-01-26T16:56:12.955Z] 16:56:12     INFO - GECKO(2236) | MEMORY STAT | vsize 11471MB | residentFast 461MB | heapAllocated 208MB
[task 2024-01-26T16:56:12.999Z] 16:56:12     INFO - TEST-OK | dom/animation/test/chrome/test_animation_observers_async.html | took 1789ms
[task 2024-01-26T16:56:13.031Z] 16:56:13     INFO - TEST-START | dom/animation/test/chrome/test_animation_observers_sync.html

Comment 10

[Andrew Osmond [:aosmond] (he/him)]

Given the test failing, I am 95% certain this is false correlation. There doesn't seem to be anything in the test case that would produce a blob image, or a canvas recording.

Comment 11

[Andrew Osmond [:aosmond] (he/him)]

That said, it does seem to be persistent.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/828e8cbfdf57
Backout changes from bug 1875364 to cache current filter for recordings. r=gfx-reviewers,jrmuizel

Comment 13

[Sandor Molnar[:smolnar]]

It turned out to be bug 1865955 the culprit after all.
Relanded

Comment 14

[Andrew Osmond [:aosmond] (he/him)]

There is inconsistency in how many tests are run in each mochitest-chrome job, when it should be the same set each time (barring some change in the test config). I see anywhere from 7 to 400+ run in the job, which explains why sometimes it takes anywhere from 2-20 minutes to run it. We need to get to the bottom of this.

Comment 15

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This is desired behavior. Only every 10th or 20th push runs the full set of test manifests (folders) while the others only run those for which CI thinks it's likely they will catch an issue (= "optimized push/target set").

Comment 16

[Andrew Osmond [:aosmond] (he/him)]

(In reply to Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout) from comment #15)

This is desired behavior. Only every 10th or 20th push runs the full set of test manifests (folders) while the others only run those for which CI thinks it's likely they will catch an issue (= "optimized push/target set").

I now appreciate that after discussing this on Firefox CI. I was unaware and having this knowledge now will help me/others navigate this in the future.

Comment 17

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/828e8cbfdf57

Comment 18

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/28597dc87a4c