1749609 - RDD/VAAPI: Crash in [@ __GI_flock ] with Nightly 96 on AMDGPU + Wayland

Status: RESOLVED WORKSFORME

(Core :: Security: Process Sandboxing, defect, P3)

Description

[Roman Stingler]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:100.0) Gecko/20100101 Firefox/100.0

Steps to reproduce:

with
Archlinux Gnome 41.2 + Wayland + AMDGPU (Radeon Pro Duo / Fiji)
Troubleshooting Information:
https://gist.github.com/romanstingler/214039ca451935294f5bfe0997e82746

media.ffmpeg.vaapi.enabled = 1
open http://demo.nimius.net/video_test/

Actual results:

opening the page immediately crashes the process and generates a coredump
https://gist.github.com/romanstingler/4fdf9f12683279b748c1e92bee6bfad1

I have tested all relevant versions and found the good and the bad one

good
https://archive.mozilla.org/pub/firefox/nightly/2021/11/2021-11-22-09-39-12-mozilla-central/firefox-96.0a1.en-US.linux-x86_64.tar.bz2

broken
https://archive.mozilla.org/pub/firefox/nightly/2021/11/2021-11-23-09-42-49-mozilla-central/firefox-96.0a1.en-US.linux-x86_64.tar.bz2

Expected results:

Doesn't crash on 96(release). 97(beta)
but on all nightly after the bad one from above.

I got a ton of crashes like this
https://crash-stats.mozilla.org/report/index/23ba7cc8-b113-4a80-93d5-44f830220111#allthreads

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Darkspirit]

Thanks for the report!

Build ID 20211123094249 (2021-11-23)

Please update to Nightly 98. Does the crash still occur?
RDD sandbox fixes happened in 97, not 96.

Comment 3

[Martin Stránský [:stransky] (ni? me)]

Can you please use mozregression tool to find exact commits?
https://fedoraproject.org/wiki/How_to_debug_Firefox_problems#Use_Mozregression_tool
I expect the bug was introduced between 95 and 96 then.

Also please run Firefox with MOZ_LOG="Dmabuf:5, PlatformDecoderModule:5" env variable, crash it and attach the log here.
Thanks.

Comment 4

[Gabriele Svelto [:gsvelto]]

There's plenty of crashes in version 97 and 98. Confirming. This is the sandbox preventing a flock() syscall.

Comment 5

[Gabriele Svelto [:gsvelto]]

The uses are coming either from here or here.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The component has been changed since the backlog priority was decided, so we're resetting it.
For more information, please visit auto_nag documentation.

Comment 7

[Darkspirit]

(In reply to Roman Stingler from comment #0)

Doesn't crash on 96(release). 97(beta)

Please attach about:support of Release and Beta here.
You might have changed other prefs which might explain the differences.

I assume rdd-ffvpx tried to use VP8 VAAPI right after VAAPI has been allowed in RDD: https://hg.mozilla.org/integration/autoland/rev/669ca27af67f

Comment 8

[Roman Stingler]

Attachment: ff-beta-troubleshooting-info

Firefox Beta 97.0b2 troubleshooting information

Comment 9

[Roman Stingler]

Attachment: ff-release-troubleshooting-info

ff-release-troubleshooting-info 96.0

Comment 10

[Roman Stingler]

(In reply to Darkspirit from comment #2)

Thanks for the report!

Build ID 20211123094249 (2021-11-23)

Please update to Nightly 98. Does the crash still occur?
RDD sandbox fixes happened in 97, not 96.

Yes happens also with latest nightly 98.

(In reply to Martin Stránský [:stransky] (ni? me) from comment #3)

Can you please use mozregression tool to find exact commits?
https://fedoraproject.org/wiki/How_to_debug_Firefox_problems#Use_Mozregression_tool
I expect the bug was introduced between 95 and 96 then.

Also please run Firefox with MOZ_LOG="Dmabuf:5, PlatformDecoderModule:5" env variable, crash it and attach the log here.
Thanks.
No I can't use mozregression since I can't launch it within wayland as non root user and as a root user I'm not allowed to open the binaries mozregression is providing.....

Therefore, I bisected it manually and this is the MOZ_LOG with latest beta
https://gist.github.com/romanstingler/343dcef20ba83451f74c048d5e23ea5c

Comment 11

[Roman Stingler]

blackout@Workstation ~ % QT_QPA_PLATFORM=wayland mozregression-gui
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.

(mozregression-gui:32611): GLib-GIO-ERROR **: 22:30:06.319: Settings schema 'org.gnome.settings-daemon.plugins.xsettings' does not contain a key named 'antialiasing'
[1]    32610 trace trap (core dumped)  QT_QPA_PLATFORM=wayland mozregression-gui

for the interested here is the ff-nightly dmabuf log
https://gist.github.com/romanstingler/e99cb04e8be174163c7bb46fa9fcc94f

Comment 12

[Darkspirit]

Thanks!

It's possible to install mozregression as a user. The commands would have been:
$ pip3 install --user --upgrade mozregression
$ MOZ_ENABLE_WAYLAND=1 ~/.local/bin/mozregression --good 95 --bad 96 --pref media.ffmpeg.vaapi.enabled:true -a http://demo.nimius.net/video_test/

Range from comment 0 is:
mozregression --good 20211122093912 --bad 20211123094249

0:02.49 INFO: Last good revision: ace2f4af2c29de1886e1e627d0fdb583e7573b59 (2021-11-22 09:39:12)
0:02.49 INFO: First bad revision: 71332992f78f548b219f6405ca970828bd22c071 (2021-11-23 09:42:49)
0:02.49 INFO: Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ace2f4af2c29de1886e1e627d0fdb583e7573b59&tochange=71332992f78f548b219f6405ca970828bd22c071

Comment 13

[Roman Stingler]

(In reply to Darkspirit from comment #12)

Thanks!

It's possible to install mozregression as a user. The commands would have been:
$ pip3 install --user --upgrade mozregression
$ MOZ_ENABLE_WAYLAND=1 ~/.local/bin/mozregression --good 95 --bad 96 --pref media.ffmpeg.vaapi.enabled:true -a http://demo.nimius.net/video_test/

Range from comment 0 is:
mozregression --good 20211122093912 --bad 20211123094249

0:02.49 INFO: Last good revision: ace2f4af2c29de1886e1e627d0fdb583e7573b59 (2021-11-22 09:39:12)
0:02.49 INFO: First bad revision: 71332992f78f548b219f6405ca970828bd22c071 (2021-11-23 09:42:49)
0:02.49 INFO: Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ace2f4af2c29de1886e1e627d0fdb583e7573b59&tochange=71332992f78f548b219f6405ca970828bd22c071

Thank you a lot, I previously installed the binary from the archlinux repository.

I run the tests and my final information is

 5:00.96 INFO: Narrowed integration regression window from [e662ecf3, 669ca27a] (4 builds) to [e24b5167, 669ca27a] (2 builds) (~1 steps left)
 5:00.96 INFO: No more integration revisions, bisection finished.
 5:00.96 INFO: Last good revision: e24b51679d83b9edd509529057ff176c364c6414
 5:00.96 INFO: First bad revision: 669ca27af67f297133783f30c87b709470b0d3fa
 5:00.96 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e24b51679d83b9edd509529057ff176c364c6414&tochange=669ca27af67f297133783f30c87b709470b0d3fa

Comment 14

[Gian-Carlo Pascutto [:gcp]]

The spike here appears to have disappeared.

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.