Closed Bug 1623086 Opened 5 years ago Closed 3 years ago

lost middle and right-click in webpages in 75

Categories

(Core :: Widget, defect)

75 Branch
x86_64
OpenBSD
defect

Tracking

()

RESOLVED FIXED
91 Branch
Tracking Status
firefox-esr78 --- wontfix
firefox74 --- unaffected
firefox75 --- wontfix
firefox76 --- wontfix
firefox89 --- wontfix
firefox90 --- wontfix
firefox91 --- fixed

People

(Reporter: gaston, Assigned: gaston, NeedInfo)

Details

(Keywords: regression)

Attachments

(5 files)

on OpenBSD, with 75.0 betas (2,3 &4) i've lost the ability to use rightclick and middle click on the webpage content. rightclick and middleclick works on the chrome/ui, but yield nothing in the web page. Happens with an empty, profile, and of course no issue with 74.0 in the same environment.

Happy to provide more details if needed, but so far i havent been able to understand what could have regressed that.

Can you run mozregression to find out exactly when this broke?

Flags: needinfo?(landry)

basically, with 75 i can only use left click on links, no way to open link in new tab or use the page context menu.

i will have a hard time running mozregression because it'll take hours, and building on OpenBSD is .. sometimes convoluted. jbeich, is it fine for you on FreeBSD ?

Flags: needinfo?(landry)

(In reply to Landry Breuil (:gaston) from comment #3)

i will have a hard time running mozregression because it'll take hours

I'm a little confused, are you limited in terms of bandwidth? Are there just no nightly/autoland openbsd builds available?

For something that is perhaps easier to check, are there errors in the browser console in 75 when things fail?

If not, if you open the browser toolbox (ctrl-alt-shift-i) and set a breakpoint at https://searchfox.org/mozilla-central/rev/d69ec052bed8700af7a283e37b60b4af22734930/browser/base/content/nsContextMenu.js#93 , does the breakpoint get hit?

Does reducing the content sandbox level fix it?

Flags: needinfo?(landry)

(In reply to :Gijs (he/him) from comment #4)

(In reply to Landry Breuil (:gaston) from comment #3)

i will have a hard time running mozregression because it'll take hours

I'm a little confused, are you limited in terms of bandwidth? Are there just no nightly/autoland openbsd builds available?

correct, there are no OpenBSD builds available as its not a supported/tier1 platform.

(In reply to :Gijs (he/him) from comment #5)

For something that is perhaps easier to check, are there errors in the browser console in 75 when things fail?

when i right click or middle click somewhere in the webpage, no msg is shown in the browser console.

If not, if you open the browser toolbox (ctrl-alt-shift-i) and set a breakpoint at https://searchfox.org/mozilla-central/rev/d69ec052bed8700af7a283e37b60b4af22734930/browser/base/content/nsContextMenu.js#93 , does the breakpoint get hit?

trying to open the browser toolbox only yields uncaught exception: Object in the browser console.

Does reducing the content sandbox level fix it?

Openbsd has its own sandboxing (cf bug #1580271 & others) which works fine in 74. what could have change in 75 wrt sandboxing ?

Flags: needinfo?(landry)

(In reply to Landry Breuil (:gaston) from comment #6)

(In reply to :Gijs (he/him) from comment #4)

Does reducing the content sandbox level fix it?

Openbsd has its own sandboxing (cf bug #1580271 & others) which works fine in 74. what could have change in 75 wrt sandboxing ?

I don't know. :gcp ?

Flags: needinfo?(gpascutto)

The sandboxing code is per platform and :gaston is the one who's been making the OpenBSD one, so he'd know better than me! This seems more likely to be fallout from a GTK related change. Maybe the Redhat people can make educated guesses?

Bisecting does seem like the best option, even if it's going to take a while...

Flags: needinfo?(gpascutto) → needinfo?(stransky)

well, at least i have a data point, another laptop with 75.0b2 doesnt exhibit the bug, and im pretty sure i saw it with b3 on another laptop. will upgrade the working laptop to b4 and figure out if its a regression within b2/b4 or something specific to a profile/setting/environment..

on this laptop (x1 carbon 3rd gen) where 75.0b2 works, 75.0b4 also works (ie middle & rightclick) - both with my regular profile and an empty profile - thats with gtk 3.24.14. will see tmrw on the failling laptop (t430u) what can be different..

(In reply to Landry Breuil (:gaston) from comment #3)

jbeich, is it fine for you on FreeBSD ?

right/middle click within content area works fine in Firefox 75.0b4 on FreeBSD 11.3 i386 both in X11, Wayland and Xwayland. Tested within a pristine jail after passing through X11/Wayland unix(4) sockets from host.

Flags: needinfo?(stransky)

Also e10s may be a factor here, so try to run with MOZ_FORCE_DISABLE_E10S=1

Attached file aboutsupportt430u.txt

(In reply to Martin Stránský [:stransky] from comment #14)

Also e10s may be a factor here, so try to run with MOZ_FORCE_DISABLE_E10S=1

spot on - on this laptop, disabling E10S fixes rightclick and middleclick. attached all the correspond about:support extracts..

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Keywords: regression

Hi Mike, looks like an E10S thing. Can you take a look?

Flags: needinfo?(mconley)

I don't think we can directly pin this on e10s. Running with MOZ_FORCE_DISABLE_E10S is not a supported configuration, and is potentially using older codepaths that are bypassing where the bug really is.

What we really need is a regression range. If we can't get that, perhaps the Widget log that stransky requested in comment 13 will be useful?

Flags: needinfo?(mconley) → needinfo?(landry)
Attached file widgetlog-bad.txt

running with

[08:45] adventurer:/tmp/ $MOZ_LOG="Widget:5" firefox -no-remote -P default-default-1 2>&1 | tee widgetlog-bad.txt   

i definitely see mouse events upon right/left/middle click but no action happens for right/middle click, unless i use MOZ_FORCE_DISABLE_E10S=1 (and then i get the same actions logged)

[Parent 3548: Main Thread]: D/Widget Button 1 press on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget Button 1 release on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget Button 3 press on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget Button 3 release on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget Button 3 press on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget Button 3 release on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget nsWindow::Create() [0x1937a1609c00]: parent window for popup: 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget nsWindow::NativeMove [0x1937a1609c00] 0 0
[Parent 3548: Main Thread]: D/Widget moz_container_init [0x193765c8e650]
[Parent 3548: Main Thread]: D/Widget nsWindow [0x1937a1609c00] Popup
[Parent 3548: Main Thread]: D/Widget    mShell 0x1936f69c67b0 mContainer 0x193765c8e650 mGdkWindow 0x19372c933c40 0x3400040
[Parent 3548: Main Thread]: D/Widget Button 2 press on 0x19376ee11000
[Parent 3548: Main Thread]: D/Widget Button 2 release on 0x19376ee11000

Flags: needinfo?(landry)

problem still present with 75.0b9 - another data point, when on this machine where right/middle clicks fail, at session startup (ie when opening the browser) with E10S enabled, the tabs are not loaded at all, and the urls arent displayed in the url bar, ie i only have the tabs with titles and favicons, but nothing happens (ie no page load) if i switch tabs.

Hm that would need more work with gdb or so, as the events are clearly obtained from Gtk but are lost somewhere in IPC, so you see the event handler:

https://searchfox.org/mozilla-central/rev/d6f957415cf009995ecb539ef1425316d82164a9/widget/gtk/nsWindow.cpp#3057

unfortunately I'm not much experienced with the ipc code.

I'm seeing the same issue as landry with 75.b10 on OpenBSD/amd64 (package built by landry) on my x230 Laptop which has the same graphics hardware:

inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 4000" rev 0x09

as Landry's machine. On a new profile, right and middle clicks inside the browser window don't work, unless I run with MOZ_FORCE_DISABLE_E10S=1 variable.

On an x280 with the same package and

inteldrm0 at pci0 dev 2 function 0 "Intel UHD Graphics 620" rev 0x07

I see no issues.

(In reply to Landry Breuil (:gaston) from comment #22)

problem still present with 75.0b9 - another data point, when on this machine where right/middle clicks fail, at session startup (ie when opening the browser) with E10S enabled, the tabs are not loaded at all, and the urls arent displayed in the url bar, ie i only have the tabs with titles and favicons, but nothing happens (ie no page load) if i switch tabs.

This sounds like the child process is (or processes are...) failing to start. If you run with MOZ_DISABLE_CONTENT_SANDBOX=1, does that help?

Also, on the broken machines, if you turn on the pref layers.acceleration.force-enabled and restart Firefox, does that make any difference?

Part of me wonders if the basic compositor is completely broken (for you?), but I'm also not sure if that's the cause of, the result of, or unrelated to the mouse issues...

Flags: needinfo?(tb)
Flags: needinfo?(landry)

i'm not 100% sure MOZ_DISABLE_CONTENT_SANDBOX=1 does something on OpenBSD with our pledge sandboxing... but disabling the content sandbox (ie adding 'disable' to /etc/firefox/pledge.content, this is specific to OpenBSD) indeed helps, rightclick and middleclick is back. So something in the content sandboxing changed for 75 and that triggers only on certain environments/hw ? i dont see pledge violations when running with a broken browser, so i dont see what kind of new capabilities would be needed.

Tried with layers.acceleration.force-enabled forced to true (also tried with MOZ_ACCELERATED=1 in the env, unsure it achieves the same), that 'fixes' all the webGL sections in about:support but doesnt allow me to open more tabs nor rightclick anywhere (without disabling the content sandbox) etc.

jcs, care to help ?

Flags: needinfo?(landry) → needinfo?(jcs)

or maybe something changed in the sandboxing code that requires access to new paths and the unveil() configuration is too tight, files arent found and all hell breaks loose ?

i dunno if that's related or not but on this laptop where it breaks, i see this on stderr but im not sure its related:

libGL error: MESA-LOADER: failed to retrieve device information                                                                                                                                                                                
libGL error: Version 4 or later of flush extension not found                                                                                                                                                                                   
libGL error: failed to load driver: i915                                                                                                                                                                                                       
libGL error: failed to open drm device: No such file or directory                                                                                                                                                                              
libGL error: failed to load driver: i965                                                                                                                                                                                                       
libGL error: MESA-LOADER: failed to open swrast (search paths /usr/X11R6/lib/modules/dri)                                                                                                                                                      
libGL error: failed to load driver: swrast      

and enabling pledge, but disabling unveil for the content process also helps

rm /etc/firefox/pledge.content
echo 'disable' | tee /etc/firefox/unveil.content

so this is definitely related to the content process trying to access something that is now critically required, hidden by unveil() sandboxing, and failing.

I can confirm that disabling pledge or unveil helps with the problem (I do not see that noise on stderr).

Below is what I wrote before seeing landry's comments:

Neither running with MOZ_DISABLE_CONTENT_SANDBOX=1 nor toggling the layers.acceleration.force-enabled preference to true, nor both makes a difference (I did restart firefox after toggling) on the affected machines.

Some more or less random observations:

The right (or middle) clicks on a link are "registered" in that the link in question is surrounded by a rectangular dotted line. It's just that the contextual menu doesn't appear (or no new tab is opened).

I cannot navigate to about:config from the url bar (about:preferences, about:support don't work either). I have to start the browser with firefox about:config. Navigating to actual pages works fine.

I also noticed that opening the browser and then quitting it right away (either by doing Ctrl + Q or selecting Quit from the hamburger menu on the top right) doesn't terminate the processes properly. I always have a bunch of them still lying around. Like this:

$ pgrep -lf firefox
7808 /usr/local/lib/firefox/firefox -contentproc -childID 3 -isForBrowser -prefsLen 6609 -prefMapSize 211324 -parentBuildID 20200328195933 -appdir /usr/local/lib/firefox/browser 21378 tab
62662 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 222 -prefMapSize 211324 -parentBuildID 20200328195933 -appdir /usr/local/lib/firefox/browser 21378 tab
29214 /usr/local/lib/firefox/firefox -contentproc -parentBuildID 20200328195933 -prefsLen 1 -prefMapSize 211324 -appdir /usr/local/lib/firefox/browser 21378 gpu
21378 firefox
Flags: needinfo?(tb)

:gaston, jcs: It looks like the content process expects the pledge.content file to be in /etc/firefox.

On the unaffected machines, I had the default files in /etc/firefox from previous experimenation, and it worked, on the affected machines, there was no /etc/firefox directory and doing

# cp -r /usr/local/lib/firefox/browser/defaults/preferences/ /etc/firefox

or even just

# mkdir /etc/firefox
# cp /usr/local/lib/firefox/browser/defaults/preferences/pledge.content /etc/firefox

fixed all the issues I saw including the right and middle click breakage.

(In reply to Theo Buehler from comment #31)

:gaston, jcs: It looks like the content process expects the pledge.content file to be in /etc/firefox.

On the unaffected machines, I had the default files in /etc/firefox from previous experimenation, and it worked, on the affected machines, there was no /etc/firefox directory and doing

# cp -r /usr/local/lib/firefox/browser/defaults/preferences/ /etc/firefox

or even just

# mkdir /etc/firefox
# cp /usr/local/lib/firefox/browser/defaults/preferences/pledge.content /etc/firefox

fixed all the issues I saw including the right and middle click breakage.

that's very strange, because iirc the code is supposed to look for the file in /etc/firefox/XX and if not found default to the one shipped in /usr/local/lib/firefox/browser/defaults/preferences/XX..

(In reply to Landry Breuil (:gaston) from comment #32)

that's very strange, because iirc the code is supposed to look for the file in /etc/firefox/XX and if not found default to the one shipped in /usr/local/lib/firefox/browser/defaults/preferences/XX..

I agree that it makes no sense.

The ktrace seems to show that the fallback to default files code works as intended and that the pledges and unveils are applied in both cases.

The observation remains (confirmed on all my machines) that the presence or absence of /etc/firefox/pledge.content (with either the defaults or disable) determines whether right and middle clicks work or not.

Adding /usr/local/lib/firefox/browser r to /usr/local/lib/firefox/browser/defaults/preferences/unveil.content fixes the issue for me. It's probably not the correct solution though.

Note that /usr/local/lib r is unveiled, with the intent that all its subdirectories are available to the content process for reading.

There is a call to unveil the pledge.$PROCESS file in https://dxr.mozilla.org/mozilla-central/source/dom/ipc/ContentChild.cpp?q=contentchild.cpp&redirect_type=direct#4491. If the pledge.content file is in /etc/firefox, there is no harmful side effect to that unveil call. However, if the fallback to the default in /usr/local/lib/... is taken, the unveil semantics remove read access to the tree below /usr/local/lib, and a subsequent stat call for /usr/local/lib/firefox/browser fails. This ultimately results in the breakage (presumably because it's the directory specified for the -appdir argument of the content process).

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is -- (Backlog,) indicating it has has not been previously triaged, the bug's Severity is being updated to -- (default, untriaged.)

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is -- (Backlog,) indicating it has has not been previously triaged, the bug's Severity is being updated to -- (default, untriaged.)

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is -- (Backlog,) indicating it has has not been previously triaged, the bug's Severity is being updated to -- (default, untriaged.)

Severity: normal → S3

The severity of these bugs was changed, mistakenly, from normal to S3.

Because these bugs have a priority of --, indicating that they have not been previously triaged, these bugs should be changed to Severity of --.

Severity: S3 → --

The Bugbug bot thinks this bug should belong to the 'Core::DOM: UI Events & Focus Handling' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → DOM: UI Events & Focus Handling
Product: Firefox → Core
Component: DOM: UI Events & Focus Handling → Widget
Assignee: nobody → landry
Status: NEW → ASSIGNED

Since now the OpenBSD packaging infra ships the files in /etc/firefox by default since 75, i'm not sure anymore if this bug is still valid (maybe for ppl trying to run firefox from nightly without having files in /etc/firefox ? But then they'd need to add the config files to browser/defaults/preferences..), but i'm upstreaming in https://phabricator.services.mozilla.com/D116561 the patch from Theo we've been using since then.

Pushed by alissy@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/6f88e04953ea Do not unveil an already visible pledge file since that interferes with other unveils. r=gcp
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 91 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: