1640535 - severe input lag when using multiple keyboards under linux(X11)

Status: RESOLVED INCOMPLETE

(Core :: Widget: Gtk, defect, P3)

Description

[Ckat]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

I use an ergonomic split keyboard (two physically separate keyboards) and have been noticing input lag for a long while in firefox(at least since quantum). I just recently noticed this was directly related to quickly switching between different keyboards.

to reproduce just connect a second keyboard and spam keys on both (asdf<space>+jkl;<space> for example, with left and right hand on different keyboards) for a while in any input field, after a couple 1000 chars severe input lag will occur. this does not happen when typing on one keyboard.

at this time I cannot confirm this is excluxive to linux (x11), but thats what I use.

Actual results:

severe input lag after alternating between two or more keyboards on linux.

Expected results:

regardless of switching between keyboards no extra input lag should occur.

Comment 1

[Ckat]

the user agent is off because I have it setup to represent the 'most used user agent' for fingerprinting reasons. I'm currently on the latest stable (76)

Comment 2

[Ckat]

in the case of someone stumbling on this, or it never being fixed. after finding out the cause I did write a small utility to merge separate hardware inputs into one virtual one. which I have confirmed eliminates this issue from occuring in firefox as well.

Comment 3

[Clara Guerrero ( Need Info Brindusa Tot Please)]

Hi Ckat,

Do you encounter this lag in other browsers as well or just firefox?

I do not have right environment set up in order to reproduce this issue on my end but I will set the component for it and maybe one of our developers will be able to reproduce it successfully. Can you try this on the latest version of nightly meanwhile? You can download it from here: https://nightly.mozilla.org/

Thanks for the report and the additional details on the temporary fix.

Best regards,

Clara.

Comment 4

[Ckat]

(In reply to Clara Guerrero from comment #3)

Do you encounter this lag in other browsers as well or just firefox?

This lag is specific to anything firefox, I have tested pretty much every version of firefox and the lag exists there (regular esr, older versions, current stable, nightly, even odd forks such as waterfox and icecat). any browser based on something non firefix(chormium, chrome, falkon, qutebrowser, opera etc) doesn't seem to have this issue. Should be noted no other application, regardless of it using gtk, qt or something else, has this same issue, it seems isolated to firefox.

I do not have right environment set up in order to reproduce this issue on my end but I will set the component for it and maybe one of our developers will be able to reproduce it successfully. Can you try this on the latest version of nightly meanwhile? You can download it from here: https://nightly.mozilla.org/

I can confirm it persists on the very latest nightly as well.

Comment 5

[Chuck]

Attachment: Profile screenshot indicating gdk_x11_keymap_have_bidi_layouts()

I experience this also.

Concrete steps to reproduce the problem:

1. Be using X11 (not wayland).
2. Have two keyboards (eg: a built-in laptop keyboard and a USB keyboard).
3. In Firefox, with one hand on each keyboard, mash keys for about a minute / about 2000 keypresses.
4. There is now quite a lot of input delay. Touch-typists typing with one hand on each keyboard will easily out-pace Firefox's ability to display what they are typing. The problem persists until Firefox is restarted.

It doesn't appear to matter what you're typing into. The problem occurs:

• Typing into a <textarea>.
• Typing into the URL bar.
• Typing into the body of about:blank. Characters are not displayed, so there's no feedback that anything is wrong, but the UI still hangs for many seconds after you finish typing, and two-keyboard text input everywhere is slow until Firefox is restarted.

The problem appears to be related to nsBidiKeyboard::Reset() calling gdk_x11_keymap_have_bidi_layouts() frequently.

https://share.firefox.dev/33hFT8v - Profile of typing into a <textarea>
https://share.firefox.dev/3jixAPh - Profile of typing into the address bar
https://share.firefox.dev/3n5vpkq - Profile of typing into the body of about:blank

See also https://gitlab.gnome.org/GNOME/gnome-shell/issues/1858 for a similar issue in gnome-shell.

Comment 6

[Chuck]

Attachment: firefox-1640535-workaround-dont-check-for-bidi-keyboard-layouts.patch

Removing that gdk_x11_keymap_have_bidi_layouts() call (which removes support for bidirectional keyboard layouts) causes the performance problem to go away.

Comment 7

[Ckat]

I can confirm it was this bidirectional keyboard code for me as well. the problems are completely gone with this patch.

Thanks for the help, what are the chances of this making it to upstream in some form?

Comment 8

[Ckat]

Possibly a hint as to where the source of this stems from: the main difference with having multiple keyboards plugged in is they generate a MappingNotify event when a key on another keyboard is pressed. I remember looking into it before with the hope of fixing it, but I can't currently find any code in the codebase that handles these events you get flooded with while using multiple keyboards.

My guess would be these events cause the reset to be called(which probably isnt needed), which in turn calls the gdk_x11_keymap_have_bidi_layouts() excessively.

Comment 9

[Ckat]

I've written a little stub of gdk_keymap_have_bidi_layouts which just returns false as a less intrusive workaround to this issue, which can also be used for keeping count of how many times this is called. this reveals it seems to be even worse than I initially expected.

only typing out a few sentences with my previously mentioned split keyboard results in thousands of gdk_keymap_have_bidi_layouts calls. each time a key on another keyboard is pressed it also seems to be called more often. first few times it only gets called ~10 times each time a key on another keyboard is pressed, the next time even more, and so on. at the moment of typing this comment I'm seeing it get called thousands of times each time I press a key on another keyboard.

this means what I've previously mentioned related to MappingNotify events being handled wrong might not be all thats going on here, as those remain singular per 'keyboard switch'.

Comment 10

[fausto.nunez]

Slightly unrelated, but users of split keyboards are also subject to problems in X11 that lead to exactly the same issue, but OS-wide: https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1777708

I had that issue and switched to Regolith (i3 + gnome-flashback) and the problem is gone.

I now only have the Firefox-specific issue described in this thread.

Just wanted to leave this here in case other users report the X11 problem here by mistake.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mstange, could you have a look please?

For more information, please visit BugBot documentation.

Comment 12

[Markus Stange [:mstange]]

If anyone is still seeing this, please capture a new profile and link it here. The existing profiles in this bug are showing time spent in gdk_x11_keymap_have_bidi_layouts but they're missing symbols for the callers to that function.

Comment 13

[Jens Stutte [:jstutte]]

(In reply to Markus Stange [:mstange] from comment #12)

If anyone is still seeing this, please capture a new profile and link it here. The existing profiles in this bug are showing time spent in gdk_x11_keymap_have_bidi_layouts but they're missing symbols for the callers to that function.

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:stransky, since the bug has recent activity, could you have a look please?

For more information, please visit BugBot documentation.