732872 - Firefox runs extremely slowly when Windows Speech Recognition is running [meta]

Status: RESOLVED FIXED

(Core :: Disability Access APIs, defect)

Description

[Michael Trigoboff]

Attachment: sysinfo.jpg

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Build ID: 20110928134238

Steps to reproduce:

Due to wrist tendon problems, I have started using Windows Speech Recognition and Windows Speech Recognition Macros. In this situation, if I open more than two tabs in Firefox, the browser slows to a crawl. It's painfully slow. This does not happen in, for instance, Chrome. I've attached a screen shot displaying my system information.

Comment 1

[alexander :surkov (:asurkov)]

Another report on the web http://www.speechcomputing.com/node/2720. It sounds MSAA support it guilty.

Comment 2

[alexander :surkov (:asurkov)]

(In reply to Michael Trigoboff from comment #0)
> Due to wrist tendon problems, I have started using Windows Speech
> Recognition and Windows Speech Recognition Macros. In this situation, if I
> open more than two tabs in Firefox, the browser slows to a crawl. It's
> painfully slow. This does not happen in, for instance, Chrome. I've attached
> a screen shot displaying my system information.

Could you please provide simple steps to reproduce the problem? English is not my native language and I had hard times to fight with WSR.

Comment 3

[Michael Trigoboff]

(In reply to alexander :surkov from comment #2)
> (In reply to Michael Trigoboff from comment #0)
> > Due to wrist tendon problems, I have started using Windows Speech
> > Recognition and Windows Speech Recognition Macros. In this situation, if I
> > open more than two tabs in Firefox, the browser slows to a crawl. It's
> > painfully slow. This does not happen in, for instance, Chrome. I've attached
> > a screen shot displaying my system information.
> 
> Could you please provide simple steps to reproduce the problem? English is
> not my native language and I had hard times to fight with WSR.

1) Install and run Windows Speech Recognition.
2) Install and run Windows Speech Recognition Macros.
3) Open Firefox and then open 5 tabs.

On my machine, this slows Firefox to a crawl.

Comment 4

[alexander :surkov (:asurkov)]

(In reply to Michael Trigoboff from comment #3)

> On my machine, this slows Firefox to a crawl.

is it slow on WSR commands or slow on keyboard/mouse?

Comment 5

[Michael Trigoboff]

Firefox becomes very slow in its response to the keyboard and mouse.

Comment 6

[alexander :surkov (:asurkov)]

there's no single bottleneck here, UIAutomation just inspects the tree. From what I've seen sometimes we spend up to 30% for accessible states calculation, 25% is spend by UIAutomationCore.dll (no in Firefox calls).

we need bug assignee.

Comment 7

[David Bolter [:davidb] (NeedInfo me for attention)]

We need a test so that we can measure fixage.

Comment 8

[David Bolter [:davidb] (NeedInfo me for attention)]

I've trained speech rec on my Windows machine. Firefox is slower but usable, although my machine is pretty beefy. I hope to run some profiling next but it sounds like Alexander already has.

Alexander what can we do here?

Note a possible separate bug: advanced text edit WSR commands don't work in our edit boxes.

Comment 9

[David Bolter [:davidb] (NeedInfo me for attention)]

No noticeable slowdown in IE.

Comment 10

[Michael Trigoboff]

I'm the one who originally reported this bug. My machine's pretty beefy too. Intel Core i7 950, Intel SSD, 6 gig RAM. My problem may be that I like to open 20 - 30 tabs sometimes. This worked fine for me until I started using WSR and WSR Macros.

Comment 11

[David Bolter [:davidb] (NeedInfo me for attention)]

(In reply to Michael Trigoboff from comment #10)
> I'm the one who originally reported this bug. My machine's pretty beefy too.
> Intel Core i7 950, Intel SSD, 6 gig RAM. My problem may be that I like to
> open 20 - 30 tabs sometimes. This worked fine for me until I started using
> WSR and WSR Macros.

Yes thanks. My comment 9 was ambiguous but I was noticing that FF does slow down with WSR much more significantly than IE. There is definitely churn and I suspect lack of UIA implementation is to blame (as per comment 6), since we need to use the MS proxy.

Comment 12

[alexander :surkov (:asurkov)]

(In reply to David Bolter [:davidb] from comment #8)
> I hope to run some profiling next but
> it sounds like Alexander already has.

yes, see comment #6

> Alexander what can we do here?

We have two options here:
1) implement basics of UIA to avoid State() calls
2) rearchitect states getting

in general we need to do both.

Comment 13

[alexander :surkov (:asurkov)]

The build on top of all blocking bugs (the latest one is bug 756383), I don't see big lags when scrolling (there are problems on loading still). So basically we've got good perf improvement.

Comment 14

[Wayne Mery (:wsmwk)]

(In reply to alexander :surkov from comment #13)
> The build on top of all blocking bugs (the latest one is bug 756383), I
> don't see big lags when scrolling (there are problems on loading still). So
> basically we've got good perf improvement.

All th eblocking bugs are fixed. So is are the issues associated with this era essentially finished off?

Comment 15

[alexander :surkov (:asurkov)]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #14)
> (In reply to alexander :surkov from comment #13)
> > The build on top of all blocking bugs (the latest one is bug 756383), I
> > don't see big lags when scrolling (there are problems on loading still). So
> > basically we've got good perf improvement.
> 
> All th eblocking bugs are fixed. So is are the issues associated with this
> era essentially finished off?

we'd need to get some feedback how it feels now before calling it done.  Michael, can you give it a try please?

Comment 16

[Wayne Mery (:wsmwk)]

(In reply to alexander :surkov from comment #15)
> (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment
> #14)
> > (In reply to alexander :surkov from comment #13)
> > > The build on top of all blocking bugs (the latest one is bug 756383), I
> > > don't see big lags when scrolling (there are problems on loading still). So
> > > basically we've got good perf improvement.
> > 
> > All th eblocking bugs are fixed. So is are the issues associated with this
> > era essentially finished off?
> 
> we'd need to get some feedback how it feels now before calling it done. 
> Michael, can you give it a try please?

I've been in touch with Michael and he is no longer using Firefox. NI:surkov to decide disposition of bug.

Comment 17

[alexander :surkov (:asurkov)]

We still have perf problems overall in the a11y engine, I would suggest to keep this bug open until the major a11y bottlenecks are resolved. Then we can revisit this bug and make decision on it.

Comment 18

[Jason]

(In reply to alexander :surkov from comment #17)
> We still have perf problems overall in the a11y engine, I would suggest to
> keep this bug open until the major a11y bottlenecks are resolved. Then we
> can revisit this bug and make decision on it.

Hello, 

Like Michael, I've recently started using microsoft speech recognition due to problems with my wrist and fingers. Firefox is pretty unusable with speech recognition. I do not have a beefy computer, but previously I would have >10 tabs open with no significant issue. With speech recognition turned on, Firefox becomes barely usable. Even scrolling through reddit is a problem, with it increasingly lagging, causing all of firefox to become unresponsive (even with a mouse) until I close speech recognition.

I'm unfortunately going to switch to chrome because of this. However, if you've had 2 people report this bug, I'm sure a whole lot more are experiencing this issue and being forced away from Firefox :(

Thanks, 
Jason

Comment 19

[Jason]

Just to add to my above comment... when I look at the resource monitor everything seems fine (CPU & memory are nowhere near 100% utilization)

Comment 20

[Roman R.]

I am entering this text using Windows Speech Recognition, and Firefox currently probably has more than 100 open tabs.  The browser seems to be operating pretty well.

Using 55.0a1 (2017-05-25) (32-bit)

Comment 21

[morpheus]

I am experiencing this same problem. Firefox works great on my PC until I turn on Windows Speech Recognition. At that point, Firefox becomes completely unresponsive. Clicking on tabs doesn't change anything, keypresses (like CTRL-TAB) don't do anything, Firefox basically freezes. I have to turn off WSR and then kill Firefox and restart it for it to return to normal.

I have a brand new Surface Book 3 with 10th gen i7 processor and 32GB RAM.
Using FF 78.0.1 (64-bit)

OS Name Microsoft Windows 10 Home
Version 10.0.18363 Build 18363
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
System Manufacturer Microsoft Corporation
System Model Surface Book 3
System Type x64-based PC
System SKU Surface_Book_3_1899
Processor Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz, 1498 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date Microsoft Corporation 6.19.140, 4/13/2020
SMBIOS Version 3.3
Embedded Controller Version 255.255
BIOS Mode UEFI
BaseBoard Manufacturer Microsoft Corporation
BaseBoard Product Surface Book 3
BaseBoard Version Not Available
Platform Role Slate
Secure Boot State On
PCR7 Configuration Elevation Required to View
Windows Directory C:\windows
System Directory C:\windows\system32
Boot Device \Device\HarddiskVolume1
Locale United States
Hardware Abstraction Layer Version = "10.0.18362.752"
Time Zone Eastern Daylight Time
Installed Physical Memory (RAM) 32.0 GB
Total Physical Memory 31.6 GB
Available Physical Memory 20.9 GB
Total Virtual Memory 36.3 GB
Available Virtual Memory 23.8 GB
Page File Space 4.75 GB
Page File C:\pagefile.sys
Kernel DMA Protection On
Virtualization-based security Running
Virtualization-based security Required Security Properties Base Virtualization Support
Virtualization-based security Available Security Properties Base Virtualization Support, Secure Boot, DMA Protection, UEFI Code Readonly, SMM Security Mitigations 1.0, Mode Based Execution Control
Virtualization-based security Services Configured Hypervisor enforced Code Integrity
Virtualization-based security Services Running Hypervisor enforced Code Integrity
Windows Defender Application Control policy Enforced
Windows Defender Application Control user mode policy Off
Device Encryption Support Elevation Required to View
A hypervisor has been detected. Features required for Hyper-V will not be displayed.

Comment 22

[Marcela (Please NI request to Brindusa Tot)]

Hi morpheus!
I'm trying to see if we can resolve some old bugs. Thank you for your report. Could you inform us if it still occurs on your end? If it is possible, could you try to reproduce it on latest Nightly version and tell us your input? You can download it from here: https://nightly.mozilla.org/.
Thanks!

Comment 23

[Andrew]

Hi Marcela,
I am not the person you were responding to but I came upon this thread because I'm having the same issue as them. This issue is still occurring on my machine and I was able to reproduce it with the latest nightly version. I thought it might help to give my input since your comment was two months ago.

Comment 24

[geogan]

I notice this too just now. When I turned on windows speech recognition it appeared that the entire Firefox had locked up... didn't respond to any mouse clicks at all.. no idea why. According to Task Manager the task has not hung or crashed.

Comment 25

[wrobel.przemyslaw]

Same problem for me. Can't switch tabs or scroll pages. Putting recognition to the sleep mode unfreezes Firefox.

Comment 26

[bugzilla]

I also have this issue and can reproduce it using the following steps on Windows 10, but the steps should be similar to Windows 7 as well.

1. Start Firefox using your preferred method
2. Navigate to www.duck.com
3. Open Windows Speech Recognition by opening the start menu, searching for "Windows Speech Recognition" and selecting the result
   3a. optionally train the microphone, if you can skip this step and just get the floating microphone widget to appear onscreen then you've configured Windows Speech Recognition enough to start
4. Ensure Windows Speech Recognition is in Off mode by right-clicking the microphone icon and selecting "Off: do not listen to anything that I say"
5. Bring Firefox back into focus by clicking inside the application window
6. Activate Windows Speech Recognition listening mode by pressing Ctrl + Windows. The microphone should turn blue.
7. Attempt to right click inside the URL bar, observe no response from Firefox. Windows Speech Recognition may flash orange with a message indicating it did not understand what you said, even if you had your mic muted and no input. (I can only presume the message is misleading and WSR received some kind of error response?)
   7a. Optionally attempt to click around other controls in Firefox, observe they do not respond either and that WSR blips orange for a moment after you attempt the interaction
8. De-activate WIndows Speech Recognition by pressing Ctrl + Windows again. The microphone should turn grey.
9. After 10-20 seconds the URL bar context menu opens (if you did not do step 7a)

Expected results: Firefox continues to respond and I am able to use commands like 'show numbers' to expose input controls and type into textboxes using Windows Speech Recognition dictation.

Actual results: Firefox becomes unresponsive, freezing in terms of user interaction, but continuing to paint the window and appear to Windows OS to be responsive. Firefox acts as if the user were giving no input at all. 5-10 seconds after Windows Speech Recognition has de-activated, Firefox catches up to the last few interactions.

Comment 27

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:asa, since the bug has high severity and recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Comment 29

[bugzilla]

Believe the severity here would be either S2 (because there is no workaround to using WSR on Firefox) or S3 (because using WSR is probably not considered a major Firefox feature).

Comment 30

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:morgan, since the bug doesn't have a severity set, could you please set the severity or close the bug?

For more information, please visit BugBot documentation.

Comment 31

[Morgan Reschenberg [:morgan]]

:Jamie are you aware of this? / is this a roadmap item?

Comment 32

[James Teh [:Jamie]]

I am aware of it. It should be significantly improved by Cache the World, but it's not something I've had a chance to test.

If it's not sufficiently improved by Cache the World, it will be further improved by UI Automation, which is on our roadmap.

Comment 33

[James Teh [:Jamie]]

If anyone who experienced this is still watching this bug, can you provide any feedback on the situation now? Our new accessibility architecture (AKA Cache the World) shipped in Firefox 113, which should significantly improve performance for accessibility clients as compared with Firefox versions from 57 through 112.

Comment 34

[bugzilla]

I'm pleased to report that I recreated my test from 2 years ago and the behavior was much better, there was no slowdown in Firefox UI, and I could have windows speech recognition do show numbers voice command and it would highlight all of the various clickable elements within the Firefox window instantly, without delay.

Windows 10 22H2 build 19045.4046 with Firefox 122 64 bit version.

Unless others report new issues I believe this bug may be able to be closed as fixed.

Comment 35

[bugzilla]

(In reply to bugzilla from comment #34)

I'm pleased to report that I recreated my test from 2 years ago and the behavior was much better, there was no slowdown in Firefox UI, and I could have windows speech recognition do show numbers voice command and it would highlight all of the various clickable elements within the Firefox window instantly, without delay.

Windows 10 22H2 build 19045.4046 with Firefox 122 64 bit version.

Unless others report new issues I believe this bug may be able to be closed as fixed.

I want to add a thank you to the developers who made these fixes happen! It's such a nice experience to be able to report bugs and have them be fixed without having to do the work myself (I do development for my day job so I'm on the other side of the fence most of the time), great job folks!

Comment 36

[James Teh [:Jamie]]

You're welcome. I'm sorry it took so long. It required some pretty massive re-architecture, but I'm glad it's done now for many reasons.

I'm going to close this as fixed for now, but we can reopen if we get reports of the problem persisting.