[meta] Reports about serious performance regressions after the 57 update from users with a11y turned on

NEW
Unassigned

Status

()

defect
P3
critical
2 years ago
3 months ago

People

(Reporter: philipp, Unassigned)

Tracking

(Depends on 6 bugs, {meta, perf, regression})

57 Branch
Unspecified
Windows
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(relnote-firefox 57+, firefox-esr52 unaffected, firefox57+ wontfix, firefox58+ wontfix, firefox59+ wontfix)

Details

Reporter

Description

2 years ago
[Tracking Requested - why for this release]:
we're seeing reports from users on various support channels that they are experiencing serious performance problems after the update to firefox 57, like multi-second freezes/hangs/lags after all sorts of user actions (opening tabs, scrolling, typing).

we have performance profiles from three affected users - i am no expert in interpreting those but as it seems to me, the lag there is mainly taking place in functions relating to accessibility:
*"Opening new tab causes intense lag": https://perfht.ml/2AWi9GO - https://support.mozilla.org/en-US/questions/1186701
*"FF57 lagging on everything?": https://perfht.ml/2APLjX7 - https://www.reddit.com/r/firefox/comments/7dec8t/ff57_lagging_on_everything/
* https://perfht.ml/2zA5ifX - https://www.camp-firefox.de/forum/viewtopic.php?f=1&t=122400

the amount of questions we get about this problem is worryingly high i'd say and most users probably aren't aware that accessibility functions are enabled in their profile, which may be a separate issue...
Flags: needinfo?(rkothari)
Flags: needinfo?(jmathies)
Flags: needinfo?(aklotz)
Note that perf problems are documented here, among others: https://support.mozilla.org/en-US/kb/can-i-use-my-screen-reader-new-firefox

So the main question is, what client is triggering accessibility for these users. Note also that there's a checkbox in Advanced Settings telling Firefox to block accessibility from being turned on.
Reporter

Comment 2

2 years ago
i'll be tagging all the user questions at sumo that i can find that have information on them that accessibility is turned on and they experience serious lags at https://support.mozilla.org/en-US/questions/firefox?owner=all&tagged=bug1418290&show=all
Phillip, fyi there's an option under privacy for this now. You don't have to tell users to manipulate about:config.

https://support.mozilla.org/en-US/questions/1185780
Flags: needinfo?(jmathies) → needinfo?(madperson)
Reporter

Comment 4

2 years ago
ok, i see (& will adapt the reply in future responses). i mainly ni'd you as the regression engineering "boss" for 57, because you may know better who this should be escalated to.
Flags: needinfo?(madperson)
Two of the users who posted about:support text have accessibility enabled. Both are unknown injected clients.

Our support approach here is mostly correct, ask users to disable and see if it fixes the issue. However we should be careful about suggesting this to users we detect are running a known screen reader - disabling a11y for these users will complete break their ability to interact with Firefox. This information would be under the accessibility about:support section.

Also we should follow these support incidents up with a question about what accessible software they run.
Let's try to get STR of course.

In particular the about:support accessibility section as jimm mentions in comment 5 might help.

Similar information can come from "menu ≡ > help ? > troubleshooting information" which philipp mentioned in a support incident: https://support.mozilla.org/en-US/questions/1185780.

The relevant section is:
 "accessibility": {
   "isActive": true,
   "forceDisabled": 0,
   "handlerUsed": true,
   "instantiator": "UIAUTOMATION|"
 }

The important bit is "instantiator". We don't know all the things that cause UIAUTOMATION to kick in, but it sounds like a lot of users are hitting this so hopefully we can discover a case locally soon.
Priority: -- → P1
Put another way, we need to know what unexpected software is causing accessibility to instantiate and use the API so heavily that these users are noticing such a drastic regression.
Looks like the right info is in place for this, clearing ni
Flags: needinfo?(aklotz)
Reporter

Comment 10

2 years ago
a user at https://support.mozilla.org/en-US/questions/1187076 is getting fortinet av software as the "instantiator". incidentally he is also the 3rd user with this problem explicitly referencing having a surface 4 pro device...
(In reply to [:philipp] from comment #10)
> a user at https://support.mozilla.org/en-US/questions/1187076 is getting
> fortinet av software as the "instantiator". incidentally he is also the 3rd
> user with this problem explicitly referencing having a surface 4 pro
> device...

Unfortunately there are a lot of these types of applications out there. We're planning on using various sticks and carrots to try and persuading these vendors to move to a more appropriate extension mechanism.  The new browser window 'accessibility indicator', which should be rolling out in 58 or 59, will be the first step in this process.

Updated

2 years ago
Depends on: 1418514
Keywords: meta

Updated

2 years ago
Depends on: 1418521
Some people on Twitter say RealPlayer is the culprit. One person says disabling a11y services disabled the RealPlayer add-on and solved the perf issue.
(In reply to Kohei Yoshino [:kohei] from comment #12)
> Some people on Twitter say RealPlayer is the culprit. One person says
> disabling a11y services disabled the RealPlayer add-on and solved the perf
> issue.

Interesting! Thanks for the data point. I'll file a bug on that.

Updated

2 years ago
Depends on: 1418535
Reporter

Comment 14

2 years ago
would https://mzl.la/2zOShhS indicate that this is affecting 7%+ of the userbase?
In that case we should remotely disable the a11y service right away by flipping the pref...
Severity: major → critical
relnote-firefox: --- → ?
Keywords: perf
I got an about:support from one user affected by this in bug 1419028.
See Also: → 1419028

Updated

2 years ago
Duplicate of this bug: 1419028
One user that ran into this problem said it only affected their newer model HP laptop.  Their old laptop was not affected.  I wonder if we could test drive one of those at a store to see if it reproduces and get it in house to diagnose the issue.

Comment 19

2 years ago
On bug1418749 -- Problem was with RealPlayer / RealDownloader .From Kohei Yoshino [:kohei]Comment 12 ; Turning off accessibility services disables the downloader and resolves the performance issue. 
Will there be a fix for this ? RealDownloader is the only one I have found that can perform almost all of my needs.
Please ask RealNetworks.

Comment 21

2 years ago
OK . Thanks I will try to contact them .

Updated

2 years ago
Depends on: 1419418
Here is a user with a slightly different report, but solved by disabling a11y in options:

https://twitter.com/sigfig/status/933378222594646018

They report sites would not complete loading until the defocused the window.
Comment hidden (offtopic)
Comment hidden (offtopic)
Comment hidden (offtopic)
Comment hidden (offtopic)
Comment hidden (offtopic)
Comment hidden (offtopic)
Comment hidden (offtopic)

Updated

2 years ago
Depends on: 1421005

Updated

2 years ago
Depends on: 1421010

Updated

2 years ago
Depends on: 1421018
This issue has been actively investigated since unthrottling updates. We have put some mitigations in place and continue to find other low risk/high reward fixes.
Flags: needinfo?(rkothari)
(In reply to [:philipp] from comment #14)
> would https://mzl.la/2zOShhS indicate that this is affecting 7%+ of the
> userbase?

This data still indicates 6.83% as of today. According to Bug 1421010, only 0.3% of users have RealPlayer installed. I see people on Twitter still say Firefox is so slow after upgrading. Maybe whitelisting common a11y services would be better than blacklisting some rogue external apps?
(In reply to Kohei Yoshino [:kohei] from comment #31)
> (In reply to [:philipp] from comment #14)
> > would https://mzl.la/2zOShhS indicate that this is affecting 7%+ of the
> > userbase?
> 
> This data still indicates 6.83% as of today. According to Bug 1421010, only
> 0.3% of users have RealPlayer installed. I see people on Twitter still say
> Firefox is so slow after upgrading. Maybe whitelisting common a11y services
> would be better than blacklisting some rogue external apps?

White listing isn't a short term solution here but it is something we're looking at. We're also shipping the a11y indicator and working on general performance of the a11y code base to address.

FYI we have telemetry evidence that this is limited to a small subset of users. We're still researching to fully understand the scope though.

Updated

2 years ago
Assignee: nobody → jmathies
Tracking for 58 and 59. It is likely too late for a fix in 57 based on the number of dependent issues here.
This was relnoted for 57.0 with advice for screen reader users to switch to ESR 52: https://www.mozilla.org/en-US/firefox/57.0/releasenotes/

Comment 35

Last year
(In reply to Kohei Yoshino [:kohei] from comment #12)
> Some people on Twitter say RealPlayer is the culprit. One person says
> disabling a11y services disabled the RealPlayer add-on and solved the perf
> issue.

+1
(In reply to Todd from comment #35)
> > Some people on Twitter say RealPlayer is the culprit. One person says
> > disabling a11y services disabled the RealPlayer add-on and solved the perf
> > issue.
> +1

Todd, RealPlayer was blocked in Firefox 57.0.1. Are you still seeing it run on your system?

Comment 37

Last year
Real Player is installed, but not showing in Firefox.  I was experiencing performance issues after update to Quantum and disabling/preventing accessibility services from accessing browser in Privacy & Security seems to have remedied that (symptom).  The only real software difference between this machine and other endpoints on the network is Real Player.  Still researching underlying cause of the performance issue.  See  https://bugzilla.mozilla.org/show_bug.cgi?id=1437354
Too late for a fix for 59 here. But it sounds like the issue may have improved since we blocked RealPlayer.
This has turned into a meta bug, so I'll unset tracking flags for 60 as there doesn't seem to be anything directly actionable for the release here.
Reporter

Updated

Last year
Depends on: 1472530
Moving to p3 because no activity for at least 24 weeks.
See https://github.com/mozilla/bug-handling/blob/master/policy/triage-bugzilla.md#how-do-you-triage for more information
Priority: P1 → P3
Summary: Reports about serious performance regressions after the 57 update from users with a11y turned on → [meta] Reports about serious performance regressions after the 57 update from users with a11y turned on

Updated

3 months ago
Assignee: jmathies → nobody
You need to log in before you can comment on or make changes to this bug.