1638706 - Hangs when pinning tab

Status: RESOLVED WORKSFORME

(Core :: Disability Access APIs, defect, P3)

Description

[mrbenns]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0

Steps to reproduce:

1. Have some pinned tabs (8 or so)
2. Open new tab
3. Right click tab and click Pin Tab

Actual results:

FF became unresponsive approx 10 seconds, task manager shows high CPU use.

Expected results:

Prior to version 76; Firefox responds more or less instantaneously.

Profile: https://profiler.firefox.com/public/a05e835b02251a90a72b4761cfee8c3b6f10942c/calltree/?globalTrackOrder=0-1&localTrackOrderByPid=23644-1-0~21964-0~&thread=2&v=4

Comment 1

[mrbenns]

Attachment: mozregression log

Comment 2

[mrbenns]

Attachment: about:support

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 4

[Alice0775 White]

Regression window from attached mozregression log:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=8abcb30b61e2c202d34b19774e19fd9156b3bea1&tochange=39473fcaf9721589bb0a37abf03485add1223035

Regressed by:
3f9b009f56908b7a53235f48161c2fc804eb23b4 Iain Ireland — Bug 1620193: Fix GetPropertyPure r=mgaudet

Comment 5

[Iain Ireland [:iain]]

This is very weird.

Bug 1620193 is a correctness fix that shouldn't affect anything other than typed objects, which are nightly-only. The only thing it changes is adding one additional check to a fallible fast path, which can always fall back to a very slightly slower path. The slow path is not nearly slow enough to explain this (and the linked profile is not consistent with extra time being spent in JS).

I've gone over this code with a fine-toothed comb trying to figure out if we missed something relevant to this bug, and I came up empty. I think mozregression got this one wrong. I'm going to punt this back, with the suggestion of pulling on the following thread:

Just before the hang, there's a profiler sample with the following stack:

mozilla::dom::Element::AsXULButton()
mozilla::dom::Element::GetCustomInterface<nsIDOMXULButtonElement>(nsGetterAddRefs<nsIDOMXULButtonElement>)
static mozilla::dom::CustomElementRegistry::CallGetCustomInterface(mozilla::dom::Element*, nsID const&)
LifecycleGetCustomInterfaceCallback
mozilla::dom::LifecycleGetCustomInterfaceCallback::Call<mozilla::dom::Element *>(mozilla::dom::Element* const&, JS::Handle<JS::Value>, JS::MutableHandle<JSObject *>, mozilla::ErrorResult&, char const*, mozilla::dom::CallbackObject::ExceptionHandling, JS::Realm*)
mozilla::dom::LifecycleGetCustomInterfaceCallback::Call(mozilla::dom::BindingCallContext&, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::MutableHandle<JSObject *>, mozilla::ErrorResult&)
JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>)
InternalCall(JSContext*, js::AnyInvokeArgs const&, js::CallReason)

It looks like accessibility code is triggering some sort of lifecycle callback. Given that the hang spends a long time in UIAutomationCore, which appears to be a Windows accessibility library, that seems relevant.

Asa, is this a known issue? One of the people I roped in to look at this thought he remembered seeing something recently about perf regressions in accessibility.

Comment 6

[Asa Dotzler [:asa]]

I don't think we implement any of the UIAutomation accessibility API in Firefox. Our accessibility implementation relies an older Microsoft API. Adding jteh to the bug for confirmation.

Comment 7

[James Teh [:Jamie] (away returning 12 Jan)]

We don't yet implement UI Automation, but UI Automation Core does proxy MSAA (and these days, IA2), which is what we're seeing here.

It looks like some UI Automation client is trying to find an element (I have no way of knowing what), which ends up walking into a content process, resulting in cross-process calls. The question is what UI Automation client. I can't reproduce this myself even with a screen reader running, so this is specific to this particular client.

mrbenns, do you have any idea what might be using UI Automation on your system? Do you have an accessibility tool, an uncommon password manager, an enterprise SSO tool or similar running?

What's also odd is that I'd expect to see AccessibleHandler in the stack between AccWrap_Base::get_accParent and ObjectStubless, but it's not in the profile. Was this profile taken with Nightly or an extracted build (which could include mozregression)? A build which isn't installed will be missing AccessibleHandler, which has a severe (and unfortunately unavoidable) impact on a11y perf.

Comment 8

[James Teh [:Jamie] (away returning 12 Jan)]

(In reply to Iain Ireland [:iain] from comment #5)

One of the people I roped in to look at this thought he remembered seeing something recently about perf regressions in accessibility.

E10s (Firefox 57) caused serious perf regressions for a11y, but much of that has been solved. While there are certainly still known issues and I'm working on those, there haven't been any further a11y perf regressions that I'm aware of recently, certainly not as recently as Firefox 76.

Comment 9

[mrbenns]

The only thing I can think of is a tiny app that can minimise to systray? I use Lastpass as my password manager but the same bug happens when that is off. No SSO or accessibility tools

mrbenns

Comment 10

[mrbenns]

I am happy to take a profile on my normal install

Comment 11

[Jason Orendorff [:jorendorff]]

Judging by the profile, comment 5, and comment 7, this isn't a JS engine issue. Maybe this is a better component.

But I wonder if it isn't malware. If someone isn't using accessibility software, isn't it a bit concerning that the Firefox parent process is handling any accessibility-related messages at all? If so, is there a way to find out if this is a common pattern?

Comment 12

[James Teh [:Jamie] (away returning 12 Jan)]

This seems to be specific to a particular system at this stage.

mrbenns, if you exit the systray application you mentioned, does the problem persist?

(In reply to Jason Orendorff [:jorendorff] from comment #11)

But I wonder if it isn't malware. If someone isn't using accessibility software, isn't it a bit concerning that the Firefox parent process is handling any accessibility-related messages at all?

Unfortunately, quite a few things, even Windows itself in some cases, make accessibility queries. It could be Malware, but it might be something else. We have code to try to detect the app that instantiated accessibility, but it seems that isn't working any more for UI Automation. I need to look into that.

Comment 13

[mrbenns]

Yes, it is still happening when I quit out of the systray app

Stephen

Comment 14

[mrbenns]

Still happening in newest nightlies and now hangs in context menus as well, eg Take Screenshot, Add Bookmark - hangs aboput 20 sec and Task Manager process spikes

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

Clear a needinfo that is pending on an inactive user.

Inactive users most likely will not respond; if the missing information is essential and cannot be collected another way, the bug maybe should be closed as INCOMPLETE.

For more information, please visit auto_nag documentation.

Comment 16

[James Teh [:Jamie] (away returning 12 Jan)]

We still don't really understand what the third party software is trying to do here. However, Cache the World (our accessibility engine re-architecture) makes all accessibility queries much faster and it is enabled by default in Firefox 113, so that should resolve this. Please reopen/comment if the problem persists.