855232 - UIABridge should access focus state in real-time vs. relying on delayed observer events (was: sporadic soft keyboard display)

Status: RESOLVED FIXED

(Firefox for Metro Graveyard :: Input, defect)

Description

[Jim Mathies [:jimm]]

I did some debugging on this earlier in the week. I think the problem here is that we rely on accessible observer events to update focus state in our uia bridge down in widget. Unfortunately the observers fire off the refresh timer, so they can come delayed, after the last input event has been sent. For example:

mozilla::widget::winrt::MetroInput::OnPointerEntered
mozilla::widget::winrt::MetroInput::OnPointerPressed
MetroWidget::GetDPI
mozilla::widget::winrt::MetroInput::OnPointerReleased
mozilla::widget::winrt::MetroInput::OnTapped
UIABridge::GetFocus focus=0
UIATextElement::get_BoundingRectangle 0.000000 0.000000 0.000000 0.000000
mozilla::widget::winrt::UIATextElement::get_IsReadOnly
mozilla::widget::winrt::MetroInput::OnPointerExited
Bridge: EVENT_FOCUS
Bridge: Focus element flags: 1 1 0
Bridge: focus item can be focused
mozilla::widget::winrt::UIABridge::SetFocusInternal
mozilla::widget::winrt::UIABridge::SetFocus
mozilla::widget::winrt::UIATextElement::SetFocusInternal
mozilla::widget::winrt::UIATextElement::SetFocus

What we probably have to do here is query the accessible tree in realtime when windows asks us about it. Starting with the call to UIABridge::GetFocus, which queries for the focused IRawElementProviderFragment. Internally accessible should be up to date at this point.

Comment 1

[Jim Mathies [:jimm]]

Attachment: wip v.1

Comment 2

[Jim Mathies [:jimm]]

Attachment: wip v.1

Comment 3

[Jim Mathies [:jimm]]

Attachment: msaa patch

This turns off some native event msaa code that's confusing the uia implementation in metro.

Comment 4

[Jim Mathies [:jimm]]

Attachment: uia bridge wip v.2

Comment 5

[David Bolter [:davidb] (NeedInfo me for attention)]

Comment on attachment 732391 [details] [diff] [review]
msaa patch

Review of attachment 732391 [details] [diff] [review]:
-----------------------------------------------------------------

r=me. Yeah makes sense for metro mode.

Comment 6

[Jim Mathies [:jimm]]

Attachment: uiabridge changes v.1

UIABridge now queries for state directly from accessible when it needs it. I've also removed some comment cruft.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/10bcc9c3ca1d
https://hg.mozilla.org/mozilla-central/rev/1225c6016d10