1508388 - Search hand-off in Activity Stream

Status: RESOLVED FIXED

(Firefox :: Messaging System, enhancement, P2)

Description

[Kate Hudson :k88hudson]

We would like to add a search box to about:privatebrowsing that focuses on the awesomebar when a user starts typing (see user story for design spec).

The search box should show the appropriate placeholder text and search icon depending on what the user's current default search engine is.

In order to prevent lag, we can use the following technique to preemptively switch focus to the awesomebar *before* a keystroke is sent. This is similar to Chrome's strategy:

* When the in-content search input is focused, we should immediately shift focus to the awesomebar via message-passing.
* The awesomebar should apply special styling to prevent it from visually appearing focused (use caret-color: transparent; and prevent the outline on :focus from appearing)
* The in-content search should have a fake focus style that applies an outline and fakes a caret with css;
* When the user starts typing in the awesomebar, a message should be sent to content in order to hide the in-content search input
* When the awesomebar is unfocused, a message should be sent to content in order to unhide the in-content search input

Note that this will require some accessibility review.

Comment 1

[Kate Hudson :k88hudson]

We have decided to implement this on activity stream first in order to test in isolation, it will also require a pref.

Comment 2

[Kate Hudson :k88hudson]

We are looking into a search implementation similar to how Google Chrome's home page works. Visually, the focus shifts from the search input to the awesomebar when a users starts typing but the underlying implementation will shift focus immediately after focusing on the input to the awesomebar.

Is there anything in terms of the Chrome implementation we should be aware of / we should imitate in our implementation for screen readers? What do you think? Briefly looking I saw an aria-hidden tag on the in-content search box and couldn't seem to tab into it, but maybe I'm missing something. Thanks!

Comment 3

[Marco Zehe (:MarcoZ)]

I just tested this, and you canot tab to that in-content input indeed. When I click on the text that says "Enter search term or address", focus gets redirected to the URL bar at the top. This is, initially, also the place Chrome starts out at if you open it without restoring previous tabs. So the screen reader user always lands on the URL bar and *knows* that, too, because it is spoken.

I am having particular problems with the following in the current specification:

(In reply to Kate Hudson :k88hudson from comment #0)
> * When the in-content search input is focused, we should immediately shift
> focus to the awesomebar via message-passing.
> * The awesomebar should apply special styling to prevent it from visually
> appearing focused (use caret-color: transparent; and prevent the outline on
> :focus from appearing)
> * The in-content search should have a fake focus style that applies an
> outline and fakes a caret with css;

I suspect this will mess big deal with magnification software, which relies on knowing where the focus is. So low-vision users who use magnification software such as ZoomText will see that the URL bar has focus, because that's where the system will see the caret even if CSS hides it. Accessibility services always create a caret that assistive technologies can track. So in the above scenario, they'd suddenly have two carets, the actual one and the one faked by CSS, and the one that does have the actual caret (and system focus) won't have the proper styling applied.

I don't know how wide-spread Chrome is among low-vision ZoomText users, or how this works on the Mac with magnification, but I suspect if they do something similar, it will cause equal amounts of confusion.

Adding Jamie for further thoughts.

Comment 4

[Kate Hudson :k88hudson]

Thanks Marco, that's a really good point about magnification software. I also wonder how Chrome handles this issue?

Comment 5

[James Teh [:Jamie]]

For better context, I'd be grateful if you could explain the motivation behind this UX. Why do we want to do this? Isn't it jarring to have the focus "jump" like this when the user starts typing? Why not move the focus as soon as the user clicks the box? Or have buttons like the quick searches in about:newtab?

I'm not sure, but I wonder whether this focus jump may cause problems for users with attention or cognitive disabilities.

As Marco notes, we do create a caret that assistive technology products can track. However, I'm not sure exactly what they do with that caret. The caret itself is invisible; it's just a way to tell the tool where the caret is. The question is what the tool does with that information. At the very least, it will draw the user's attention to that area of the screen because the focus is also there and will probably shift their view depending on settings.

I think the only way we can find out how Chrome handles this is to test it. Eitan, Yura, would one of you be able to take a look at this in Chrome with ZoomText on Windows and/or Zoom on Mac? Kate, what sort of timeline are we talking here; i.e. how quickly do you need this investigated?

Comment 6

[(no longer active)]

One thing I noticed in Chrome which may be relevant here is that the input box inside the new tab page has an aria-hidden="true" attribute unconditionally.  Also, note that in Chrome, the in-content input element is *never* focused.  It only has a mousedown handler, which, once invoked, sets a CSS class on the <body> element that does the job of faking the caret and returns false.

Comment 7

[Eitan Isaacson [:eeejay]]

Yeah, I think Chrome solved the a11y issue by simply disabling the feature for keyboard users entirely. A screen reader/keyboard user would only really be able to focus in the url bar.

I would make ours a11y friendly in the following way:

* "mousedown" will do the behavior mentioned above (draw a fake cursor on the <button>, focus on the awesomebar but visually hide the focus state).
* "keydown" of enter/space will simply move focus to the awesomebar with no focus styling tricks (ie. the user will see the focus jump).

(you can achieve the same thing with "click" by checking clientX == clientY == 0 for a keyboard activated click)

So a mouse user would get the same experience shown in the design spec. A keyboard user could tab to the "field", but it would be styled more like a focused button than a focused text field (ie. no fake caret). Pressing enter/space would visibly send the focus to the url bar. A hidden perk for keyboard users could be that you can "activate" the field with any key, not just space/enter, and it would prepopulate the url field with that value after focus is moved to it.

Comment 8

[Eitan Isaacson [:eeejay]]

Oh, I forgot to mention, in my proposed implementation, the AS search field would be implemented a <button>, but styled as a field.

Comment 9

[Kate Hudson :k88hudson]

We're planning on shipping the experiment in Firefox 66 (we definitely want to see how it performs first)

Comment 10

[Kate Hudson :k88hudson]

As for the motivation, the idea is that since we are investing so much in making the awesomebar better, we'd like to provide that same experience for all users who engage with search box instead of a reduced experience.

Comment 11

[James Teh [:Jamie]]

Thanks. Just for my understanding, why not just remove the search box from the home page or make it a button which redirects to the address bar? I'm guessing the reason is that less experienced users won't necessarily think to search from the address bar (or click on a button), whereas a search box on the home page is more readily identifiable and intuitive.

Comment 12

[Yura Zenevich [:yzen] (please use "needinfo")]

(In reply to Eitan Isaacson [:eeejay] from comment #7)
> Yeah, I think Chrome solved the a11y issue by simply disabling the feature
> for keyboard users entirely. A screen reader/keyboard user would only really
> be able to focus in the url bar.
> 
> I would make ours a11y friendly in the following way:
> 
> * "mousedown" will do the behavior mentioned above (draw a fake cursor on
> the <button>, focus on the awesomebar but visually hide the focus state).
> * "keydown" of enter/space will simply move focus to the awesomebar with no
> focus styling tricks (ie. the user will see the focus jump).
> 
> (you can achieve the same thing with "click" by checking clientX == clientY
> == 0 for a keyboard activated click)
> 
> So a mouse user would get the same experience shown in the design spec. A
> keyboard user could tab to the "field", but it would be styled more like a
> focused button than a focused text field (ie. no fake caret). Pressing
> enter/space would visibly send the focus to the url bar. A hidden perk for
> keyboard users could be that you can "activate" the field with any key, not
> just space/enter, and it would prepopulate the url field with that value
> after focus is moved to it.

Eitan's solution sounds reasonable.

Comment 13

[Ricky Rosario [:rrosario, :r1cky]]

The current search box on Activity Stream is a text input followed by a button:

<input type="text" ... /><button title="Search" ...>...</button>

Both elements are in the tab order for keyboard access and behave as expected:
* The textbox gets focused.
* Pressing the search button sends you to a search results page for the selected engine.

My plan is to keep those two elements when the search hand-off feature is enabled, but change the behavior as follows:
* Remove the textbox from the tab order. Tabbing will take you straight to the button instead.
* Mousedown, ENTER and SPACE on the button will shift focus to the awesomebar and it will be styled like it normally is when focused.
* Mousedown on the textbox will do the tricky focus hand-off. We'll style the in-content search box to appear focused but really focus the Awesomebar and hide it's focus styles.
* Once the user starts typing in the Awesomebar, we'll apply the focused styles to it and we'll hide the in-content search box.
* Blur of the Awesomebar will reset everything to how it was before we started.
* Mousedown of the Awesomebar will keep it focused and apply focus styles if they aren't already. The in content search box gets reset to default state.

I am pretty sure the net effect of this is the same as the proposal in Comment 7.

Comment 14

[James Teh [:Jamie]]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #13)
> * Remove the textbox from the tab order. Tabbing will take you straight to
> the button instead.

How will you remove the text box from the tab order? Depending on how you implement this, users of assistive technology apps (e.g. screen readers) will be able to see the text box. If it's programmatically focusable (i.e. reacts to the .focus() method, even if not in the tab order; e.g. tabindex="-1"), they will also be able to focus it *without* using the mouse. What will happen in this case? This is the reason for the guidance in comment 7 and comment 8.

Comment 15

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to James Teh [:Jamie] from comment #14)
> How will you remove the text box from the tab order? Depending on how you
> implement this, users of assistive technology apps (e.g. screen readers)
> will be able to see the text box. If it's programmatically focusable (i.e.
> reacts to the .focus() method, even if not in the tab order; e.g.
> tabindex="-1")

Actually, in the prototype I have so far, I am listening for the focus event and not mousedown. So if I call `textbox.focus()` programmatically, it still works. I also have `tabindex="-1"` set on it. I guess I should also add `aria-hidden="true"` like Chrome. Is that good enough?

I am just trying to keep the markup+css the same since we'll have this feature behind a pref for a while and need to support the existing code for while.

Comment 16

[James Teh [:Jamie]]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #15)
> Actually, in the prototype I have so far, I am listening for the focus event
> and not mousedown. So if I call `textbox.focus()` programmatically, it still
> works.

It works, but it will be confusing for users using assistive technology who also have sight because "real" focus (and accessibility focus) will hit the address bar, while "visual" focus will remain in the in-page text box.

> I guess I should also add
> `aria-hidden="true"` like Chrome. Is that good enough?

That will hide it from accessibility clients, yes.

I'm concerned that sighted keyboard users will be confused by the text box not being in the tab order. They may try to tab to the text box, only to find they can't. Yes, tab will hit the search button, but normally, a button is used to "submit" the search, not to begin search entry.

Comment 17

[Marco Zehe (:MarcoZ)]

(In reply to James Teh [:Jamie] from comment #16)
> I'm concerned that sighted keyboard users will be confused by the text box
> not being in the tab order. They may try to tab to the text box, only to
> find they can't. Yes, tab will hit the search button, but normally, a button
> is used to "submit" the search, not to begin search entry.

I share that concern. If they can't type in the search box, they'll never think of hitting the search button in the first place. Search is something they expect to just type in, not have to press a button first. If we add telemetry for how often the awesome bar will be activated through the search button, my prediction is that this number will be *very* low.

Comment 18

[Ricky Rosario [:rrosario, :r1cky]]

Alright. I guess we're back to the proposal in Comment 7 :)

I'll try that today.

Comment 19

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4599

Comment 20

[[github robot]]

Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/4a5d8c84759313ae3c47cdbb67fcafae9aed5e35
Fix Bug 1508388 - Search hand-off (#4599)

Comment 21

[Ricky Rosario [:rrosario, :r1cky]]

Notes for QA testing:

* This feature is behind a pref: "browser.newtabpage.activity-stream.improvesearch.handoffToAwesomebar".
* It is currently enabled by default on Nightly.
* Disabling the pref should switch Activity Stream back to the current search experience. It is very important that nothing is regressed in current experience.

With the pref enabled, the expected behavior is:
* Clicking (with mouse) the in-content search box will make it appear focused. When you start typing, the awesomebar appears focused and takes over the search. While interacting with the awesomebar, the in-content search remains hidden.
* When done with the awesomebar (blur), the in-content search should reappear and not look focused.
* The in-content search box should be accessible from the keyboard. Clicking it (space or enter), should transfer focus to the awesomebar.

I think that covers it. Let me know if you have any questions. Thanks!

Comment 22

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/c0f59d493460

Comment 23

[Ricky Rosario [:rrosario, :r1cky]]

:MarcoZ, can you do an a11y check of this feature in nightly? Or suggest who should do it? Thank you!

Comment 24

[Marco Zehe (:MarcoZ)]

I tested this with NVDA on Windows, and found that the keyboard focus transitions properly from the button to the awesome bar, and that there is no search field visible for NVDA on the about:newtab page itself, only the button. I also saw that I can tab to the button using the keyboard, and that it transitions to the awesome bar when I press Enter or Space.

The one thing I cannot check are the visuals of this. But from the dependent bugs, I see that there's already QA going on on this feature.

Comment 25

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Eitan Isaacson [:eeejay] from comment #8)

Oh, I forgot to mention, in my proposed implementation, the AS search field
would be implemented a <button>, but styled as a field.

By changing the <input type="text".../> to a <button/>, we lost the ability to middle click to paste or right click to get the context menu specific for text boxes. See Bug 1518197 and Bug 1518301, for example.

Any thoughts on that? Ideas to workaround?

Comment 26

[Eitan Isaacson [:eeejay]]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #25)

(In reply to Eitan Isaacson [:eeejay] from comment #8)

Oh, I forgot to mention, in my proposed implementation, the AS search field
would be implemented a <button>, but styled as a field.

By changing the <input type="text".../> to a <button/>, we lost the ability to middle click to paste or right click to get the context menu specific for text boxes. See Bug 1518197 and Bug 1518301, for example.

Any thoughts on that? Ideas to workaround?

Yikes. Yeah, I guess faking an input has its limits. I can think of a solution for those issues, but others might crop up as well. Here is a workaround:

1. Make the button contenteditable.
2. Listen for paste events, preventDefault(), stopPropogation(), and send their clipboardData to awesomebar.

If you do the above, you can get rid of the ctrl+v listener that's there, since the paste listener should handle it.

Comment 27

[James Teh [:Jamie]]

(In reply to Eitan Isaacson [:eeejay] from comment #26)

1. Make the button contenteditable.

The problem with this is that it will expose the button as editable text (editable state) to accessibility. That means, for example, that NVDA will switch to focus mode (to allow for interaction) when you press enter on the button instead of clicking it. There's currently no way to prevent this, although various parties have encountered this problem and there has been some discussion about having an aria-editable="false" or similar.

Comment 28

[Eitan Isaacson [:eeejay]]

(In reply to James Teh [:Jamie] from comment #27)

(In reply to Eitan Isaacson [:eeejay] from comment #26)

1. Make the button contenteditable.

The problem with this is that it will expose the button as editable text (editable state) to accessibility. That means, for example, that NVDA will switch to focus mode (to allow for interaction) when you press enter on the button instead of clicking it. There's currently no way to prevent this, although various parties have encountered this problem and there has been some discussion about having an aria-editable="false" or similar.

That's a good point. It would defeat the purpose of making it a button. I dare say you can have a hack where the actual contenteditable element is inside the button and aria-hidden.

Comment 29

[Eitan Isaacson [:eeejay]]

Messed around with a bit, and yeah, that could work. Although it would be going down the deep end..

This would make it so that mouse users can right-click on the button and get the edit actions, while keyboard users would not (and screen readers would present it as a button).

1. I would add another child div to the <button> element:

<div class="fake-editable" tabindex="-1" aria-hidden="true" contenteditable=""></div>

2. CSS:

.fake-editable:focus {
  outline: none;
  caret-color: transparent;
}

.fake-editable {
  position: absolute;
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
}

3. Add a paste listener to that element to intercept pastes and send them to the awesomebar.

Comment 30

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Eitan Isaacson [:eeejay] from comment #29)

I'll try this out today. Thanks a bunch!

Comment 31

[Mehmet]

Hi, thanks for this feature. Something important was missing here, what Chrome's NTP also can: Pressing the ESC-Key should bring back the Search Bar on Firefox NTP. Is this already a known issue or should I file a new one. Thanks.

Comment 32

[Ed Lee :Mardak]

Looks like the behavior in chrome is to clear out the address bar but focus still remains there. Visually the in-content search box does reappear, so I guess it's more of a click target reminder (hey you can click here to.. move focus back to where it already is in the address bar)

Comment 33

[Mehmet]

(In reply to Ed Lee :Mardak from comment #32)

Looks like the behavior in chrome is to clear out the address bar but focus still remains there. Visually the in-content search box does reappear, so I guess it's more of a click target reminder (hey you can click here to.. move focus back to where it already is in the address bar)

Probably, yes. Can Firefox follow this behavior? That would be great.

Thanks in advance.

Comment 34

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Mehmet from comment #33)

Probably, yes. Can Firefox follow this behavior? That would be great.

I guess we could listen for "Esc" and have the in-content search re-appear. Mind filing a bug for that? Thanks!

Comment 35

[Mehmet]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #34)

Mind filing a bug for that? Thanks!

Done: bug 1519141. Thanks.