1048861 - Voice search integration in search activity

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: Search Activity, enhancement)

Description

[Lucas Rocha (:lucasr)]

We should probably consider integration voice search somehow. Feedback from Paul W. on G+.

Comment 1

[:Margaret Leibovic]

Is there some system IME support for voice input? Have we ever thought about this for Fennec in general?

Comment 2

[Aaron Train [:aaronmt]]

Voice search from within the activity, or voice search to activate the default search activity (i.e, "Ok Google" to launch Google Search from the Android Google Experience Launcher on Nexus devices) ?

Comment 3

[:Margaret Leibovic]

I assumed we meant voice input for search, like a microphone icon at the end of the input box.

Comment 5

[Paul [pwd]]

Think this has been resolved wrongly. This bug is about being able to click an icon within the new Firefox Search interface and then being able to the phone and have it pull up a search. i.e. parity with Google Now

Bug 774309 from my quick read seems to be about the lack of intents that Firefox has when compared to competitors.

Comment 6

[:Margaret Leibovic]

(In reply to Paul [pwd] from comment #5)
> Think this has been resolved wrongly. This bug is about being able to click
> an icon within the new Firefox Search interface and then being able to the
> phone and have it pull up a search. i.e. parity with Google Now
> 
> Bug 774309 from my quick read seems to be about the lack of intents that
> Firefox has when compared to competitors.

I agree, this bug was filed about using voice input in the search activity. It seems like bug 774309 was filed about getting Fennec to open when using the google search bar widget.

I also see that bug 778588 was filed about adding voice input support to the Fennec location bar.

Comment 7

[(inactive) Jim Chen [:jchen] [:darchons]]

(In reply to :Margaret Leibovic from comment #6)
> I agree, this bug was filed about using voice input in the search activity.
> It seems like bug 774309 was filed about getting Fennec to open when using
> the google search bar widget.

That's my impression as well. The IME API doesn't handle voice specifically (the keyboard handles it and just hands us the text), but I imagine there's a separate voice input API that we can use.

Comment 9

[Wesley Johnston (:wesj)]

I think we'd just fire off one of these intents and wait for the result:
http://developer.android.com/reference/android/speech/RecognizerIntent.html#ACTION_RECOGNIZE_SPEECH

Just need some designs if we want to do this.

Comment 10

[:Margaret Leibovic]

Also looping in antlam, since this would affect the visual design if we want some sort of microphone icon.

Just to be clear, this bug is about the search activity. Bug 778588 is about the location bar. But perhaps we can work on bug 778588 as part of the more general toolbar refinements happening in bug 1052004.

Comment 11

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Yeah I've thought about this for both the activity and the location bar. There are a lot of animations and visuals that come with something as powerful as voice search since it tends to demand different forms of feedback to show the user "hey I'm on!". As far as an icon on the end of the input bar, that's pretty much what I was thinking too but what happens afterwards is the finicky part as far as I can tell right now.

I do have questions about weather or not this would lock users into Google search suggestions or if there's any privacy concerns here? I don't know why but I feel like there might be.. just a thought!

Comment 12

[Robert (rrayborn, he/him)]

I just want to point out that there is a less aggressive intermediate solution that could be used and/or as a bridge to a future solution.  Right now Search Activity is missing the built-in Speech-to-Text on the Keyboard (HTC M8 and N7 tested).  This keyboard feature is present in Nightly browser.  I'm not sure how/why it's not there.

For the record, I do think that a fancier UI element is a better long-term solution.

Comment 13

[Eric Edens [:eedens]]

(In reply to Rob from comment #12)
> I just want to point out that there is a less aggressive intermediate
> solution that could be used and/or as a bridge to a future solution.  Right
> now Search Activity is missing the built-in Speech-to-Text on the Keyboard
> (HTC M8 and N7 tested).  This keyboard feature is present in Nightly
> browser.  I'm not sure how/why it's not there.
> 
> For the record, I do think that a fancier UI element is a better long-term
> solution.

Bug 1049282 enables voice search on the keyboard (as well as gesture).

Comment 14

[Aaron Train [:aaronmt]]

I think this should really be considered of high importance. In Nightly, I see text "Quickly search for anything you want". The quickest way I search on my personal device is through voice. It's been a fundamental thing in Android forever. It's quicker for me to search using Google Now as I can launch it with voice and if I hit the Google Bar (widget) voice icon, it launches with it in voice mode and I can access voice from the search bar.

Comment 15

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Aaron Train [:aaronmt] from comment #14)
> I think this should really be considered of high importance.

Contrast that with me, who has never used voice entry for anything, especially in public places.

Comment 16

[:Margaret Leibovic]

I agree with Aaron that this is something that can be a core feature for some people. I think it would be worth exploring how much work it would be to add this. Sounds like wesj already looked into it a bit.

However, this would impact the current search branding UX we're adding in bug 1049600, so we would need designs for where we would put the microphone icon.

Comment 17

[Paul [pwd]]

I too agree with Aaron

Comment 18

[Robert (rrayborn, he/him)]

I think it sounds like something that a minority will be passionate about.  A quick search returned this article which claims that of ~2,000 iOS users surveyed late last year, only 15% of them recall using Siri: http://www.ibtimes.com/apple-ios-7-85-percent-people-havent-used-siri-46-percent-think-apple-oversold-its-release-1437900

Comment 19

[Richard Newman [:rnewman]]

I know someone who's done a lot of recent research in this space; let me know if you want me to get a summary of current findings (e.g., re Siri).

Historically speaking -- I worked on voice search at Tellme from 2005-2010 -- speech is only worthwhile if it's *good*, and even then it's a minority feature. (Not well suited to crowded spaces, noisy cars, public spaces when issuing sensitive queries, at work where you'll bother coworkers…)

Re quality: we would need to match or exceed Google's speech-to-text capabilities (both recognition quality and reco speed) to make it worthwhile. Using Google's recognizer (Comment 9) is a good step, but the UX has to be *incredibly* responsive, too.

Comment 21

[Jan Henning [:JanH]]

This is an ex-Search Activity now (bug 1221344).