967896 - Add 'continuous' support for WebSpeech API and adapt PocketSphinxService

Status: RESOLVED DUPLICATE of bug 1185233

(Core :: DOM: Core & HTML, defect)

Description

[Joe Olivas]

Attachment: adding-continuous.patch

The WebSpeech API has support for a continuous mode, in addition to the 'single shot' recognition.

I order for more than once command to be recognized, this needs to be implemented.

My first attempt at basic support is attached. It works, but is likely not ideal.

Comment 1

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8370405 [details] [diff] [review]
adding-continuous.patch

> SpeechRecognition::NotifyFinalResult(SpeechEvent* aEvent)
> {
>-  ResetAndEnd();
>+  if(!mContinuous) {
>+    ResetAndEnd();
>+  } else {
>+    // continue recording.
>+    mEndpointer.Reset();
>+    SetState(STATE_WAITING_FOR_SPEECH);
>+  }
So what happens to mSpeechDetectionTimer?
We should still timeout the request at some point, I think.


> SpeechRecognition::GetContinuous(ErrorResult& aRv) const
> {
>-  aRv.Throw(NS_ERROR_NOT_IMPLEMENTED);
>-  return false;
>+  return this->mContinuous;
> }
> 
> void
> SpeechRecognition::SetContinuous(bool aArg, ErrorResult& aRv)
> {
>-  aRv.Throw(NS_ERROR_NOT_IMPLEMENTED);
>+  this->mContinuous = aArg;
>   return;
> }
We don't tend to use this-> since m-prefix tells already that it is a member variable.

But yeah, almost like this patch should be enough.

Comment 2

[Joe Olivas]

It seems that in continuous mode, it'd be good to receive recognition events as they come in, rather than waiting for a soundend event.

Is there any reason to not implement an event 'EVENT_RECOGNITIONSERVICE_RESULT_READY' so that the state machine can better handle recognition callbacks? The event could be handled by STATE_WAITING_FOR_SPEECH and STATE_RECOGNIZING, while I think STATE_WAITING_FOR_RESULT would just jump back to STATE_WAITING_FOR_SPEECH while in continuous mode.

If this seems like a reasonable approach, I'll start prototyping it.

Comment 3

[Joe Olivas]

Attachment: adding-continuous.patch

Updating to support and 'out-of-band' callback for a continuous mode so that there isn't any blocking of audio input while waiting for results to return.

Seems to be working for me so far.

Comment 4

[Joe Olivas]

Attachment: adding-continuous.patch

Missed a part of the previous patch

Comment 5

[Andre Natal]

I will implement this on PocketSphinxService in few days. Some event when utterances come from decoder should be raised to javascript. @smaug, maybe we need another bug on the draft?

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

I don't understand the question. Events for what? 'result' event isn't enough?

Comment 7

[Andre Natal]

Continuous recognition (In reply to Olli Pettay [:smaug] from comment #6)
> I don't understand the question. Events for what? 'result' event isn't
> enough?

Continuous recognition return results while you speak realtime.

Search for dragon dictation and test it. 


If you say I can raise more than one 'result' in a session that does not block the thread, ok. Its enough and this bug have no reason to exist.

 But if I cant raise results while I am speaking, then there is no support to continuous recognition.

Comment 8

[Olli Pettay [:smaug][[e-mail]]]

What has anything of this to do with thread blocking?
But yes, there can be several result events.

Comment 9

[kdavis]

Just found this dupe.

I'll try and make a patch for Bug 1185233 this week. So, I'll mark this as a dupe.

Comment 10

[André Natal]

(In reply to kdavis from comment #9)
> Just found this dupe.
> 
> I'll try and make a patch for Bug 1185233 this week. So, I'll mark this as a
> dupe.
> 
> *** This bug has been marked as a duplicate of bug 1185233 ***

And what's the reason for not keep the original bug instead open a new one?

Comment 11

[kdavis]

(In reply to Andre Natal from comment #10)
> (In reply to kdavis from comment #9)
> > Just found this dupe.
> > 
> > I'll try and make a patch for Bug 1185233 this week. So, I'll mark this as a
> > dupe.
> > 
> > *** This bug has been marked as a duplicate of bug 1185233 ***
> 
> And what's the reason for not keep the original bug instead open a new one?

Bug 1185233 was, unfortunately, opened and commented on, before I saw this bug.

The patch, when submitted, will make sense with respect to the comments on bug 1185233
and thus the patch should be attached to bug 1185233 and not this one as the patch will
be easier to interpret in the context of that bug.