1038061 - Support speech recognition on specific media stream

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: speech-input-opt-stream.patch

Current speech recognition API cannot specify media stream, we could introduce an optional parameter in SpeechRecognition.start() to enable following use cases:
  1. multiple microphone selection via gUM media constraints
  2. remote audio stream (WebRTC)
  3. stream from audio file

Comment 1

[Olli Pettay [:smaug][[e-mail]]]

So the current spec doesn't support MediaStreams [1], but it definitely should and while the API
was designed, use of some sort of MediaStreams was discussed.
Could you file a spec bug about this[2].

[1] https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html#speechreco-section
[2] https://www.w3.org/Bugs/Public/enter_bug.cgi?product=Speech%20API

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8455178 [details] [diff] [review]
speech-input-opt-stream.patch

The else {} in SpeechRecognition::Start could use a proper indentation.

Comment 3

[Guilherme Gonçalves [:ggp]]

Comment on attachment 8455178 [details] [diff] [review]
speech-input-opt-stream.patch

+1 on indentation, otherwise looks good to me.

Once we have this, it may also be possible to simplify our testing code, which uses a bit of a hack to inject a custom MediaStream into an existing SpeechRecognition object: http://dxr.mozilla.org/mozilla-central/source/content/media/webspeech/recognition/test/head.js?from=content/media/webspeech/recognition/test/head.js#120

Comment 4

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Spec bug is filed: https://www.w3.org/Bugs/Public/show_bug.cgi?id=26336

Comment 5

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: speech-input-opt-stream.patch

nit picked and update test cases.

try result:https://tbpl.mozilla.org/?tree=Try&rev=2074ce8e0d72

Comment 6

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: speech-input-opt-stream.patch

need bypass gum permission check for test_call_start_from_end_handler.html.

try result: https://tbpl.mozilla.org/?tree=Try&rev=251643e053b7

Comment 7

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8455962 [details] [diff] [review]
speech-input-opt-stream.patch

ggp, please check the test changes

Comment 8

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: speech-input-opt-stream.patch

fix intermittent error on b2g emulator debug build by avoiding gUM in testcase, carry smaug's r+ and sr+.

try result: https://tbpl.mozilla.org/?tree=Try&rev=213dfd8f6324

Comment 9

[Guilherme Gonçalves [:ggp]]

Comment on attachment 8456672 [details] [diff] [review]
speech-input-opt-stream.patch

Looks good, thank you!

Comment 10

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a038f26e9bbd

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a038f26e9bbd