It is not possible to use Vista's speech recognition application to dictate into a text field on a webpage

NEW
Unassigned

Status

()

Core
Editor
--
major
11 years ago
2 years ago

People

(Reporter: David Gaskill, Unassigned)

Tracking

({access})

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

11 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6

It is not possible to use Vista's speech recognition application to dictate into a text field on a webpage.  Attempting to do so causes the dictation correction panel to open and display the dictated text.  If the appropriate item is then selected text may be transferred into the webpage field.

There is no problem dictating directly into a text field in Internet Explorer.

Reproducible: Always

Steps to Reproduce:
1.Open any web page with a text field 
2.Place cursor in field 
3.Dictate 
Actual Results:  
Dictation correction panel opens displaying one or more options 

Expected Results:  
Dictated words should appear in webpage text field
Component: Form Manager → Editor
Product: Firefox → Core
QA Contact: form.manager → editor
Duplicate of this bug: 395631
Duplicate of this bug: 444914
(Reporter)

Comment 3

10 years ago
Vista is shortly to become the only Windows operating system supported by Microsoft.  It is surely a serious shortcoming of both applications if neither Firefox or Thunderbird will work correctly under this 0S.  Does Mozilla have any plans to address the problem?

Updated

10 years ago
Severity: minor → major
Keywords: access

Comment 4

10 years ago
Vista speech technologies use MSAA so it may be accessibility bug (see http://msdn.microsoft.com/en-us/library/bb875962.aspx):

Windows Vista automatically provides basic speech capabilities to any application that is designed to work with two Windows accessibility technologies: Microsoft Active Accessibility (MSAA) and Microsoft Windows UI Automation (WUIA). At runtime, when speech is used to open or switch to an application, the speech engine queries that application to determine which accessibility features it supports, then works through those.
(Reporter)

Comment 5

10 years ago
From this I presume that neither Firefox nor Thunderbird will work with MSAA or WUIA?  Are there any plans to address this shortcoming?

(In reply to comment #5)
> From this I presume that neither Firefox nor Thunderbird will work with MSAA or
> WUIA?  Are there any plans to address this shortcoming?

Firefox has been supporting MSAA since version 1.5. The question is what calls Vista's speech recognition makes that Firefox doesn't react to, or reacts improperly to.

Comment 7

10 years ago
I would assume we should implement IAccessibleEx interface to add UI Automation. I'll try to look more deeply. Marco, can you confirm the bug since I haven't Vista?
I am not sure I can confirm the bug. I am not certain I can actually use speech recognition since its controls may be visual (training it etc.).
(Reporter)

Comment 9

10 years ago
Vista speech recognition will work, albeit with poor recognition accuracy, "out of the box", without any training.

I don't know how useful this is but I have just tried dictating into Open Office and a text editor called EditPad, (http://www.editpadpro.com/).  Neither will work.

Unsurprisingly it is possible to dictate into any Microsoft application which will accept text.
(In reply to comment #9)
> I don't know how useful this is but I have just tried dictating into Open
> Office and a text editor called EditPad, (http://www.editpadpro.com/).  Neither
> will work.

That doesn't surprise me: OpenOffice.org doesn't implement MSAA at all yet, and EditPad Pro uses a custom widget as its text editor that implements all the standard APIs for a Windows Edit control, but the speech recognition software doesn't know that a window with class TJgSoftEditorControl is behaving like a normal edit.

> Unsurprisingly it is possible to dictate into any Microsoft application which
> will accept text.

Right. This probably uses the text edit API which even NotePad implements, or the RichText API that WordPad implements, and probably has special rules for MS Word.

As for Firefox, even though we expose textboxes as MSAA type Edit, we probably don't implement something else the speech recognition software needs, but which isn't clearly spelled out. Screen readers for the blind have no problem identifying the textboxes, either in HTML or XUL, as such, so I would assume the implementation is generally correct.

Comment 11

10 years ago
If I get correctly from articles http://msdn.microsoft.com/en-us/accessibility/cc307845.aspx then we have two choices a) implement UI automation or b) implement IAccessibleEx (some kind of partial implementation of UI automation), sure if we want to get firefox more accessible on windows vista (I guess speech is only one issue we meet).

Marco, iirc there was some discussion about UI Automation implementation on a11y.org but I don't remember details. Do you?

Comment 12

10 years ago
If the industry can come together than we might be able to merge IAccessibleEx with IAccessible2 somewhat, which would make this much easier. That's dependent on how things go at http://www.accessinteropalliance.org/
Status: UNCONFIRMED → NEW
Ever confirmed: true

Comment 13

10 years ago
BTW, like IAccessible2, IAccessibleEx is also built on top of MSAA. That means we support part of IAccessibleEx. I would have though it's enough for basic control, because for simple things like buttons, links, menus, that's all you really need.

We probably need to ask Microsoft why the basics aren't working.

Comment 14

10 years ago
Aaron, if you keep someone from Microsoft then can you write them a letter and cc me because I'm actually not sure IAccessibleEx will help here since it provides UA Automation partially only.

Comment 15

10 years ago
So, microsoft guys said we need TSF (Text Services Framework, http://msdn.microsoft.com/en-us/library/ms629032(VS.85).aspx). The irony is we have been asked to be mentors for TSF implementation in Gecko (http://groups.google.com/group/mozilla.dev.accessibility/browse_thread/thread/76b3fdb5a807a2f6/058b4ba3e0361978?lnk=gst&q=TSF#058b4ba3e0361978) in the google summer project bounds.
Duplicate of this bug: 441375

Comment 17

10 years ago
So TSF is not going to make it into 3.1? When will I be able to use Speech Recognition with Fx? To me this is not an necessarily an accessibility feature. Wouldn't we all prefer to speak to our computers instead of having to type and use the mouse? Speech Recognition in Vista is vastly improved over XP. Other aspects of Speech Recognition work with Fx. However, the most useful part is being able to dictate to a text box in Fx for forums. Plus, I don't use Thunderbird but if I did I would want it to accept Speech Recognition like Outlook Express does.
(Reporter)

Comment 18

10 years ago
I am the guy that first reported this bug over a year ago now - see the beginning of this thread.

I couldn't agree more with the previous posting.  Vista's speech recognition application is outstanding but it can't be used with Mozilla products.

How difficult is it to incorporate TSF?

Hopefully someone will find time to do it in the next 12 months…

David

Comment 19

10 years ago
From comment #80 in bug https://bugzilla.mozilla.org/show_bug.cgi?id=88831 TSF missed the feature freeze for Fx 3.1 "by a lot" and thus it cannot be pushed into 3.1. But it should land on the trunk in November.  So, if you like to use the nightlies then not too much longer to wait.  If you read the referenced bug you will see that it started in 2001 and just now is Fx about to get TSF. At one point, they were sure it would make it into 3.1 so it's coming soon.

Comment 20

10 years ago
it looks like we're looking at yet another year of waiting for this issue to be resolved.  A shame really, considering, as you mentioned above, just the vast amount of potential Vista's speech recognition has been showing since its inception.
(Reporter)

Comment 21

9 years ago
Have all attempts to fix this bug been abandoned?  It has been present since Vista was launched and now Vista is about to this superseded by Windows 7.  The problem also exists in Windows 7 beta.  Maybe this will be fixed just in time for the launch of Windows 8?
David, Bug 88831, which introduced support for the Text Services Framework, was fixed about a month ago on Firefox trunk AKA 3.6a1pre. So if you're willing, you can test out how well Vista's speech recognition fairs with that development version.
(Reporter)

Comment 23

9 years ago
I'd love to test the development version but I am only a user not a programmer and have no idea how to get the development version from the trunk even if I knew what the trunk was...

Would I be right in thinking that the same fix would fix the same bug which also exists in Thunderbird?  How could I get somebody to do something about this?

As I said previously the same problem exists with Windows 7 beta.  Will the fix that fixed the problem in Vista also work for Windows 7?  I'd be happy to try that as well if only I knew how to get the development version from the trunk.
(Reporter)

Comment 25

9 years ago
Marco kindly e-mailed me instructions for downloading the development version.  This installed Minefield version 3.2a1pre.

Sadly I have to report that it is still not possible to dictate into text fields.  I tried several different websites with text fields all of which I could dictate into when they were opened in Internet Explorer.  None of them would accept dictation when opened in the development version of Firefox.

Comment 26

9 years ago
You have to enable TSF first by going to about:config and changing the value of intl.enable_tsf_support from false to true.
Yes Aiko is right, at least until bug 478029 is fixed (then it should be true by default).
(Reporter)

Comment 28

9 years ago
Could somebody tell me how to find "about:config"?

Comment 29

9 years ago
Sure. Just type or copy & paste about:config into the address bar and press Return. Please read the warning before you click the button. Now, type intl.ena into the filter; intl.enable_tsf_support should be the only option still available, double click it to change its value.

I'm not sure whether this option is just checked once at startup so you'd better restart the browser at this point.

Please make a backup of your current profile or create a new one before you start any lengthy experiments, this combination of a nightly build and a somewhat experimental feature is not likely to be very stable.
(Reporter)

Comment 30

9 years ago
firefox-3.6a1pre.en-US.win32.installer.exe allows text to be dictated from Vista's speech recognition application into a text field such as the "Additional Comments" field on this thread but only if dictation is continuous.

To see this effect dictate a short phrase into the field.  The phrase is printed.

Pause for a few seconds.

Dictate again.  The new dictation is printed.

Pause for a few seconds

Dictate again.  Nothing happens

The field has apparently lost the focus of the speech recognition application and it can only be regained by manually deleting all the text in the field.

The effect can also be seen by pasting some text into an empty field and then trying to dictate into it.

Typing into an empty field and then dictating into it allows this initial dictation to be printed but the field then loses focus.

Updated

8 years ago
Duplicate of this bug: 556952

Updated

8 years ago
Duplicate of this bug: 391781

Comment 33

8 years ago
See bug #556952:  not even the address bar works with Speech Recognition.

Steps to reproduce:

1. Open Speech Recognition
2. Open Firefox.
3. Say "go to address."  Doesn't work.
4. Say "press control-L."
5. Dictate a URL.
(Reporter)

Comment 34

8 years ago
Jonathan, I originally reported this bug  back in the days of Vista but the problems are exactly the same under Windows 7.

As you can see from this thread fixing the bug depends upon the implementation of TSF.  This must be extremely difficult because it has not proved possible for nearly three years.

Updated

8 years ago
Duplicate of this bug: 599855
(Reporter)

Comment 36

7 years ago
Nearly a year has elapsed since my last comment and still nothing has been done and apparently nobody is working on it.

Vista came and went and it was never possible to dictate into either Firefox or Thunderbird .

Windows 7 has been around now for a couple of years and still it isn't possible to dictate into either Firefox or Thunderbird.

If no date can be put upon the implementation of TSF would it be possible to put a date on when work might start on attempting the implementation?

Comment 37

7 years ago
If you enable TSF does this work as expected? ('intl.enable_tsf_support' = true)
(Reporter)

Comment 38

7 years ago
No.  Please see Comment 30.

Comment 39

7 years ago
(In reply to comment #38)
> No.  Please see Comment 30.

Ok, so with TSF enabled, it works, except for the focus problem you mentioned. The focus problem should be filed as a separate bug and marked as blocking bug 478029. This bug can then be resolved as a dupe of bug 478029. (Or alternatively, morph this into the bug about the focus problem and mark it blocking bug 478029.)

I would suggest not mentioning "Vista" in bug titles, a better description would be something along the lines of "focus problem in TSF support with Windows speech recognition".

Comment 40

7 years ago
[as far as your original question goes about default support - I'd suggest asking 
Masayuki in bug 478029 for a status update. I'd be curious about that myself, since tablet / touch input support is going to be *real* important with Win8.]

Comment 41

7 years ago
I am surprised that you can use Speech Recognition. When I switch it on I have problems when scrolling (stops often) and problems to enter text - has delays too.

This all under Win7 and every version of Firefox (3.6 to 4). No problems with other browsers.

You don't have these problems?
(Reporter)

Comment 42

7 years ago
I didn't say I can use speech recognition in either Firefox or Thunderbird.  The problems I described under Comment 30, (which are essentially the same as you are experiencing, make it completely unusable.

As requested by Jim Mathies I have filed a separate bug report, 650833, on the loss of focus. (Incidentally I haven't figured out how to put a hyperlink on a bug reference).
OS: Windows Vista → All
(Reporter)

Comment 43

7 years ago
I didn't say I can use speech recognition in either Firefox or Thunderbird.  The problems I described under Comment 30, (which are essentially the same as you are experiencing, make it completely unusable.

As requested by Jim Mathies I have filed a separate bug report, 650833, on the loss of focus. (Incidentally I haven't figured out how to put a hyperlink on a bug reference).
(Reporter)

Comment 44

7 years ago
It is coming up to the 4th the anniversary of my filing of this bug report and I still cannot dictate into  Firefox text fields.  Even more importantly than this I can't dictate into Thunderbird.  Do you think TSF will ever be implementedin these application?
You need to log in before you can comment on or make changes to this bug.