Note: There are a few cases of duplicates in user autocompletion which are being worked on.

[meta] Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app

NEW
Unassigned

Status

()

Core
Widget: Cocoa
--
enhancement
12 years ago
2 months ago

People

(Reporter: Prachi Gauriar, Unassigned, Mentored)

Tracking

(Depends on: 4 bugs, Blocks: 1 bug, {meta})

Trunk
All
Mac OS X
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [parity-Safari][parity-Chrome][lang=obj-c][tpi:-])

Attachments

(3 obsolete attachments)

(Reporter)

Description

12 years ago
Camino (2005071808) doesn't integrate with Tiger's Dictionary.  In many other
applications (including Safari), if one presses cmd-ctrl-d (US keyboard) while
hovering the mouse over a word, a panel pops up with the definition of that
word.  This doesn't happen in Camino.

The dictionary lookup and panel display is apparently handled by the
PopupDictDaemon, which can be found at
/Applications/Dictionary.app/Contents/SharedSupport/PopupDictDaemon.app.  While
it is unclear how this daemon works, I suspect it uses the Accessibility APIs to
determine what word is under the cursor.  As such, I believe that making Camino
Accessibility friendly would be necessary (and perhaps sufficient) to fix this.

These two links should be helpful.  Baiscally, the browser view just has to
implement the NSAccessibility protocol.

http://developer.apple.com/documentation/Cocoa/Conceptual/Accessibility/index.html
http://developer.apple.com/documentation/Cocoa/Reference/ApplicationKit/ObjC_classic/Protocols/NSAccessibility.html
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)
*** Bug 310874 has been marked as a duplicate of this bug. ***
Comment hidden (obsolete)
Comment hidden (obsolete)
*** Bug 323556 has been marked as a duplicate of this bug. ***
*** Bug 323556 has been marked as a duplicate of this bug. ***
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)

Updated

11 years ago
Blocks: 336306
Comment hidden (obsolete)

Comment 17

11 years ago
No, this is to support the Dictionary popup window that you get when holding Command-Control-D over any word.
Target Milestone: --- → Camino2.0

Updated

11 years ago
Depends on: 352329
Comment hidden (obsolete)

Comment 19

11 years ago
*** Bug 328153 has been marked as a duplicate of this bug. ***
*** Bug 363340 has been marked as a duplicate of this bug. ***

Updated

11 years ago
Duplicate of this bug: 376098

Updated

10 years ago
Duplicate of this bug: 383254

Updated

10 years ago
Duplicate of this bug: 397317
Summary: Gecko doesn't integrate with Tiger's Dictionary (needs to support accessibility) → Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app (needs to support accessibility)

Updated

9 years ago
Duplicate of this bug: 435618

Comment 25

9 years ago
Re comment 18: Håkan, you mention that we'd be a lot closer in fixing this bug when bug 352329 is fixed. That bug was eventually fixed in 2006. 

What is it that remains before this bug can be fixed?

Comment 26

9 years ago
Last I looked, it was not 100% documented which methods are required for Dictionary.app to work, but it seems that it has to do with different text queries that are sent to the top-most view that it uses in order to find out the word at a specific point. We implement some of these in ChildView (nsChildView.mm), to what extent I'm not sure.

Part of this bug is finding out which methods (probably on ChildView) are required for this... It's too bad Dictionary.app is not open source!

Comment 27

9 years ago
One more thing:  basically the creator of this bug and others (including me) previously assumed that Dictionary support was tied to Accessibility support, but it turns out that maybe a11y is not a requirement for this to work. 

Maybe Dictionary only uses some of these text callbacks on NSViews to find out the currently selected text. If that's the case, this bug really doesn't depend on bug 157209 (full a11y support on OS X) but can be fixed independently.
Comment hidden (obsolete)
Comment hidden (obsolete)
Comment hidden (obsolete)
Hardware: PowerPC → All
This bug includes all Mozilla-based applications.
Does finishing the OS X Services implementation have any bearing on this?
(In reply to comment #32)
> Does finishing the OS X Services implementation have any bearing on this?

If Håkan's "new" theory in the second half of comment 27 is correct and if the Services implementation provides them, perhaps.  Otherwise, no ;)  

I looked at ADC not too long ago and still couldn't find any documentation on what the hover-based Cmd-Ctrl-D lookup required to work; it's a black box.  (For instance, OOo Aqua does have Accessibility support that more-or-less works with VoiceOver, Cmd-Ctrl-D does not work there.)

Comment 34

8 years ago
You have to implement the NSTextInput protocol on one of your NSViews.
QA Contact: os.integration → accessibility-apis
Sounds like something for ChildView to do, then; thanks, smfr!  --> Widget:Cocoa
Component: Disability Access APIs → Widget: Cocoa
QA Contact: accessibility-apis → cocoa
Summary: Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app (needs to support accessibility) → Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app (needs to implement NSTextInput protocol)
Looks like our NSTextInput implementation is already relatively complete - maybe it's easy to add the rest that's required for Dictionary support. Masayuki, do you want to look into this?
Comment hidden (obsolete)
also, it looks like NSTextInput is deprecated in 10.5 and above, and the NSTextInputClient protocol is its replacement.
(In reply to comment #37)
> what happened to this bug?

Nothing. Feel free to pick it up; our NSTextInput implementation is somewhere in nsChildView.mm.

(In reply to comment #38)
> also, it looks like NSTextInput is deprecated in 10.5 and above, and the
> NSTextInputClient protocol is its replacement.

NSTextInputClient was introduced in 10.5, so we can start using it when we've finally decided that we'll truly drop 10.4 support. However, converting to the new interface will be very simple because they're very similar, so there's no need to wait for that before fixing this bug.
Comment hidden (advocacy)

Comment 41

7 years ago
There's a plugin which adds a 'Look Up in Dictionary' context menu for the selected word and opens up Mac OS X's Dictionary.  https://addons.mozilla.org/en-US/firefox/addon/7261
Comment hidden (off-topic)
You can use MXR to browse Firefox source code and search for file names:
http://mxr.mozilla.org/mozilla-central/find?text=&kind=text&string=nschildview.mm

I'd start with adding a "#define DEBUG_IME 1" which will printf messages whenever an NSTextInput method is called. Then I'd check which methods are called when pressing Cmd+Ctrl+D and add implementations to those that aren't implemented yet.
Comment hidden (off-topic)
Comment hidden (off-topic)
Comment hidden (off-topic)
I'm wondering if the recent observance and tackling of the '100 Papercuts' project would consider the inclusion of this bug, see [1] 


[1] https://wiki.mozilla.org/Firefox/Projects/UX_Priorities_3.7
[2] http://limi.net/articles/firefox-ux-team-update-8/

Updated

7 years ago
Duplicate of this bug: 566161
Created attachment 445588 [details] [diff] [review]
wip

This is a rough start that only works in focused text fields.
> -  nsIWidget* rootWidget = mGeckoChild->GetTopLevelWidget();
> -  NSWindow* rootWindow =
> -    static_cast<NSWindow*>(rootWidget->GetNativeData(NS_NATIVE_WINDOW));
> -  NSView* rootView =
> -    static_cast<NSView*>(rootWidget->GetNativeData(NS_NATIVE_WIDGET));
> -  if (!rootWindow || !rootView)
> -    return rect;
> +  NSRect rect = NSZeroRect;
>    GeckoRectToNSRect(r, rect);
> -  rect = [rootView convertRect:rect toView:nil];
> -  rect.origin = [rootWindow convertBaseToScreen:rect.origin];
> +  rect = [self convertRect:rect toView:nil];
> +  rect.origin = [[self window] convertBaseToScreen:rect.origin];

Why is this change needed? Doesn't this change break the method behavior on XUL panel?

> characterIndexForPoint

You can use NS_QUERY_CHARACTER_AT_POINT event. See the patch of bug 308471. I cannot land the patch soon for IME because there are some issues. But you can use a part of my patch for this bug.
(In reply to comment #50)
> > -  nsIWidget* rootWidget = mGeckoChild->GetTopLevelWidget();
> > -  NSWindow* rootWindow =
> > -    static_cast<NSWindow*>(rootWidget->GetNativeData(NS_NATIVE_WINDOW));
> > -  NSView* rootView =
> > -    static_cast<NSView*>(rootWidget->GetNativeData(NS_NATIVE_WIDGET));
> > -  if (!rootWindow || !rootView)
> > -    return rect;
> > +  NSRect rect = NSZeroRect;
> >    GeckoRectToNSRect(r, rect);
> > -  rect = [rootView convertRect:rect toView:nil];
> > -  rect.origin = [rootWindow convertBaseToScreen:rect.origin];
> > +  rect = [self convertRect:rect toView:nil];
> > +  rect.origin = [[self window] convertBaseToScreen:rect.origin];
> 
> Why is this change needed? Doesn't this change break the method behavior on XUL
> panel?

Ah, panels... I see. It's not needed, I just wanted to simplify the code.

> > characterIndexForPoint
> 
> You can use NS_QUERY_CHARACTER_AT_POINT event. See the patch of bug 308471. I
> cannot land the patch soon for IME because there are some issues. But you can
> use a part of my patch for this bug.

Oh, good!
Depends on: 308471
No longer depends on: 157209
Created attachment 448502 [details] [diff] [review]
v1

With the patch for bug 308471, all we need to do here is implement the textStorage method (which isn't even part of the NSTextInput protocol, but of NSTextView).
This will at the moment only work for focused text boxes. Moreover, the text that the dictionary popup overlays on top of the targeted word will have the wrong font and font size. That's because the NSAttributedStrings we return only contain plain text.
Attachment #448502 - Flags: review?(masayuki)
Attachment #445588 - Attachment is obsolete: true
Depends on: 569331
Depends on: 569334
I've filed bug 569331 and bug 569334 on the focus and visual issues.
Comment on attachment 448502 [details] [diff] [review]
v1

Isn't the right result type (NSTextStrage*)?

http://developer.apple.com/mac/library/documentation/Cocoa/Reference/ApplicationKit/Classes/NSTextView_Class/Reference/Reference.html#//apple_ref/occ/instm/NSTextView/textStorage
Indeed!
Created attachment 456363 [details] [diff] [review]
v2

with NSTextStorage instead of NSAttributedString
Attachment #448502 - Attachment is obsolete: true
Attachment #456363 - Flags: review?(masayuki)
Attachment #448502 - Flags: review?(masayuki)
Attachment #456363 - Flags: review?(masayuki) → review+

Comment 57

7 years ago
Will this land in time for Firefox 4? This was reviews back in July
I haven't landed this yet because I haven't had a chance to look into bug 569331. The current implementation only works for focused elements, and I think landing it in this state would make for a quite frustrating experience.
Comment hidden (off-topic)
Comment hidden (advocacy)
Comment hidden (advocacy)

Updated

6 years ago
Duplicate of this bug: 626566
Comment hidden (advocacy)
Comment hidden (advocacy)

Updated

5 years ago
Blocks: 778544

Updated

5 years ago
Blocks: 687026

Updated

5 years ago
Whiteboard: [parity-Safari][parity-Chrome]

Updated

5 years ago
Duplicate of this bug: 837451
Comment hidden (advocacy)
I have found the documentation for this on Apple's website: https://developer.apple.com/library/mac/#releasenotes/Cocoa/AppKit.html#//apple_ref/doc/uid/TP30000741

---------------

 Quick Look Gesture
There is now API to receive the Quick Look gesture (also known as the dictionary look up gesture). The user can perform this gesture with a three finger single tap, or the appropriate keyboard shortcut (Cmd-Ctl-D by default). There are two new NSResponder methods: an event method and an action method.

/* Perform a Quick Look on the content at location in the event. If there are no Quick Look items at the location, call super.
   Also, see quickLookPreviewItems: further below.
*/
- (void)quickLookWithEvent:(NSEvent *)event;

/* Perform a Quick Look on the text cursor position, selection, or whatever is appropriate for your view. If there are no Quick Look
   items, then call [[self nextResponder] tryToPerform:_cmd with:sender]; to pass the request up the responder chain. Eventually
   AppKit will attempt to perform a dictionary look up. Also see quickLookWithEvent: above.
*/
- (void)quickLookPreviewItems:(id)sender;

To support the new event responder method, there is also a new NSEventType, NSEventTypeQuickLook. The only valid properties of and NSEventTypeQuickLook event are: -locationInWindow and -modifiers. A quick look event does not come in through the normal event mechanism, therefore there is no corresponding event mask for it, nor should you attempt to look for it in sendEvent: or with nextEventMatchingMask:… methods.

NOTE: NSApplication's default implementation of quickLookPreviewItems:, will perform a dictionary lookup. NOTE: There is a bug that causes the quickLoop responder methods to jump from NSWindow directly to NSApp bypassing NSWindowController, NSDocument, and NSAppDelegate.

-------------------------------

Stephen, would you be willing to mentor this bug?
Flags: needinfo?(spohl.mozilla.bugs)
(In reply to Jared Wein [:jaws] from comment #67)
> Stephen, would you be willing to mentor this bug?

Sure. :-)
Flags: needinfo?(spohl.mozilla.bugs)
Whiteboard: [parity-Safari][parity-Chrome] → [parity-Safari][parity-Chrome][mentor=spohl][lang=obj-c]
Duplicate of this bug: 787092
Comment hidden (advocacy)

Updated

4 years ago
Duplicate of this bug: 952958
Comment hidden (advocacy)
(Assignee)

Updated

3 years ago
Mentor: spohl.mozilla.bugs@gmail.com
Whiteboard: [parity-Safari][parity-Chrome][mentor=spohl][lang=obj-c] → [parity-Safari][parity-Chrome][lang=obj-c]
Comment hidden (off-topic)
Comment hidden (off-topic)
I've spent some time last week to read up on the history of this bug and to understand what still needs to be done. Unfortunately, comment 67 turned out to be a bit of a red herring. The Quick Look gesture API is only available on 10.8 and above, which is very limiting for us. More importantly however, its main purpose is to override the default lookup gesture behavior, which is the dictionary lookup. Since we're specifically trying to make the default dictionary lookup work reliably, this API is of no use to us.

To make the dictionary lookup work, we basically need to do three things:
1. Implement NSTextInput -characterIndexForPoint. Tracked in bug 308471.
2. No longer constrain content query events to focused elements. Tracked in bug 569331.
3. Get the text styling right for the dictionary lookup.

Since the first two steps already have dedicated bugs, I'm inclined to use this bug to track the work for the proper text styling. We should also be able to land each of these steps in isolation, which should gradually improve our dictionary lookup experience.
Comment hidden (obsolete)
(In reply to Stephen Pohl [:spohl] from comment #75)
> To make the dictionary lookup work, we basically need to do three things:
> 1. Implement NSTextInput -characterIndexForPoint. Tracked in bug 308471.
> 2. No longer constrain content query events to focused elements. Tracked in
> bug 569331.
> 3. Get the text styling right for the dictionary lookup.
> 
> Since the first two steps already have dedicated bugs, I'm inclined to use
> this bug to track the work for the proper text styling.

Turns out that I missed the dependent bug 569334, which is already tracking the work for proper font styling. It looks like this bug here is merely a meta bug at this point.
Summary: Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app (needs to implement NSTextInput protocol) → Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app (needs to implement NSTextInput protocol) [meta]
Keywords: meta
Summary: Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app (needs to implement NSTextInput protocol) [meta] → [meta] Gecko doesn't support Cmd-Ctrl-D lookup in Mac OS X Dictionary.app
This bug works for me in the latest Nightly. Does anyone know what bug do we now track for that implementation?
(In reply to Xidorn Quan [:xidorn] (UTC+11) from comment #78)
> This bug works for me in the latest Nightly. Does anyone know what bug do we
> now track for that implementation?

This is now a meta bug. All depending bugs should be fixed (or removed from the list) before closing this bug.

Comment 80

3 years ago
I can confirm that Cmd-Ctrl-D and the three-finger-tap gesture successfully opens the Dictionary popover with the caveats detailed in Bug 569331 and Bug 569334. Should we create a bug regarding the lack of a "Look up in Dictionary" context menu entry, or at least make that bug blocked until Bug 569331 is fixed?
(In reply to Nathan Yee [:nyee] from comment #80)
> Should we create a bug regarding the lack of a "Look up in
> Dictionary" context menu entry, or at least make that bug blocked until Bug
> 569331 is fixed?

Filing a bug for the lack of a context menu entry sounds like a great idea, and making it block this bug here (bug 301451) seems like the right thing to do since this bug is a meta bug, tracking all the work related to dictionary lookup. Thank you!

Updated

3 years ago
Depends on: 1116391

Comment 82

3 years ago
I have done the bug report. It would be much appreciated if :spohl or someone else can check and confirm it.
Depends on: 1123625
Depends on: 1145224
Depends on: 1177943
Whiteboard: [parity-Safari][parity-Chrome][lang=obj-c] → [parity-Safari][parity-Chrome][lang=obj-c][tpi:?]
No longer depends on: 569331

Updated

a year ago
Whiteboard: [parity-Safari][parity-Chrome][lang=obj-c][tpi:?] → [parity-Safari][parity-Chrome][lang=obj-c][tpi:-]

Updated

9 months ago
Depends on: 1277095
See Also: → bug 1329799

Updated

6 months ago
Depends on: 1275486
You need to log in before you can comment on or make changes to this bug.