158856 - Mouse operation on Mac IME

Status: RESOLVED DUPLICATE of bug 1262363

(Core :: Widget: Cocoa, enhancement)

Description

[Frank Tang]

currently the mac ime does not work with mouse operation. We need to make 
the mouse opearating work.

Comment 1

[Frank Tang]

add shanjian to the cc list
this bug is depend on shanjian's ime mouse operation on Window bug. Once that
get done, we know what need to change in the layout, and we should implement
that on mac. 

Comment 2

[Frank Tang]

m1.3 alpha

Comment 3

[Frank Tang]

change the qa to kasumi since she file the window one.

what we need to change the following code instead of 
always return kTSMOutsideOfBody


1876 long nsMacEventHandler::HandlePositionToOffset(Point aPoint,short* regionClass)
1877 {
1878   *regionClass = kTSMOutsideOfBody;
1879   return 0;
1880 }

and we need to add code inside
nsresult nsMacEventHandler::HandleTextEvent(PRUint32 textRangeCount,
nsTextRangeArray textRangeArray)
to track the geometric information

Shanjian implement the window version at bug 36975
we probably can borrow some design from it. 
see http://bugzilla.mozilla.org/attachment.cgi?id=100757&action=view

other document about this

http://developer.apple.com/techpubs/macosx/Carbon/interapplicationcomm/AppleEventManager/Apple_Event_Manager/Enumerations/kTSMOutsideOfBody.html
http://developer.apple.com/techpubs/mac/Text/Text-419.html

kTSMInsideOfBody
kTSMInsideOfActiveInputArea

I think it is enough to just return kTSMInsideOfActiveInputArea and
kTSMOutsideOfBody. I don't think we need to return kTSMInsideOfBody

also we should only support the unicode only operation. 

Comment 4

[Frank Tang]

aim chat w/ shanjian about this issue (w/ editing)

shanjianLi (3:22:53 PM): When I was working on win part, the most difficult job
is to track the geometric position of composition string.
shanjianLi (3:27:12 PM): If the event location is in the application window
(including the active input area), the input method may want to know which
character the event corresponds to, in order to locate the caret or define
highlighting.
yungfongta (3:29:41 PM): so you need to keep tracking for every input right ?
shanjianLi (3:30:21 PM): the above sentence is quoted from apple's documentation.
shanjianLi (3:31:15 PM): On window, I have a RECT array to track the position of
each character in composing string.
shanjianLi (3:32:55 PM): Whenever I got reply from nsTextEvent, I will use the
reply info to fill in this array.
shanjianLi (3:33:41 PM): if there is still information missing when the array is
being used, I will use guesswork and calculation to complete the array.
yungfongta (3:37:16 PM): so... you keep tracking the array based on each handle text
yungfongta (3:37:17 PM): event
shanjianLi (3:37:26 PM): yes.
yungfongta (3:37:32 PM): that give you the information about the longest one ,
right?
yungfongta (3:37:46 PM): and if you have something missed in between you average
it ?
shanjianLi (3:37:52 PM): Not exactly.
shanjianLi (3:38:23 PM): The reply of nsTextEvent is in fact the caret position.
shanjianLi (3:38:54 PM): each character in composing string has a RECT structure
corresponding to it,
yungfongta (3:39:18 PM): EACH character in EVERY composition string?
shanjianLi (3:39:25 PM): yes.
shanjianLi (3:41:15 PM): The mouse position is just a screen position, and IME
need to know the offset in composing string.
shanjianLi (3:42:01 PM): We can assume each character is of same width, but that
is not always true. so I don't use this assumption until I have no way to go.
shanjianLi (3:42:38 PM): In layout code, they have the geometric info, but
widget have no easy way to get it.
shanjianLi (3:43:23 PM): So I have this array of rect, each rect (top, bottom,
left, right) descript a character in composing string.
shanjianLi (3:43:51 PM): All 4 a necessary because composing text can span
across multiple line.
shanjianLi (3:44:17 PM): in the very beginning, the whole array is initialize to
unknow value.
yungfongta (3:44:33 PM): what do you mean "very beginning" StartComposition?
shanjianLi (3:44:54 PM): Yes,
shanjianLi (3:45:02 PM): before use type anything.
shanjianLi (3:45:51 PM): because composing strirng may change dynamically, no
necessarily one by one, so no assumption is made until last minute.
yungfongta (3:46:21 PM): what does "last minutes" mean?
yungfongta (3:46:26 PM): when the mouse click ?
shanjianLi (3:46:55 PM): each time we got a reply from nsTextEvent, the reply
indicates the current caret position, and we can also figure out the current offset.
shanjianLi (3:47:03 PM): (yes to your question).
yungfongta (3:47:39 PM): so each time we call handletextevnet we got one caret
position (x and y only) right ?
shanjianLi (3:49:04 PM): that's say each reply let us be able to filling
following info to the array, the right side of last characterr, most likely the
left side of next character, the top and bottom of last character, and most
likely the top and bottom of next character.
shanjianLi (3:49:37 PM): (we got caret's left, right, top, bottom).
shanjianLi (3:50:12 PM): I say most likely because next character may later
appears in next line.
yungfongta (3:50:14 PM): shoudl the caret.left == caret.rightt?
shanjianLi (3:50:44 PM): Yes,
shanjianLi (3:51:00 PM): but it does not matter.
shanjianLi (3:51:18 PM): If caret has width, we can use its mid point to mark
the boundary.
shanjianLi (3:51:40 PM): I guess that depends on os.
yungfongta (3:52:24 PM): so each time we got a reply back , we update 6 info
shanjianLi (3:53:12 PM): When mouse click even happens and this array need to be
resorted, I check the array and filling missing info through calculation.
yungfongta (3:53:30 PM): I shoudl say current(top, bottom, right) and next (top,
bottom, left)
shanjianLi (3:53:42 PM): last (top, bottom, right), possibler next(top, bottom,
left).
\yungfongta (3:54:13 PM): I don't understand - When mouse click even happens and
this array need to be resorted, I check the array and filling missing info
through calculation.
yungfongta (3:54:42 PM): what do you mean "this array need to be resorte"?
shanjianLi (3:54:43 PM): As a simple example, caret will neven be displayed in
the beginning of a line, right?
shanjianLi (3:55:01 PM): need to be referenced.
yungfongta (3:55:16 PM): how you cacluate ?
shanjianLi (3:55:34 PM): So the left side of the first character of a line will
never be filled in.
yungfongta (3:55:40 PM): I don't think that is os depend, I think that is IME depend
shanjianLi (3:55:58 PM): yes , ime dependent
shanjianLi (3:57:04 PM): here we use the assumption that all character are of
same width.
yungfongta (3:57:52 PM): is the left pos of the first character the only case
you need to "guess" ?
shanjianLi (3:57:57 PM): if we have character2, because the position of
character 2 is known, we can assume character 1 has the same width.
yungfongta (3:58:24 PM): say you type a "a" in Japanese
yungfongta (3:58:28 PM): you only have on character
yungfongta (3:58:34 PM): how can you know the left of it ?
shanjianLi (3:58:46 PM): The left position of first character in a line, not
necessarily the first character in composing string.
yungfongta (3:59:09 PM): ic
yungfongta (3:59:52 PM): when you receive the mouse click
shanjianLi (4:00:05 PM): in most cases, only when composing string span across
multiple line, we need this calculation.
yungfongta (4:00:17 PM): the "left" of a pos may not be the right one if the x-1
top != x.top , right ?
shanjianLi (4:00:39 PM): yes, exactly.
yungfongta (4:00:47 PM): I mean you need to recaculate x.left if (x-1).top != x.top
shanjianLi (4:01:00 PM): yes.
yungfongta (4:01:14 PM): but how can you know 0.left ?
shanjianLi (4:02:27 PM): That's a good question I never thought of.
shanjianLi (4:02:31 PM): let me see.
shanjianLi (4:06:14 PM): this will only happen when there is only one character
in the composing string, right?
yungfongta (4:06:22 PM): yea
shanjianLi (4:09:22 PM): in such case, it make less sense for user to position
caret inside composing string, since there is not inside to position the caret.
shanjianLi (4:14:23 PM): the composing string (the only character) will be commited.

Comment 5

[Simon Montagu :smontagu]

Per intl bug triage meeting -- we need more information on what functionality is
missing and how important this feature is, so that we can prioritize this bug.
Is it an accessibility requirement?

Comment 6

[Samir Gehani]

i18n triage team: nsbeta1-

Comment 7

[Frank Tang]

what a hack. I have not touch mozilla code for 2 years. I didn't read these bugs
for 2 years. And they are still there. Just close them as won't fix to clean up.

Comment 8

[Travis Chase]

Mass Reassign Please excuse the spam

Comment 9

[Travis Chase]

Mass Re-opening Bugs Frank Tang Closed on Wensday March 02 for no reason, all
the spam is his fault feel free to tar and feather him

Comment 10

[Travis Chase]

Reassigning Franks old bugs to Jungshik Shin for triage - Sorry for spam

Comment 11

[Jim Mathies [:jimm]]

Makoto, what do we have to do here and what's the priority? Should we try to get this fixed?

Comment 12

[Makoto Kato [:m_kato]]

(In reply to Jim Mathies [:jimm] from comment #11)
> Makoto, what do we have to do here and what's the priority? Should we try to
> get this fixed?

I think that these is already no issue for mouse support on OSX.  I can close this bug.