4826 - [IME]Need Offset To Coordinate Mapping APIs

Status: RESOLVED WONTFIX

(Core :: DOM: Editor, defect, P3)

Description

[tague]

To support input methods on the Macintosh we need APIs to emulate a flat buffer
with linear offsets in Ender.

The Macintosh Input Method protocol requires support three messages - all of
which deal with offsets into a content buffer.  The Mac IME protocol uses both
fixed offsets and relative (from the start of the
preedit/inline-input/composition area whatever your prefered term is) offsets.

Macintosh IME support requires mapping a fixed offset into it's global
coordinates and vice versa.  It also requires mapping a global coordinate in
the active input/composition area to a relative offset.

Finally we also need to be able to modify the selection and the content
model based on offsets and ranges returned from the OS.

I can explain these requirements more in detail as necessary.

Comment 1

[Simon Fraser [no longer active]]

cc:ing editor guys.

Comment 2

[kinmoz]

The TextServices module I'm writing does this to allow 3rd party features  like
spell checking, find+replace, etc to work without knowing how the internal
document is represented.

Tague, let's get together and see if the Mac IME stuff can use it.

Comment 3

[tague]

Sure.  I'd really like to avoid having multiple input paths - one for IME input
one for english typing.  I would prefer if some of the base functionliaty could
be part of the editor, but let's take a look at what you've got.

Comment 4

[bobj]

Blocker for M5 IME deliverable.

Comment 5

[Greg Kostello]

Assigning to Kin

Comment 6

[kinmoz]

Changing status to assigned.

Comment 7

[kinmoz]

Changed milestone to M6.

Comment 8

[kinmoz]

Changed target milestone to M7.

Comment 9

[tague]

This is blocking IME development.  We need a solution ASAP.  Will this be ready
for M7?

Comment 10

[buster]

moved to M8.

Comment 11

[buster]

From a recent mailnote...

-------------------
Regarding this part of your note:

     I've been looking at implementing the TSM PositionToOffset
     handler on the Macintosh...and I don't see a good way to send
     the query and get a reply back from ender or a text widget.

I'm not quite sure how the editor fits into the PositionToOffset problem.  The
editor doesn't know anything about view positions or layout.  You'll have to get
that information from layout, presemably by doing a search through the frames to
find the the frame containing the point you are given.  There must be existing
code that can be leveraged:  mouse targeting, selection, etc.

Given an offset into a frame, you can figure out which character is targeted in
the same way Mike does for selection.  That's entirely in layout, no involvement
from the editor there.

Finally, you need to map that character to an offset relative to the start of
the document as if the document were a flat text buffer, right?
You could just leverage the Text Services code Kin uses to do spell checking,
except (I think) you need the offset to be corrected for white-space
compression.  Again, white-space compression is 100% a layout phenomena so you'd
have to get that from layout.  Kin's Text Services code is (intentionally)
ignorant of white space compression.

I think the way you would do this would be to iterate the frame tree looking for
text nodes, and building your own data structure to map the character to an
absolute offset factoring in compression.  This data structure might mimic Kin's
Text Services code in some ways, maybe mapping frames to regions in a flat array
in a way that allowed you to maintain the array when content and/or frame
changes occur.  Or, it could be with IME you can make certain simplifying
assumptions like you rebuild the buffer every time an IME Begin
Composition event comes through and you know the frame info can't change until
the End Composition event (is that really true, can IME input cause wrapping
which would change the frame structure?)  Mike is your best contact for
questions regarding the white-space compression rules.
-----------
So the debate continues.  Kin and I are still too ignorant of IME to understand
how the editor is involved in mapping offsets to coordinates, given the
discussion above.  It seems like the information tague needs is entirely
layout-based.  Tague wrote in a recent email:
    "Responding to the Position2Offset apple event requires knowledge of the
     internal IME buffer, which only the editor has."
While I see that tague has put that IME buffer in the editor, I don't see how
that implies an editor API is needed to get at the info he requires.  From his
event handling code in the editor where he maintains the buffer, he has access
to layout's frame model.

Comment 12

[tague]

pasted below is the reply to the above message


---- begin paste ----
Steve :-

There are several reasons the editor is involved :-

(a) only the editor knows about an active input method editing region,
so it is the only component which can process the PositionToOffset
calculations when they fall inside that active area. This was explained
in a section of the IME document that I wrote, quoted below.

>The Position2Offset event requests a "hit test" of a particular point in the
global >screen coordinates. The application responds to
this event by constructing a AppleEvent >reply consisting of the buffer offset
and region class of that particular point.  The >region
class indicates if the point is outside the applications windows, inside one of
>the applications windows or inside the active
composition area. The offset is a fixed >(linear) offset from the beginning of
the document text buffer or a relative offset >(from the
start of the active input area) depending on the region class of the point.
>Offset2Position does the reverse mapping taking an offset
and converting it into a >global screen coordinate.  The offset is relative to
the start of the composition >region, if it exists;
otherwise, it is relative to the start of the text buffer.

(b) The only access I have to Ender is through events consumed by an
observer.  This meachnism isn't rich enough to support the functionality
that I need.  I would like to modify the observer event handlers to
contain a reply -- which of course would involve changing alot of your
handlers.

(c) Since you have already implemented most of this mechanism to support
Text Services, it would seem to make sense that you would provide the
API.

Also, as I said in the mail -- right now I'm just interested in working
on the plumbing so that I can pass some kind of event or make a call
down into ender and get a reply back.  Once that is in place, we can
work on constructing an actual reply.

/t

---- begin paste ----

Steve,Kin if you need more information about IME's please check out
http://www.mozilla.org/projects/intl/input-method-spec.html and
http://www.mozilla.org/projects/intl/input-method-spec.html#Refernces

and let me know what can be clarified or added.

Comment 13

[kinmoz]

Moving all spellchecker/text services related bugs to M11 since I will still be
working on AutoComplete features through M10.

Comment 14

[kinmoz]

Assigning this bug back to [e-mail].

The editor and international team had a meeting a couple of months ago regarding
this bug. In the meeting, we came to an agreement that tague would do the work
to make this possible, and that [e-mail] would provide him with any
editor support he needed.

Comment 15

[Frank Tang]

*** Bug 14183 has been marked as a duplicate of this bug. ***

Comment 16

[Frank Tang]

Move to M12. This is needed for Mac IME only. I will start the design and prototype now but I don't think I can make it for M11.

Comment 17

[bobj]

spelling in summary: dogfoox -> dogfood

Comment 18

[leger]

Putting on PDT+ radar.

Comment 19

[Frank Tang]

The need to have this will be reduce after I check in my Mac IME rewrite code
which always return the position in the bottom of the caret. That solution won't
be ideal but probabaly approximate closer enought to the real work. Once I check
in that code, I will take out the [dogfood][blocer]and [PDT+] flag from this
bug.

Comment 20

[Frank Tang]

remove "[PDT+]blocking IME work" and "[dogfood][Blocker] " since I add work
around for pop up candidcate window on Mac. This is still needed, but no longer
dogfood, PDT+ , or blocker. Mark this bug M17

Comment 21

[Frank Tang]

Move to M20 and Help wanted

Comment 22

[Blake Ross]

Adding helpwanted keyword to all bugs with the old HELPWANTED in their status 
whiteboard.

Comment 23

[Frank Tang]

mark this bug as future and add garywade to the cc list.

Comment 24

[Frank Tang]

IME related bug, move to yokoyama and keep it as Future.

Comment 25

[Roy Yokoyama]

Mac issue: assign to [e-mail]

Comment 26

[nhottanscp]

Reassign to ftang.

Comment 27

[Frank Tang]

reassign to yokoyama.

Comment 28

[Roy Yokoyama]

I can't believe this bug came back to me.
I think ftang wants me to learn Mac IME.  
Accepting for future.

Comment 29

[louis bennett]

classic builds are no longer supported. please re-open and move to os x if
applicable.

-> WONTFIX

Comment 30

[Torben]

What louis said.