Fill out ctypes charset conversion API




8 years ago
23 days ago


(Reporter:, Unassigned)


(Depends on: 2 bugs, Blocks: 1 bug)

Dependency tree / graph

Firefox Tracking Flags

(Not tracked)




8 years ago
Right now, for converting from ctypes.char.ptr and ctypes.char.array(), we have a readString() function that assumes UTF-8 and gives you back a JS string.

As defined in, we want readString() to take a 'charset' string arg. We then use nsIScriptableUnicodeConverter to do the magic.

We'll need ctypes to expose a JS_SetUnicodeConverter(callbacks) API, which the embeddor (i.e. Gecko) uses to pass in fnptrs to the methods on nsIScriptableUnicodeConverter, or whatever other implementation the embeddor wants. (Since we don't want to drag that implementation into jsengine.)

This is particularly important for Windows, where UTF-8 is not the customary charset.

Comment 1

8 years ago
Also need to implement cdata.writeString() here.

Comment 2

8 years ago
P1, need this for 1.9.3. (API change.)
Priority: -- → P1

Comment 3

8 years ago
-> me, but not for 2.0.
Assignee: nobody → dwitte

Comment 4

8 years ago
Reassigning to nobody. If anyone wants to work on this, feel free!
Assignee: dwitte → nobody
I will need both readString and writeString. So, I may take the bug if nobody works on it.
Assignee: nobody → dteller
My main usage scenario is for streams. With the current specifications, this is not possible, so I will need to make some changes to accommodate incomplete strings.

In my current prototype, when an encoding is supplied, |readString|
- succeeds even if only a prefix of the array/pointer can be converted;
- returns not a |string| but a |String|, with an extra field |bytes_converted|, containing, as you have guessed, the number of bytes in the prefix.

Similarly, I consider the following behavior for |writeString|
- if |length| is supplied, is >0, and not sufficient, throw a |TypeError|;
- if |length| is <=0, if this is a pointer or an array that is too short, |realloc| the buffer.
Dividing in two bugs, one for readString, one for writeString.
Plan B: now that bug 763848 has landed, we can implement this as an external C library.
Depends on: 763848
OS: Windows 7 → All
Hardware: x86 → All
I am dropping that bug. See bug 770215 for an example of implementing this feature as an external C library.
Assignee: dteller → nobody
See Also: → bug 770215
Priority: P1 → P5
You need to log in before you can comment on or make changes to this bug.